A Look Beyond Agile Methods and Frameworks
“As to methods there may be a million and then some, but principles are few. The man who grasps principles can successfully select his own methods. The man who tries methods, ignoring principles, is sure to have trouble.”
– Ralph Waldo Emerson
It is easy to get lost in the rabbit hole that is Agile. There are dozens, if not hundreds, of different frameworks and methodologies. What started as a declaration by software development consultants to deal with toxic work environments has since spread to other work domains and sprouted a movement towards organisational agility. It is not enough anymore for delivery teams or projects to be agile, but entire organisations are also pushing towards better adaptability and responsiveness to change. And there are frameworks for that too. With an army of consultants purveying them.
I did not learn about Agile the usual way, which is to be involved in software development. I did work in large-scale IT implementation projects for a few years, but those were anything but agile. My journey started by first studying innovation and entrepreneurship. I was trying to understand experimentation-driven development, where instead of making a plan from beginning to end, you focus on step-by-step learning and let the plan emerge as you make progress and gather empirical evidence.
As I was contrasting this seemingly more chaotic way to get things done to the project experience I had, I came upon a revelation: fundamentally, what we are talking about when we compare a planning-driven approach to a more agile and iterative, experimentation-driven one, is strategy for dealing with complexity and uncertainty.
In a traditional plan & control approach we assume that uncertainties can be resolved when making the plan. We know what we don’t know. Therefore we can make a plan for creating that missing knowledge AND (crucially!) we can predict what impact that missing knowledge will have on the rest of the plan or the project outcome. Same goes for risk management, where the assumption is that project risks can be identified at the outset and contingency plans can be created for dealing with them.
If we look beyond the agile methods and frameworks, what we are really talking about is epistemology. What can be known? How is knowledge created? What will its impact be? The real big difference that Agile brought, first to software development and then elsewhere, was not the tools, the methods, or even the culture, but the shift in thinking, mindset and the way we see the world.
Agile starts with the assumption that the (project) outcome and the way to get to that outcome are uncertain. And it’s not the “known unknowns” kind of uncertainty that is expected in a plan & control approach, but that nastier unforeseeable uncertainty, or unknown unknowns. It’s not only that we don’t know what we don’t know, but by definition we cannot know until after the fact. And this may be hard to grasp, as it was for the manager who made it to the Real Life Dilbert Moments by saying "What I need is a list of specific unknown problems we will encounter."
Everything will be affected by how you view the world: organisational strategy and structures, team practices, processes, tools, behaviours, culture... The approach you take will come down to seeing the world as inherently predictable and subject to careful planning, or as complex, unpredictable, continuously emerging and unfolding.
This is why so many organisations are having trouble adopting agile tools and methods. They try to operate them with a mindset that is firmly rooted in classical Newtonian / Cartesian deterministic worldview. I would suggest a better approach is to practice becoming more sensitive to uncertainty. Plan what can be planned and controlled, and use a different set of tools for what can’t.
Other sources and references:
Pich, M. T., Loch, C. H., & De Meyer, A. (2002). On Uncertainty, Ambiguity, and Complexity in Project Management. Management Science, Vol. 48, No. 8, 1008-1023.
Louth, Jonathon (2011). From Newton To Newtonianism: Reductionism And The Development Of The Social Sciences. Emergence: Complexity & Organization, Vol. 13, No. 4, 63-83.