During the first day of GOTO Aarhus conference there was a track called
When the Agile Manifesto isn't Enough, where a bunch of speakers talked about stuff related to agile, but not converted by the
Agile Manifesto.
I didn't see the whole track, but I did saw 2 of the first 3 talks (I left during the second talk), and enjoyed Dan North's
Agile doesn't scale, and what you can do about it, and Russell Miles'
Without Simplicity there's just no Agility, both of which addressed why agile projects tend to fail at being agile over time, but in different ways. I thought I'd write a little about the first of those talks.
Dan North's talk was on how agile doesn't scale well (as the title indicates), and he started out by trying to explain what he meant by "scale", and basically he means working on bigger problems in bigger programmes. Big problems in individual projects are well handled by agile, but when you have many projects working together on bigger problems, agile doesn't help.
Simply put, the domain of agile is within the projects, but when we're talking multiple projects, we need to think about delivery assurance and governance (which put together is portfolio management).
Delivery assurance covers areas like cross-team concerns, product trade-offs, and technical trade-offs. Or put differently, covers the coordination between teams and the decisions which should involve several, if not all, teams (such as choosing the same database or even programming platform).
Governance covers areas such as organisational concerns, investment trade-offs, and portfolio balancing. In other words, this is where there requirements of the organization, as a whole, is evaluated, and decisions relating to which projects are prioritized are made.
Agile has no opinion on delivery assurance or governance, and this is why it doesn't scale. This doesn't mean that we should stop using agile, it just means that we have to recognize that we need to do something on top of agile. So agile should be done in the projects, while portfolio management should happen between/across projects.
What I've written above, is, of course, a rather simplified version of the nearly hour-long talk Dan North gave, but I think those are the important points.
Dan North's talk covered an area which I've been thinking of recently, since I've come across the problems with agile across multiple projects in the past, and I think he is absolutely spot on. I think agile is a great idea, and something which you would be stupid not to use in one form or another in your project, but I also think that the allure of agile has been harmful, since it has made a lot of people think that they could reject all classical project management, including such things as portfolio management. This is probably one of the major reasons why projects still fail when doing agile.
One of the reasons I think Microsoft is currently good at creating programming languages and tools that support them, is that they put a lot of effort into portfolio management. The different language teams coordinate on what they want to add to the languages, and they make sure that the tools (e.g. Visual Studio) are up to date, so they support the new features in meaningful ways. It obviously also helps that they have some great people involved in this, but without the coordination, the best people in the world would not be able to make something that anyone could use.
So, to sum it up, agile is useful within projects, but we must not forget that there often is a need for coordinating, or even prioritizing, between projects, and agile is inadequate for that purpose. Instead one should turn to classic project or even organizational management tasks covered by portfolio management.