Like politics, software development seems to advance according to Hegel’s dialectic.
Many see Agile as the culmination of the software engineering experience, making it the perfect process for all human endeavors even beyond the software development field for which it was created. We are probably at the second stage of the dialectic for Agile and will move on to another state, one that takes the best ideas from Agile and mixes them with other good ideas to arrive at a new synthesis.
Hegel’s dialectic consists of three stages:
a thesis, giving rise to its reaction; an antithesis, which contradicts or negates the thesis; the tension between the two being resolved by means of a synthesis.
The Agile Antithesis
Agile may be seen as a reaction to the weaknesses of Waterfall, the previously dominant management process, which is most often, yet incorrectly, associated with CMMI. It is argued that while Waterfall may work well for building bridges, where requirements are known and surprises are few, software development is, by its nature, more an act of discovery. Previous practices like CMMI are seen as heavyweight, requiring too much documentation and too many measurements.
As more people apply Agile and Scrum to more situations, its limitations become clear. Organizations often find that when they switch to Agile, their projects tend to go on forever and they frequently end up with technical “surprises.” Even when teams are able to build functionality fast, the lack of documentation can make a company beholden to the developers who built the system as only they know how it works.
CEOs of companies who have bought into the pie-in-the-sky dreams of internal Agile proponents increasingly feel bamboozled as the latest fad continues to leave their companies in the same position: suffering with unpredictable schedules and technical surprises that result from difficult and risky issues being ”put off” sprint after sprint so that teams can report their continuing progress.
Proponents of Agile built their case by pointing out high profile failures of past methodologies. However, the almost universal reaction to high profile Agile failings, like the website for the Affordable Care Act, healthcare.gov, is to say that the process must not have been implemented correctly. In a famous YouTube video, Ken Schwaber, one of the pioneers of Agile, says that if you have a poor team with bad engineering practices, Scrum will generate crap for you time after time.
The ‘Silver Bullet’
At Bleum, we see many companies that try to implement Scrum in many different ways; some successful, most not. Agile was supposed to be the answer to previous difficult-to-implement processes, but that did not seem to work out in real life. I would be surprised if five percent of our customers implement Agile the way it is “supposed” to be implemented.
If you poorly implement Agile, Waterfall, CMMI or any other process, you are bound to get poor results. In my opinion, it would be best for organizations to start there rather than looking for the next silver bullet that will magically solve all of their implementation problems.
That is not to say that they shouldn’t use Agile. While plenty of successful projects were implemented before Agile came along, including landing a man on the moon, Agile does have value when used in the right place and implemented correctly. The idea of small, productive teams breaking work up into small iterations and delivering frequently is a helpful one. The continuous build concept wasn’t invented by Agile but is often part of it and helps to dramatically reduce rework downstream.
Likewise, the control and precision that comes from a data-driven paradigm like CMMI has enormous value to help organizations improve their performance over time. Expending resources on things like documenting designs and test cases may not help a current project finish faster or better, but it is crucial where a production system is going to need to be supported and enhanced over a long time.
How to Improve Agile with CMMI
One good way to marry Agile and CMMI is to use tools to automate the collection of performance and technical metrics. Tools like Bleum’s Hydra Code Quality, Sonar, and CAST can provide a snapshot of the quality of the source code and enable some sort of apples-to-apples comparison of progress being made on quality.
Another important process step that many lose but should consider returning to is formal reviews. Reviews are quality gates that take advantage of a second set of eyes to make sure that as we build, we build on a solid foundation. Review activities can be measured in terms of whether or not they were planned, whether or not they were conducted , how much time was expended, how many issues were found, and how many defects escaped. This gives a holistic view of the quality of the reviews. Besides being a quality screen, reviews also serve as mentoring and coaching opportunities for inexperienced developers.
Agile methods are strong when the teams take ownership but tend to be better at completing functionality while being prone to inefficient architectures and a short-term mentality.
By bringing in organization level measurements around the quality and productivity of the work being done and by ensuring that reviews become formal, (i.e. documented and utilizing evolving checklists), the technical capability of all team members can be raised to a predictable level.
What are your experiences with Agile and CMMI? Have you had success combining them for your projects? Share your thoughts in the comments section below.