|Book Review: Balancing Agility and Disicpline|
Balancing Agility and Discipline: A Guide for the Perplexed
Written by: Barry Boehm and Richard Turner. Boston: Addison-Wesley, 2004. 266+xxvii pages.
A subtle conflict is smoldering in the product development community. On one side are those who favor relatively formal phased methods, such as Stage-Gate® or PACE®. On the other are those who prefer dedicated colocated teams, just-in-time planning, minimal documentation, and similar techniques to operate faster and more flexibly but without a safety net.
This conflict has broken into open flames in the software development community, and I believe that we can learn from their experience.Balancing Agility and Discipline is an excellent introduction to the conflict. More important, it provides middle-ground solutions that tap the strengths of both approaches. Consequently, this is a book for those who are interested in product development methodologies, who want to align their process closer to their objectives, who wish to improve predictability or consistency, or who need to shed excess baggage. In case you are concerned, you do not have to be a computer geek to understand it.
Barry Boehm is a very senior and highly respected member of the software development community, and Richard Turner is a professor of engineering management and an author of a leading software methodology. The cover lists three forewords: one by a charismatic leader of the agile movement, one by the U.S. government official responsible for the life-critical software running the air traffic control system, and one by a self-proclaimed veteran of three software methodology wars. This sets the tone of the book: viewpoints from the agile perspective, the disciplined perspective, and the middle ground, respectively.
First, let me offer some background. Prior to about 1990, most software development applied a sequential phased model, called waterfall, similar to the phased methods usually employed in product development today. Since then, the advent of object-oriented technology has prompted a shift from a phased to an iterative style in which each of several iterations produces a working product (although normally not a salable one in early iterations). Iterative development, in turn, has spawned the possibility of agile development, although the mainline in software development today is what the authors call plan-driven development. The goal of agile development is achieving rapid customer value and responding to change, whereas the goal of plan-driven methods is predictability, stability, and high assurance. Although most of the warriors have taken sides, Boehm and Turner show us that each approach has its strengths and limitations under certain circumstances.
The Agile Manifesto (www.agilemanifesto.org) captures the distinction crisply:
Chapter 1 introduces the plan-driven and agile approaches, providing some key characteristics and examples of each. For instance, plan-driven approaches emphasize verification (work products reflect the original requirements) and validation (final product satisfies its intended mission), while agile ones emphasize simple designs based on YAGNI (you aren't going to need it), a concept of not designing anything that is not needed currently—because, with change, it may never be needed. Because product developers following a stages-and-gates approach would be familiar with the plan-driven style, I will provide examples of the perhaps more foreign agile approach.
Arguably, the most-discussed agile methodology is eXtreme Programming (XP) (Beck, 2000). Its basis is four fundamental values: communication, simplicity, feedback, and courage, and it encompasses 12 carefully selected and mutually reinforcing practices. Thus, the purists would say that you must employ them as a whole and not tamper with them, but Boehm and Turner's objective is to adapt their essence to other situations. I have already mentioned one practice—simple design and its associated YAGNI. Another is pair programming, in which two programmers sit at one computer with one keyboard and mouse, working together. This practice arose by observing the benefits of colocation and early code inspection, combining them, and, as the XPers would say, “turning the knobs up to 10.” Similarly, they noticed the feedback value of early testing, so they turned the knob up to 10 and established the practice of writing a module's test before writing its associated code.
Chapter 2 looks at the two approaches from another angle by selecting key project characteristics, such as project goals, customer relations, and product requirements (specifications). The authors discuss the strengths and weaknesses of agile and plan-driven approaches with respect to each of these characteristics. For example, close customer contact is an advantage of agile, but it can hurt if your selected customer is not representative or is flighty. Success depends on having CRACK (collaborative, representative, authorized, committed, and knowledgeable) customers to guide development. The caliber of your personnel is critical to many methodologies, both plan-driven and agile. As the authors put it, “… The unavoidable statistic [is] that 49.999 percent of the world's software developers are below average” (p. 47). Here, the book draws upon the work of agilest Alistair Cockburn (2002) to establish five levels between −1 (may have the technical skills, but unwilling to collaborate or follow shared methods) and 3 (able to revise a method—break its rules—to fit an unprecedented situation). In summary, the authors establish the “home grounds” of both agile and plan-driven approaches and formulate five critical factors that distinguish them: project size, product criticality, dynamism of the environment, personnel caliber required, and cultural adaptability. A very useful graph of these five factors on calibrated scales helps to map methodologies to the needs of a given project.
Chapter 3 captures the feeling of each approach by watching a day in the life of a project pursued by a plan-driven methodology and by an agile one. The authors also show variations for a crisis day. Their conclusion is this: “The bottom line, however, is that getting the people factors right is much more important in the end than technology or method considerations” (p. 81). Chapter 4 then moves into hybrid approaches with a case study of scaling up an agile methodology by adding some planning and another case study of applying agility to streamline a plan-driven project. Conclusions: scaling up requires a relatively scarce Cockburn Level 3 process master, but “tailoring down” is unpalatable inherently.
This brings us to a resolution of this dilemma and the creative heart of the book, chapter 5, which proposes a means of balancing agility and plan-driven approaches to suit the needs of a given project. The balancing depends on looking at the risks of swinging too far in either the plan-driven or the agile directions. For instance, consider YAGNI. In a predictable environment, YAGNI leads to rework to add the ignored feature to the design later, but in a highly fluid environment, incorporating features on speculation means being wrong much of the time, which leads not only to wasted design time but also to unnecessarily complex code that is more difficult to maintain. Thus, the ideal amount of YAGNI to apply depends on the dynamism in the environment (recall this critical factor aforementioned). Boehm and Turner make this balancing easier by applying a standard list of about a dozen generic risks divided into the categories of environment, agile (one of which is YAGNI), and plan-driven risks. To illustrate the technique, they apply it to three projects: an agile-biased one, one in between, and a plan-driven-biased one.
The last third of the book comprises five useful appendices, especially for “outsiders,” such as product developers. For instance, Appendix A offers short summaries of 13 contemporary software development methodologies, including XP, spanning a range from agile to plan-driven, and Appendix B covers the Agile Manifesto. Appendix E provides the empirical evidence and business effectiveness connected with many of the topics addressed. I suggest that you familiarize yourself with the appendices first so that you can refer to them while reading the book.
A nice feature of the book is thumbnails of each paragraph printed in the margin, which facilitates shifting between reading and scanning. The book's main shortcoming for product developers is that it uses many programming terms that would be obvious to the core audience but are undefined or do not appear in the index. Examples are COTS, UML, timeboxing, GUI, spiral model, and KSLOC. Fortunately, few of these are necessary to understanding the concepts, but a glossary would overcome an unsettling feeling for us product developers.
I highly recommend studying this book to assess whether your development process is where it should be in our dynamic world.