Part Seven: In Which the Notion of Development Success as Posited in Part Five Is Examined and Some of the Precursors to the New “New Methods” Are Documented

Before prototyping, iterative, and dynamic development projects, the notion of a successful project was largely defined by its completeness. If a system was delivered to the business in the declared schedule, was within budget, and functioned according to the specification, it was a success.

In those days of what I call “static development”—when a system was described, specified, coded, tested, and then delivered—the measure of completeness was initially the correlation between the functional specification and the system as implemented—functional equivalence, if you will. In an age in which complete and detailed requirements could be expressed and documented at the start of the project, functional equivalence implied or equated to requirements equivalence.

“Requirements equivalence” is the state whereby the system delivers all the functionality the business required. In the early days of the methods camp, functional equivalence and requirements equivalence were effectively identical, but as businesses became more dynamic in both their nature and behavior and as developers started to tackle and computerize those less well understood and less well documented requirements, a divergence occurred.

This desire for functional equivalence gave us such wonders as change control warfare—whereby the developer spends huge amounts of time making sure that as the initial statement of requirements is clarified and expanded by the analyze and specify stages, those changes don’t get into the functional specification. In this way, systems developers either achieved functional equivalence and missed requirements equivalence by a great distance or achieved requirements equivalence but missed deadlines and budgets repeatedly.

A new way was needed. IT looked toward the engineering community for an answer. In particular, this problem of slowly emerging requirements was identified by the early workers in commercial artificial intelligence and knowledge-based systems. These people were tackling systems of unknown complexity, and they knew that the complexity was unknown, in fact was initially unknowable. These were the people who pioneered prototyping and what have now become known as dynamic development methods. Not only do we owe a debt to these people, but we also need to understand what they hoped to achieve with prototyping; for, as you’ll see, prototyping didn’t have the same emphasis as do the current crop of formalized dynamic methods.

These knowledge engineers used a word that has unfortunately slipped from the mainstream IT vocabulary—elicitation. They understood that they needed to draw out gradually the detail and the breadth of the problem domain as it had initially been stated. They also realized that feedback was essential to this process of elicitation. It was with these knowledge-based–systems (KBS) practitioners that the practicalities of iterative developments with scheduled deliveries and critique sessions were worked out.

Keep in mind that these knowledge engineers were among the best and most gifted among the IT staff. They knew best practice, and they knew when it wasn’t best for them and their projects. And when that was the case, they knew how to create new best practices.

• They developed practices and processes that focused on drawing out requirements and understanding both from the business and from themselves and used them throughout the project along with well-structured feedback cycles.
  
  
• They allotted design its own part in the cycle; they learned to iterate through designs and to evaluate them as requirements emerged.
  
  
• They started using timeboxes to better manage and balance the needs of the developers and the business.
  
  
• They developed relationships with the business that avoided the constant bickering and change warfare that they saw in traditional IT. They developed ways of managing KBS projects that guaranteed that by the end of a project everyone involved could see some successes. Even failure to solve a problem did not constitute the traditional type of failure—it was in fact a kind of success because of the lessons learned.
  
  
• They learned, in effect, to define and run projects that did not focus fixedly on either functional equivalence or requirements equivalence. They taught themselves how to define success in an entirely cooperative way.
  
  
• They regularly juggled timeliness, budgetary constraints, user satisfaction, and requirements equivalence. And in so doing, they filled their toolboxes with a legion of tricks, techniques, and methods from which they could assemble best practice on the fly for a multitude of project types and shapes.

When the KBS bubble burst, these pioneers went back into mainstream development and tried to take with them all that they had learned. Alas, the culture of mainstream development had changed, and they found a development community polarized between the IT department’s interest and that of the business—as if these could somehow be at odds! The IT departments they returned to were obsessed with managing the business requirements. Such practices were enshrined in their methods—central even, to IT’s concept of success. Consequently, the tools they brought back were adopted largely to wage the same war.

Timeboxing was applied to requirements gathering in order to close off scope-creep. Prototyping of functionality was used to get sign-off on the user interface. Feedback cycles were introduced to pacify business and again to get sign-off. But at heart the war still raged. The IT developers didn’t learn the lessons of cooperative development. They didn’t really learn to deal with the slow process of emerging requirements—at best, they learned to control it.

And then, of course, they formalized what they had taken—what they could see from their warped perspective—and they turned it into a variety of “new methods.” They took the flexible and made it inflexible. They took a way of working that had to do with learning, changing, refining, and adapting, and they froze it in place.

Consequently, the rapids, the prototypings, and the dynamics of the new “new methods” share the same adversarial model of development inherited from the traditional development world while seeming to wear the clothes of the cooperative development world of KBS. The fundamental problem of fixing and formalizing best practice in a fast-moving world had struck again. However, the concept of development success had changed forever, and the disparity between functional equivalence and requirements equivalence was now more widely understood.