It was striking to notice the important number of speech on software architecture at JAOO. Most of them - that’s interesting - placed in a natural way (sometimes implicitly) the architectural concerns in the contexts of Agile processes.
Without basically putting back into question the progress carried out thanks to agile methods, architecture was on the front of the scene. Several speakers handed forward the role of the architect. Such an emphasis rang in my opinion as a reminder of important values to beware not to leave behind.
Eric Evans was among the ones who noted this tendency. He said during the end panel that for him, “the theme of the conference had been the balancing of agile, iterative processes with design and architectural values”.
Note: The other main subject of JAOO was “how make business experts and programmers understand each other, how make them work closer together”. Domain Driven Design, Domain Specific Languages and Model Driven Development were addressed a lot. The watchword was given by Charles Simony: “Bring domain orientation at a new level”.
“Are you really sure Agile is not bad for Design?” Actually it’s quite important, because in order to be sustainable and successful, agile methods must be able to deliver software consistently and to achieve required quality… and quality problems are often the result of inappropriate software architecture! Performance, for example, dominates the user experience. Bad performance affects the perception of quality. Thus, Agile methods must be able to deal with design and they must be able to scale.
So, would architecture and design be neglected? In particular, are Agile practices detrimental to any important things and could Agile practices make us loose the focus on these important things?
These questions are not new. It is almost a custom to mitigate practices against experience or studies, hounding weakness of On-site Customer, Pair Programming, TDD, Iterations, for example. Sometimes, it sounds like gossip or gratuitous fuss. On the other side, some claims that “of course, Agile is sustainable when things are done well”. Sometimes, it sounds like autosuggestion J.
Anyway, these are healthy challenges. Philippe Kruchten, in Voyage in the Agile Memeplex: in the world of agile development, Context is key (pdf, review on infoQ), underlines that counterbalancing beliefs and considering the context of assertions (”source and applicability”) is essential to decipher messages because they are often reduced to their “essential bones”, “to the slogan level”. Perhaps the most important is to not suffering from what Kruchten calls “Agilitis”. James Coplien talks about blindness beliefs and religious resurgences (Religion’s Newfound Restraint on Progress, review on infoQ Religious Industry).
Architecture and design won’t be neglected and Agile practices are not detrimental to design 1) if we care explicitly and consciously, 2) if we consider context. A well-known drawback is to believe that design will draw by itself. Appropriate design certainly won’t emerge thanks to the faithfully adoption of any given practice.
Exclusive bottom-up approach is a pitfall…
This is a point to be aware of: Agile processes accustomed us to deliver tangible value incrementally, and to attack pieces of code in order to gradually increase the implemented functionality. And actually, it is often difficult to divide architectural problems into small pieces and solve them bottom-up. Kevlin Henney, in his speech at JAOO called “Performance Art” underlined that some concerns are difficult to understand from a modular perspective: “For example, performance is often an emerging property of the system”.
Things are even more difficult when there is no clear shared vision of the software architecture over the team.
The comment of James Coplien during his session “Scrum Architecture” created a controversy: “TDD has been proved to erode architecture”. Hard to go against orthodoxy, isn’t it!
What he essentially criticized is TDD as a design method. An exclusive Test Driven Development is not suitable for design because of its bottom-up nature. Moreover, TDD is not known to be good at finding bugs.
The goal of Unit testing must be to define system boundaries and to specify invariants (Design Driven Testing: Using tests as specifications of system’s invariants and contracts design – Sadek Drobi)
… So, maintain a big picture of the design, by leaning on reviews…
The need for maintaining a big picture and a broad goal all along the process is obvious. Modeling meetings and architecture reviews (work on Ubiquitous Language, application domain analysis, overall design) allows adapting evolutionary design and must be supported by the development process.
Mary and Tom Poppendieck propose in their book Lean Software Development, An Agile toolkit to “see the iterations as a prototype of a piece of the overall design […]. An iteration should be considered a demonstration of a possible solution”.
Cleanroom software engineering goes further and put an emphase on quality: “the quality of each increment is measured against pre-established standards to verify that the development process is proceeding acceptably. A failure to meet quality standards results in the cessation of testing for the current increment, and a return to the design phase” (Wikipedia). According to Kevlin Henney, such pre-established standards should be ratios (e.g. 90% of the database queries must be under 0.30ms).
As a consultant, Rebecca Wirfs-Brock conducts software architecture reviews. She had a session at JAOO, “lessons learned from architecture reviews”, where she gave useful advices to software architecture reviewers and architects. Her propositions should also apply during retrospectives and design meetings.
Rebecca underlined that the important is the customer value of an architecture and to be able to evaluate it. “As a reviewer, I have to be able to visualize the benefits of the architecture”. Quality can be an operational requirement and such kind of requirements have impact on design and architecture, e.g. extensibility of the system leads to develop adapters or plug-in components. In an Agile process, quality and architecture-related requirements must appear as items in the iteration backlog and must be demonstrable.
The reviews must involve right people. Even Product Marketing can help to take a technical decision.
Rebecca recalls that architecture must support unhappy parts of business Use Cases: “Propose “rainy days scenario” early: what happens when things go wrong? What should be done? It is hard to say if not addressed at the right moment.
… And seek convergence, relying on architectural models or prototypes…
Kevlin Henney also suggests using architectural models (to get a sense of what bottlenecks might exist) and partial prototypes (they provide empirical feedback). Partial prototypes support option thinking and set-based design (Lean). Set-based design is a way to reduce criticality. It leads to convergence rather than emergence. “The entire problem is divided into segments to be addressed concurrently by small teams. The design process converges on a solution, rather than selecting one at the onset” - Mary and Tom Poppendieck, Lean Software Development, An Agile toolkit.
During the end panel of JAOO, Eric Evans described a way to design the architecture of a system. You should work from realistic scenarios that architecture should support. You choose them to illustrate the hard part of what they are doing. Over time, you evaluate if you can continue to work with these concrete scenarios. This is a key to judge if your solution is “simplistic”, or if you have excessively complicated things.
Language patterns, Architectural patterns or Integration patterns (see DDD patterns, Layered Architecture, Anti corruption layer, Shared Kernel…) must support these realistic scenarios while fitting naturally the domain. There is no « one size fits all ». For Rebecca Wirfs-Brock, the goal is to achieve right architecture quality (no over nor under-engineering).
…while supporting the requirements and domain analysis…
Requirements must drive the choices of Tools and Roles that structure the design. For James Coplien, Use Cases are an essential tool to express suitably the user’s conceptual model as a whole coherent set of behaviors, responsibilities and interactions. Here, Use Cases are most efficient than XP User Stories that are more atomic and sometimes unhappily decorrelated one from each other.
Application Domain analysis must consist of a Commonality / Variability analysis (structure, meaning, state, algorithm…), identifying families and appropriate paradigm.
… without leaving operational requirements aside…
While in Agile processes we focus on delivering value to the customer, some operational requirements are difficult to make explicit. However, leaving them aside is a dangerous pitfall.
As Kevlin Henney says, “Premature optimization is the root of all evil” is too often understood… “Optimization is the root of all evil”! The danger is that, where performance matters, it is left until too late to be able to address it. So, because performance slices and crosses functionalities it is essential to evaluate performance by iteration. Fix approach is not suitable. Prototypes can help once again.
Kevlin suggested that “Where performance matters, it needs to be clearly stated as a requirement. However, although performance requirements make for good product backlog entries, they do not make for good use cases or user stories—they are cross cutting, but we still need to be able to define what “done” means ” (Definition of Done).
He recalls that testing is a way of showing that you care. It lends significance to a requirement, making it concrete and visible. Moreover, tests reveal all assumptions you made and this is essential to improve performance.
