Lean and Agile Software Architecture - How Much Up Front Architecture?

I was recently asked by a colleague a question about how much 'upfront architecture' should be done if the systems development project is going to use and Agile approach. My response is below...hopefully it helps others when they struggle with answering this question.

I would say it depends (big surprise) on several factors – A lot of this comes from Lean thinking, not Agile.

In Lean Architecture (Coplien & Bjornvig) the authors say Lean is typified by a Plan-Do-Act cycle, whereas Agile is typified by a Do-Inspect-Adapt cycle. In other words, Lean = Think then Act, Agile = Act then Think. Both have their place, so knowing when and how much to do of each is important.

Here are some factors that might drive one to more thinking before acting or acting then thinking:

• Does spending time up front thinking offset by an increase in ‘fraction correct and complete’ of work (i.e., decrease rework) enough to justify the time thinking. This could be quantified with modeling, but we would need historical information in order to calibrate the model.
• How well can you predict the future? Forecasts are wrong, forecasts further into the future are more wrong. If the future is highly variable, it might lead to less up front planning and more adapting. As long as this adapting (rework!) is planned into the project schedule/budget.
• How important is it to get it right up front? In space satellites you only really test the satellite completely when you launch it, so you better get it right the first time. In pure software-as-a-service system you can write a few hours or days of code, push it into production, if it’s wrong you can rollback and try it again. I’m guessing our projects are at neither extreme.
• Projects at scale require intentional architecture; laying runway for features. A big piece of intentional architecture is defining the interface requirements. (It’s also part of requirements management). Maybe one can be lean by defining the interfaces using self documenting code (javadoc style) with stubs and example clients (so that the clients become part of the documentation, and give one a feel for the elegance or lack thereof of the API). Projects success is highly correlated with the quality and completeness of the interface requirements (including API and human-computer interface) agreed up front.
• A study of several systems projects said 5-15% of the total project budget should be spend in CD (concept design phase) and PD (preliminary design phase) to bring about the most successful outcome to the project. This 5-15% cost expenditure, depending on the project, may be up to ½ of the project schedule. (INCOSE Handbook of Systems Engineering).

This probably doesn’t answer the question, but maybe it gives some guidelines that will help you decide which way to go.