Saturday 9 February 2013

Agile software development under TOGAF

Phases G & H: the Architecture Governance Iteration

In a blog post last year, I covered how Agile and TOGAF are perfectly compatible and outlined the similarities that they have.

The title of the post won't resonate much with agilists out there, as the term 'governance' is something that most think of as 'management bureaucracy'. However, the truth of the matter is that if you look at Kent Beck's seminal work on XP and consider the principles and practises section, you'll see that these are in effect, principles and governance guidelines.What I mean by this is probably best illustrated via an example.

Imagine you are an agile QA. In order to adjudge whether or not a piece of software has been developed solidly, agile QA's often carry out the role of checking code coverage and complexity measures and monitor that coverage against metrics defined in their testing strategy. This can be automated and they instantly see a governance measure that they can check against what are effectively their organisational unit's (software development department, say) principles of TDD/BDD etc.

For software developers and architects out there, remember, TOGAF is a much broader and more abstract framework to develop enterprises under (I steer clear of just applying it to software, since it isn't technically a software development methodology. It is an EA framework, or systems development methodology if you look at it through those particular eyes). It covers business architecture, data architecture and technical architecture as well as application architecture and at the highest level, it does this without recourse to any technological concepts at all in those stages (so there are no explicit, detailed references to Visual Studio 2012, SQL Server 2012, MySql, PHP etc.). The aim is to consider the organisation as a whole as a system.

Statics and Dynamics

Any system, in whatever form, only require two high-level concepts to fully define it's operation. These are:

  • Statics - which in the case of business are things like the structures, entities, artefacts, roles, people, systems etc. etc. 
  • Dynamics - How these static elements interact and indeed how the organisation as a whole behaves.
In the world of software, we have seem this many times. For example, the GoF books were split into patterns of structure and those of behaviour and we also have languages to express those interactions, such as UML class and collaboration diagrams. These days, systems thinking is finally making inroads into the software world, despite those of us already familiar with the concepts having used it for nearly 15 years (some of you out there will have used it longer than I have).

Just like businesses as a whole, you have different levels of abstraction which define the organisation. You can define the system dynamics at varying levels of details and just like you have, say, different levels of UML diagram (use-cases to compartmentalise the operation of a software unit, with activity diagrams which define the functions within it. In themselves, these functions can be another use case diagram's use case, in the case of say, a component or package), the same is true of business. BPMN's ability to run sub tasks is another example of this. However, I don't want to get hung up on the language of expressing a business, as this is different from the business itself.

fig 1 - TOGAF ADM with different sub iterations (from Mike Walker's blog on MSDN, copyright The Open Group)
And just like a business, TOGAF allows different levels of EA for each level of the architecture partition or indeed organisational unit.

fig 2 - TOGAF ADM applied at different levels of partitioned architecture (copyright The Open Group)

How does this fit?

Well, a business is a system. Software is a system. Software development is a system. Everything in this context is just a system. Apologies to people who really hold on to the psychological element of agile methods, but you are being kept a happy cog in a big wheel :-) This is not to say it shouldn't happen, but you are there to develop software at the end of the day. Indeed, remember, you and your team use elements like Kanban to 'manufacture' software just like a they do in a factory.

AGILIST: "OK, stop flaming us! Really, how does it fit?"

Joking aside, during a software development project, you will often use BDD/TDD to develop acceptance criteria and the amount of coverage in these scenarios is your governance metric. Additionally, some agile teams are aware of risks and issues that they have to monitor. They also use metrics to define 'productivity' such as throughput and cycle time They also run retrospectives to continuously improve. Software dev peeps should pay particular attention to phase G in TOGAF gives you all that when defining a system representing your organisation. Software, Business, Data and Tech Architects may be involved in earlier phases, maybe even A to E, depending on the model of TOGAF used at each level of architecture partition. But for devs, phase G of  the ADM gives you:
  • Project name, description and objectives [Project name/Description/Epics]
  • Scope, deliverables and constraints
  • Efficacy metrics [Throughput/Cycle time/Business Value]
  • Acceptance criteria [BDD]
  • Risks and issues [Risk Log]
I have linked the Agile concepts in square brackets. As you can see, we have everything required for the initiation of an agile development project right there. 

Remember, when going through an iterative enterprise architecture process such as TOGAF, there is no BDUF exercise. Each transition goes through an iteration of the ADM, which creates an incremental change to the organisational system as a whole. Within each iteration, at phase B, businesses are architected or rearchitected and IT systems are developed to facilitate that business operation. Same as always. 

Some developers see themselves as coming into play quite late in the process (Phases G and H - last n the cycle). It seems to them that there is a BDUF exercise going on, but really the business is defining itself at the beginning of the cycle. After all, you can't have acceptance criteria defined by the business if there is no business :-)

By contrast, when you get a BDD spec for an existing, well understood business, the business system (with it's statics and dynamics) is already defined for the business owners. They have lived it and worked it for years. The business owner knows what they want and work with the 'three headed monster' (BA, QA, dev) to spec that out in Gherkin for example. In this case, there is no need to reengineer the business to then submit acceptance criteria to the devs. In TOGAF, it is exactly like going through the Architecture Definition Iteration (cycles of ADM phases B to F) with the existing formal baselines already in place, but having no EA change requirement. So you can get through those in very little time and go straight to phase G, which is where these BDD criteria form the acceptance criteria for the IT systems (I appreciate using TOGAF here is overkill, but the point is about illustrating the pattern).

Remember this...

Whilst TOGAF is an EA framework and, say, SCRUM is an Agile software development practise, they both work to define systems. Both are iterative and incremental in nature. They just work at different levels and to some degree, can have very different cycle times. After all, TOGAF would also concern itself with the definition of an organisation's services, people required to carry out enterprise functions within that service, the information (not 'data') they require to do their job, and provide services which may or may not use IT at all (think shop assistants, call centre operatives, cleaners, warehouse staff etc. etc.) and their roles have to be at least understood if not engineered into the business system as a whole. This is something IT personnel take for granted when defining software. Even in RUP they assume actors already exist, but I would say how do they think those actors came to exist? TOGAf is a way of making that happen.