Cone Head!!

A topic that seems to come up time and time again that folk seem to either take to or not, is the idea of an 'uncertainty cone'. I briefly touched on this in a previous post where I was violently disagreeing with Woody Zuil and Nick Killick, not on their principle of #NoEstimates, since the method has definite merit, but on the specifics of the merit that it has.

I'll take this time to explain a little more about the cone of uncertainty for those who are not familiar with it, or who would like to see a more practical example of what it is. To do so, let's consider 10 flips of a coin as the example. There are 2 to the power of 10 possible combinations of 10 head or tale results.

Before rolling the die a first time, I want you to guess what the final total may be. How many heads do you think you'll get?

Well, if you think about all the combination (0 heads, 1 head, 2 head...) and thus build a histogram of all results, you get this:

number of heads when flipping a coin 10 times - University of North Carolina (via Google images)

I'll come back to this later, but you should have a number in your head. Let's now consider the range of all possible numbers of heads at the end from this point. i.e. before the first flip. You can either get a minimum of 0 heads, or a maximum of 10 heads, or of course, anything in between right? Cool.

1st Flip

When you flip the coin the first time, it comes up say, tails. This does two crucial things:

1. It gives you an actual result to work with, so you now have 9 uncertain results and 1 actual result.
2. Now that you have flipped a tail, you cannot get 10 heads. Given that in the above histogram which applies all the time, there is only one scenario, that scenario is now out! The best you can hope for is 9 heads, given you've flipped 1 tail.

Drawing up the table of min and max heads after the first flip we can see:

2nd Flip

Flipping the coin the second time, it comes up say, heads. This also does two crucial things:

1. It gives you an actual result to work with, so you now have 8 uncertain results and 2 actual results.
2. Now that you have flipped a head, you cannot get 0 heads, because you have at least 1. Given that in the above histogram which applies all the time, there is only one scenario with 0 heads, that scenario is now out! Your rage is now 1 head to 9 heads.

Put this in the table and flip again. Follow the rule that if you flip a head, you increment the minimum by one, otherwise you have flipped a tail so decrement the maximum heads by one (because you now don't have enough flips to get the previous maximum).

<<Fast Forward>>

10-Flips

10-flips completed

So we've completed the whole 10 flips, incrementing the minimum if we get a head and decrementing the maximum if we get a tail. Surprise surprise, by the end, we have two ends that meet in the middle (which is correct, because by that point, we have 10 actual results and thus, no uncertainty at all). You can double check this by counting the number of heads you got, which is 4 in this case, against the meeting point of the maximum and minimum, which is 4. If you don't, then you've banjaxed your counting, so you might want to ask a 3 or 4 year old for help next time.

Making the Cone

From this table, we simply have to plot the flip number against the minimum and maximum number of heads. So let's do that. I've also included the trend lines, in black, which show the trajectory of the minimum and maximum numbers. The gap in the middle is the level of uncertainty or variance:

cone of uncertainty

Let's recap what happened. At the beginning, we had no idea where we were going to end up [with how many heads], aside from the range of 0 to 10 heads. As we progressed, we reduced the size of the range of possible 'options' or heads we could get and by the end, we were where we were.

Map this to typical IT projects. At the beginning, we have no idea where we're going to finish. As we progress and choices are made (which honestly do sometimes seem random), we reduce the total number of potential options that we have (which isn't always a bad thing, especially if we discount the highest waste or risk options) and eventually, we come to rest somewhere. Also, despite everything, we always know where we're are starting. We're starting 'here'. The end of the last cone (or part thereof).

And the First Histogram?

Returning to the histogram, which is built up from a knowledge of all possible combination of coin flip (it is a closed probability space mind you, which isn't always the case in software), you can see straight from this that the best options for your guess is 5 closely followed by 4 and 6 heads. The curve is a bell curve, aka Normal Distribution, and in this case it is fine.

Epilogue

The only real difference with development is the probabilities in software development are somewhat conditional, since the decisions we make are not random, but somewhat stochastic, or at least Bayesian, since we ourselves learn and make better decisions or become more productive, which help us descend the cone faster. It's good enough, so should still be used, but if you're a masochist, then I best at least tell you that something has recently come to my attention in the field of theoretical statistics which may be useful for the part that is currently quantified normally. That something is the Tracy-Widom distributions, which appear ever to slightly skewed to the right. It's not something I've used [yet] and it is somewhat advanced, but I am excited to see where this field goes.

Evolving & Emerging Architecture: Agile-EA

Going into companies is always an interesting experience. You get to view the way they work and in agile organisations using physical boards, you can walk the floor and view how the work is progressing. That is well known in the agile project management arena, product owner roles etc. but it's also an extremely useful technique in the Agile Enterprise Architecture world (Agile-EA). Before going into why, we need to recap a couple of definitions.

1. Conway's Law

In 1968, Melvin Conway provided this now tried and tested gem.

"organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations"

This is a huge revelation for some companies, but it got hidden by process oriented techniques, which masked the problem for decades.  

In the agile space, it's a very different story. The reliance on teams to develop their own processes, and having the greatest expressive power within them as opposed to across them has resulted in this being more applicable than ever. You can turn this to your advantage and having elements of a workflow which make up the greatest proportion of it, contained within teams really improves the rate of delivery of work, as I have covered in previous posts and is a much better facilitator of outcome than having a distributed development team. This also means that cross-team communication encourages the development of a system through that communication channel.

2. Consumer Driven Contracts

A consumer driven contract can be considered a Kanban pull signal. This pulls a feature from another team and in software development spaces, this also includes a sub-suite of tests from the consuming team. When a feature provided by another team is needed, a cross-team card, often of a different colour is placed on the team board who are to provide that functionality (at Laterooms,com for example, they used rainbow coloured cards at one point). That team then pick the card up and code against the tests to make them pass. 

Consumer driven contracts are pretty much the most central concept in true just in time, cross team software delivery. They often have a subtask link to the source board and effectively provide an architectural link between application components or business features.

This provides opportunities for architects in both the enterprise and/or solution sphere to see how their estate is evolving by simply going to all stand-up meetings, and building a picture of the emerging architecture or estate by following the chain of tickets. Once such a ticket appears, this introduces an architectural dependency and over time, more and more tickets appears which fill the space between things that architecture lives in.

Walking the Floor

Architects live between things. As architects we should aim to get to every stand-up within our sphere. When we do, we should look out for these cross team tickets, or features being played which we are aware touch something else or perhaps could leverage a feature elsewhere in our estate. 

Imagine we go to the stand-ups and see the following boards after, iteration/sprint 10. 

Sprint 10 board states

Sprint 11 sees a purpose ticket appear on team board 1, required by team board 2 (who are pulling it). The purple ticket represents a cross-team 'pull' (indicated in team 2's 'X' ticket for convenience) since it requires a feature from team 2. The dotted line now represents the relationship between the two teams, which remember by Conway's law represents a link between the system or business features:

Sprint 11 emerges an architectural link

This continues to be played and supposing by the end of Sprint 11 it and its compatriot pull are both done. This ticket is now complete and the architectural link has been delivered. So the architecture of the estate now looks like the following (Arrowed lines represent a dependency):

Sprint 11 Architecture - Note, no Team 4 features live, so architect doesn't care

Sprint 12 and two purple tickets appear. One from team 3 to 1 and one 4 to 2. The solid line represents the delivered link from Sprint 11:

Sprint 12 Architecture

When this is delivered, the architecture then looks like:

Architecture after Sprint 12

And so the cycle continues. 

Conclusion

Each one of the dotted lines represents a communication flow between two teams and brings them closer for that phase of the development. When it's done, the link still exists in the system (solid line), but they can separate and form relationships with other teams, as team 1 did with team 3, after team 1 finished with team 2. Of course, there are times when the platform will need to delete those links, removing them from the estate, but the communication still happens regardless. 

As an architect, attending the stand-ups is a great way to see how the system evolves. You can walk the boards looking for these special cards and draw out the architecture as you proceed. Indeed, if you're an active contributor (and you should be) you can and should facilitate the teams to make decisions which are systemically optimal and forming links when the team could otherwise expend a lot of effort would save them and the company time and money. As the organisation matures and the role of software architect embeds in the teams (as roles over job descriptions), it is these individuals who will go to other team stand-ups and hence, facilitate the creation, updating or destruction of these architectural links.