Cone Head!!

A topic that seems to come up time and time again that folk seem to either take to or not, is the idea of an 'uncertainty cone'. I briefly touched on this in a previous post where I was violently disagreeing with Woody Zuil and Nick Killick, not on their principle of #NoEstimates, since the method has definite merit, but on the specifics of the merit that it has.

I'll take this time to explain a little more about the cone of uncertainty for those who are not familiar with it, or who would like to see a more practical example of what it is. To do so, let's consider 10 flips of a coin as the example. There are 2 to the power of 10 possible combinations of 10 head or tale results.

Before rolling the die a first time, I want you to guess what the final total may be. How many heads do you think you'll get?

Well, if you think about all the combination (0 heads, 1 head, 2 head...) and thus build a histogram of all results, you get this:

number of heads when flipping a coin 10 times - University of North Carolina (via Google images)

I'll come back to this later, but you should have a number in your head. Let's now consider the range of all possible numbers of heads at the end from this point. i.e. before the first flip. You can either get a minimum of 0 heads, or a maximum of 10 heads, or of course, anything in between right? Cool.

1st Flip

When you flip the coin the first time, it comes up say, tails. This does two crucial things:

1. It gives you an actual result to work with, so you now have 9 uncertain results and 1 actual result.
2. Now that you have flipped a tail, you cannot get 10 heads. Given that in the above histogram which applies all the time, there is only one scenario, that scenario is now out! The best you can hope for is 9 heads, given you've flipped 1 tail.

Drawing up the table of min and max heads after the first flip we can see:

2nd Flip

Flipping the coin the second time, it comes up say, heads. This also does two crucial things:

1. It gives you an actual result to work with, so you now have 8 uncertain results and 2 actual results.
2. Now that you have flipped a head, you cannot get 0 heads, because you have at least 1. Given that in the above histogram which applies all the time, there is only one scenario with 0 heads, that scenario is now out! Your rage is now 1 head to 9 heads.

Put this in the table and flip again. Follow the rule that if you flip a head, you increment the minimum by one, otherwise you have flipped a tail so decrement the maximum heads by one (because you now don't have enough flips to get the previous maximum).

<<Fast Forward>>

10-Flips

10-flips completed

So we've completed the whole 10 flips, incrementing the minimum if we get a head and decrementing the maximum if we get a tail. Surprise surprise, by the end, we have two ends that meet in the middle (which is correct, because by that point, we have 10 actual results and thus, no uncertainty at all). You can double check this by counting the number of heads you got, which is 4 in this case, against the meeting point of the maximum and minimum, which is 4. If you don't, then you've banjaxed your counting, so you might want to ask a 3 or 4 year old for help next time.

Making the Cone

From this table, we simply have to plot the flip number against the minimum and maximum number of heads. So let's do that. I've also included the trend lines, in black, which show the trajectory of the minimum and maximum numbers. The gap in the middle is the level of uncertainty or variance:

cone of uncertainty

Let's recap what happened. At the beginning, we had no idea where we were going to end up [with how many heads], aside from the range of 0 to 10 heads. As we progressed, we reduced the size of the range of possible 'options' or heads we could get and by the end, we were where we were.

Map this to typical IT projects. At the beginning, we have no idea where we're going to finish. As we progress and choices are made (which honestly do sometimes seem random), we reduce the total number of potential options that we have (which isn't always a bad thing, especially if we discount the highest waste or risk options) and eventually, we come to rest somewhere. Also, despite everything, we always know where we're are starting. We're starting 'here'. The end of the last cone (or part thereof).

And the First Histogram?

Returning to the histogram, which is built up from a knowledge of all possible combination of coin flip (it is a closed probability space mind you, which isn't always the case in software), you can see straight from this that the best options for your guess is 5 closely followed by 4 and 6 heads. The curve is a bell curve, aka Normal Distribution, and in this case it is fine.

Epilogue

The only real difference with development is the probabilities in software development are somewhat conditional, since the decisions we make are not random, but somewhat stochastic, or at least Bayesian, since we ourselves learn and make better decisions or become more productive, which help us descend the cone faster. It's good enough, so should still be used, but if you're a masochist, then I best at least tell you that something has recently come to my attention in the field of theoretical statistics which may be useful for the part that is currently quantified normally. That something is the Tracy-Widom distributions, which appear ever to slightly skewed to the right. It's not something I've used [yet] and it is somewhat advanced, but I am excited to see where this field goes.

#NoEstimates and Uncertain Cones

It has been a good month for alienating people... if that can ever be considered a good thing. The first was outing a chap as not really knowing what he was talking about on a certain social media platform. The second was a discussion about #NoEstimates.

I am not going to go over #NoEstimates again, as you can read my initial opinion in a post from last year. The thin slicing of stories really help with this and the concepts in #NoEstimates (which are somewhat shared by ideas like the Elephant carpaccio) show a lot of promise. However, we have to remember that this in itself won't afford continual improvement. That's the missing link.

On a very recent twitter stream, I got into an argument... sorry, passionate debate, with Woody Zuill, a long time advocate of the #NoEstimates movement. One of the things I take issue with is that the term is misleading and has not just zero, but possibly damaging connotations, on the grounds that as people misunderstood agile, they'll misunderstand this term too.

I also reiterated that estimates themselves never killed projects, it was the variance between the estimate and the actual delivery effort that killed projects. That's actually rather obvious when we step out of the somewhat siloed mindset that we developers often have (and sadly criticise others for).

fig 1 - context of discussion before moving to here

If we're aiming to step up into lean and improve the rate of these tiny deliveries to the point at which they are a true flow (if, mathematically speaking we are aiming to see what happens when 'delta-t' tends to zero) and attain consistency, we need to reduce this variance around tasks (and the flow), which is the reason that #NoEstimates often works in the first place. The truth is you're always going to be delivering thin slices, not a 'zero-effort' task and as such, stats has a huge part to play in that as the variance itself increases by over-unity as the task size increases. So to summarise the agreed points, thin slices = good. Name change #NoEstimates -> #BeyondEstimates also agree.

Effing CHEEK!

*meh*

Teams don't always know the solution or even problem they're trying to solve at the beginning of a project. This introduces huge uncertainty into projects at that point. It is the point that estimates are at their most useless and are as good as guaranteed to be wrong.

As we deliver working pieces of code into production, through small increments ideally (as they naturally have a lower individual variance) then we learn more and more about the environment we're working in, the platform were working on, the problems were trying to solve and crucially, the bit that most teams miss, things about themselves and their performance in that environment.

I mentioned the cone of uncertainty in that twitter storm. The cone of uncertainty basically maps out the levels of uncertainty in projects as they move through delivery cycles. The most obvious and visible part of this is seeing the numbers of spikes reduce (or the effort required for them), as you need less and less to understand the problems that are outstanding in the domain. Granted the cone of uncertainty did exist in ye olde world, but if you follow the cone of uncertainty through the lifetime of the project, you'll very typically see the blue line, which is also matched by the funding in the diagram:

fig 2 - image from agile-in-a-nutshell (incremental funding)

Now, this is fine. You can't do much about uncertainty in itself, aside from reducing it as quickly as possible. So where I agree with Mr Zuill et al is that delivering working software often is a good thing and slicing it up thinly is also a good idea. However, this doesn't change the level of uncertainty or that developers still need to improve to deliver the value, which remember, as an investment, includes their time and remuneration (see ROI and ROR for a financial explanation of why that's the case. It's a crucial business value metric for the product owner and developers in a team who are aiming to be trusted or self-sufficient would do well to remember that. Those who have worked LS or have run their own business will already be well aware of these ).

The comment I was probably most 'insulted' by was:

"Woody Zuill ‏@WoodyZuill  

@EtharUK ...Such as delivering early and often."

At this point I wondered whether people think I am some sort of #@&!?!

I replied explaining this goes without saying. Delivering small stories early and often allows you to descend the cone of uncertainty and plateau much faster than you otherwise would, in turn getting to the point where Little's Law kicks in and you attain predictability, which is were #NoEstimates really works but the name totally misses the point of this. Which is why I agree with Zuill's assessment that it should be called #BeyondEstimates.

If you don't know where you are or what direction on that curve you're travelling (it's possible for uncertainty to increase. Ask science) then you really aren't going to make any progress. That's what I think #NoEstimates is missing.

As mentioned, the cone of uncertainty ultimately manifests in agile as an inconsistent flow and cycle time with a significant variance (statistical definition of significance). As you deliver more of these slices faster, you start to reach significance thresholds where the use of a normal distribution starts to become viable (30+ samples ideally) and you can infer statistical significance and hence provide control limits for the team. Again, #NoEstimates works well with this, as these thin slices get you to significance faster.

Flow is a concept that features extremely heavily in queuing theory. The cycle-time and throughput effectively manifest in Little's law's equation, which when applicable, can be used to answer questions like "How much effort is involved in this *thing*". However, when you start, the variance is far too high for Little's law to work effectively or return any meaningful predictive information apart from a snapshot of were you are. There is just too much chaos.

When your flow stabilises, the chaos has eradicated and the uncertainty ratio tends to get much closer to 1, i.e. the ability to track how much effort or how long each of these thin slices takes relative to the other thin slices. It's this relative comparison that then allows the predictability to manifest (lowering the standard deviation and hence variance). This makes Evolutionary Estimation easier because you have only got to think about breaking tasks down into tiny, think slices and this is where this understanding helps this evolution. Go too small, then the overhead of build->test->deploy far outweighs the development costs and in the case of ROR, monetary uncertainty increases again. 

This is something I assume everyone who knows about agile development knows about. Given his response, I am assuming that's not the case. So I am obviously in a different place from 'them' (whoever they are). However, as well as being a good talking point amongst the capable, it also seems to manifest in amateurs who just like to code, which is the same pattern I saw when I first came across XP in 2002.

History repeating itself? I really hope not! So I second Zuill's proposal to change the name to #BeyondEstimates

Signed,


Disgruntled of the web :(