Lowering Chances, Mitigating Risks or Both?

I was talking at Lean-Agile Manchester this week. It was a choc-full event which necessitated the adoption of extra chairs.

A number of the XP Manchester folk were in, which is always entertaining, since the two groups have overlapping common interests but as with many agile vs lean schools, we don't necessarily come to an agreement on the best way forward for things.

There were some great questions through the night! Including the ones form the hecklers. It centred around data from some graphs I showed from a previous blog post tried not to go into the maths of due to the typical spread of the audience. So I offered to take it offline so as not to bore the audience, but there wasn't the appetite form the questioner, so smackdown happened and they then agreed to take it offline but never got back to me, darn it! (#invitestillopen)

Background

What's the reason for the graphs?

Several years ago, I was working in a company which was on the proverbial agile journey. They were still thinking in very big-design ways and were managing programmes of work through standard programme and project management methods. The company's attempt to have conversations around agile programming were not really working and the second attempt at them (i.e. just do the work and they will come) didn't reach far enough for anyone in positions of enough power to take the effort seriously. This resulted in a somewhat disconnected hybrid method which saw lower levels doing the work with upper levels of management and EA imposing design on the teams, with PMs backing up the EAs as authority on that work.

In addition to that, teams spent the vast majority of retrospective time generating new ideas for working together (good, bad, change) including grouping tasks, voting and setting options for the next iteration. However, no retrospective ever came back to check that these did indeed improve the process and any overhead we introduced as part of the each task was actually worth it. Further actions just built on top of these actions and you gradually built up greater overhead in each iteration.

The team had successfully implemented WIP limits (though that started off quite painfully) and were measuring cycle time and throughput since this was easy for them to visualise in a JIRA Dashboard. We saw a burn down but it wasn't clear whether our flow was any good and indeed, whether we were improving at all.

Add to this the need from classical project management to get an idea of the length of time things would take as well programme management to align the streams of work meant we had to get to know something about whether we can actually hit the hard deadline. Those that know me know I think aligning work the SAFe way or classical PERT way introduces inherent risks, but the environment was what it was and each change begins with a small step, not a 'Big-Destroy Enterprise Programme'. After all, as a dev, you're an easy replacement anyway to that style of culture (not that you necessarily have to worry about it in the IT game but it's an important consideration).

Who wanted it?

The graph/points estimation wasn't necessarily to get the team to improve delivery per se. That was not the purpose of the exercise. It was to give confidence that when we were challenged to produce an estimate, we could do so reliably and provide some confidence to the supporting classical thinking personnel we're talking to that we can and have delivered x features in t. It was to lower the variation and give confidence to those who wanted to support us that we could deliver and were improving. This was a tool to help them do that and get the buy in they needed, which took half an hour a week for someone to do (indeed, I did it - but any scrum-master or tech lead can do it in an enterprise context).

Why should you care?

The answer depends on the context you work in. In an agile-sympathetic environment, this isn't really necessary at all. After all, everyone is confidence and comfortable with change. However, where a hybrid exist or companies are transitioning, sometimes these conversations are necessary. Later on, they may not be relevant any more. Enterprises can evolve as much as people do.

The Follow-up Questions

During the talk, some questions were asked and I agreed to produce some follow-up graphs from the data. In order to understand some parts of this, I'd suggest you go back and read the method presented in that blog post, as this will explain what look like 2-pt and 5-p story 'anomalies' as we shifted our understanding of story sizes.

Cone of Uncertainty - Variation Over-time

Specifically, taking the variation between our expectation and actual delivery, plotting it and calculating the Coefficient of Variation to standardise the scales of the graphs, we can plot the change in the coefficient over time. What we see for each story size (in points) is this:

Story point variation (CV) and polynomial trend line

To keep things simple(r), I've added a cubic polynomial trend line to illustrate a smoothed variation. I haven't done anything else to the trend line and Excel has chosen the shape that minimises the sum of squares. We can relate actual uncertainty to the variation in story point figures. The same downward trend on variation is seen in linear and logarithmic trend lines. As you can see, most trends show the reduction in uncertainty as we recalibrate our positions.

Limitations

The only exception to the general trends are the 8-pt story sizes, which curve slightly upwards (not significantly enough over linear to be concerned about). Additionally, due to the team rightly reducing larger 13 point stories into smaller stories, there are only a few 13 points stories in the dataset. I argued there were not enough to come to a conclusion or indeed worry about going forward, especially most became 8-point stories as a natural part of story splitting and recalibration (again, read the previous blog post).

Conclusion

As I explained in the talk the other day, estimation such as this isn't an end goal. This is a technique in the repertoire to provide confidence for those who can support us to become more agile. After all. working in the Enterprise Architecture space necessitates communicating in many different companies, with many different types of stakeholder, including non-technical personnel/those without a software development background. Not ever EA problem is a software development problem. Indeed, to approach it from that perspective architects before it's necessary, if it needs it at all!

Digression

As an example, consider walking skeletons, which can be just as problematic in code, since they make explicit choices on the technology stack way before a decisions is needed on the suitability or otherwise of the tech, but they are useful tools to experiment when you have a tech stack already and gain certainty. However, employing just a walking skeleton is like having Maslow's Hammer. It risks introducing technology into a non-existent current stack when the basics of what people want are unknown. In this case, you don't need a skeleton per se. Just throw together a UI mock up and deploy that to a static environment (even a file system) to get people using it to input data that never gets stored. This can be done in a few minutes compared to creating a walking skeleton which can take a couple of hours to get the same amount of feedback and can be potentially constrained by infrastructure problems and will require some prerequisite work. So bang for buck, if the question is trying to find out of Henry Ford's customers wanted faster horses, this would be cheaper to do than a walking skeleton and yields just as much value. The second meeting can fill this out with a skeleton if you want, since by this point you have more information to base choices on.

Risk and Sensitivity

You have two non-mutually exclusive choices to deal with risk. The first is to reduce the chance of it occurring, which this technique fits into. The other is to mitigate the impact should the risk occur. Which this doesn't address and isn't intended to. So this can only be one of many tools in the team's arsenal in dealing with tracking, recalibration and risk reduction and as we can see, there are specific scenarios this addresses really well. The question is, what other techniques exist to address the same problem?

Further Updates

I will answer some of the other questions in time and post them as updates to this blog.

The Agile Manifesto: Ten Years on (A critics view)

The agile manifesto has been around for coming up to 11 years. Most of the time proponents of agile methods always espouse its capabilities and why people should value agile people over process and always hark back to the 12 principles that underpin the agile manifesto.

However, through some of the interactions I had with one or two of the founders of said manifesto back in 2002/3, the leading agile method at the time (eXtreme Programming) was often compared to waterfall like method or other processes and the reasons given for their deficiencies were normally viewed through failures I the development process, when it was often the case that it was the failure to implement them correctly that caused issues.

Additionally, after working for a few companies which have used agile methods quite extensively and rigorously, I have come to notice a number of problems with some of the agile philosophy and indeed the way some of the more ‘formal’ agile approaches also exhibit some of the very same deficiencies that agile filibusters accused the more process oriented sectors of the industry of being (note, by ‘formal’, I mean those agile methods which have defined methods associated with them. Note, even XP had a method, as simple as it was, ‘User story->Test first->Develop…’)

Now, this is not meant as a blasting of agile methods. Agile methods have brought more structured ‘methods’ to an industry which has traditionally shunned any attempts to introduce engineering rigor. It is as if agile methods allow a ‘bridge’ into a world most so called developers find is usually above their heads. Plus, some of the concrete tools and techniques made famous by agile methods are good regardless of methodology (automated unit testing, continuous builds and deployment, QA processes and the like). However, there are a number of problems which have come about and I shall cite these in the 12 principles:

1. Satisfy the customer by rapid delivery of useful software.

This one is OK as a principle. It tends to lend itself to proto-typing methods with refinement (such as the ‘walking skeleton’ approach that some developers advocate). However, as has been found in other RAD methods, the customer will often want the prototype, resulting in code that was never production ready seeping into the codebase which will need refactoring later on as a technical debt exercise. Often, this tech debt is never repaid and if it affects the software architecture of an organisation, then there are problems, as other systems will couple themselves to that set of inadequate interfaces, requiring greater work later on.

2. Welcome changing requirements, even late in development.

This one is more of an issue. Being facetious, you can’t develop functionality in zero time, however agile you are. Indeed, the idea of ‘failing fast and early’ is completely counter to this point, as if an unforeseen failure causes a change in requirements is noted earlier, the extra time that comes about by gaining this ‘derived requirement’ earlier in the process is designed to mitigate risk. So I still maintain the longer it takes some requirements to come about, the less welcome it is.

3. Working software is delivered frequently (weeks rather than months)

This one is a holy grail, so I have no issue with it at all, as long as the quality is maintained throughout the process. If quality fails, the whole software becomes inadequate given often unseen or otherwise unknown requirements to the business (a technical debt repayment exercise that other steps may have introduced, is a prime example)

4. Working software is the principal measure of progress

From a business value standpoint, this is correct. However, putting my architecture hat on for a second (a hat that some XP exponents believe shouldn’t exist in the development process), the phrase ‘working software’ states nothing about ‘quality software’. So this often introduces inefficient software systems from a non-functional requirement standpoint.

Basically, all the quality metrics surrounding real enterprise and large scale systems are shunned for the preferred end-goal of addressing current functional requirements alone. QA and performance testing is a relatively recent adoption into agile developments (when it has existed in other methods for decades). As a result, elements of performance, security, maintainability and the like are almost never addressed in older agile minds. This has bedded in as a preferred method of working in most companies, as this is the way developers (who are really just programmers) like to work. Cut code that does something. The industry is pre-disposed to preferring this method as it is the principle they already hold and is the path of least resistance.

For some examples, imagine going to your bank ATM, knowing you can always withdraw money from your account because it works, but also knowing that the non-functional requirement of security is not high and everyone else can withdraw money from your bank account too. Not exactly confidence building.

Using an example closer to developers’ homes, if you develop highly coupled point to point enterprise systems and your tech debt is the refactoring and the refactoring is never done because you are only to deliver working software, then the coupling to other systems may explode exponentially, meaning addressing that technical debt becomes exponentially more costly and time consuming (especially when the interfaces for these connected elements are all different).

5. Sustainable development, able to maintain a constant pace

This is an ideal principle in heavier engineering methods anyway, so no arguments here. It allows for predictable timings for people who have to report and plan developments in the business. It is also particularly well suited to having Kanban and lean processes applied to it and optimising using those tools. So no arguments :-)

6. Close, daily co-operation between business people and developers

I like the idea in theory. However, this one is often even more idealistic than the agile proponents’ accusation that business people don’t read specifications and documents in heavyweight methods. In reality, companies can’t afford to hand over staff or there are political reasons why it can’t happen. Without a document for these decisions, agile methods can’t work at all.

Consider an organisation who buys into the agile process. They instruct the manager of a department to send a representative stakeholder to the agile team to sit with them for at least half a day every day. The wages of this individual are being paid out of the business budget and not IT. Whilst they are with IT, they lose the business department 4 or 5 times their salary in work (ether by loss of revenue or the combination of temporary cover, overtime and loss of productivity).

IT may also be paid out of the business budget (where cross invoicing occurs) and so the department are paying for their system and losing the revenue generated by that individual. This can be a process which sees departments lose so much that recouping the cost of the system would be difficult in the short term, agile or not and that is compounded by non-committal methods of agile development. It could be cheaper to pay for the overtime of the individual, get the individual trained in how to understand and write success criteria and have them send a story document, with success criteria that has been reviewed back to the IT team.

This one, in some companies, is a huge time-bomb waiting to happen. The only thing that has saved such methods in such organisations, is that no one has actually noticed that the cost to their department of £10,000 in wages paid, £40,000 of no revenue generated by that employee, plus the cost of the system on a cross departmental charge of, say, £50,000 far outweighs the £60,000 saving that the department will recoup on the existing setup.

Invoices such as the ones presented in department-department charging only show the cost of the development work, not the lost cost of the employee being relocated to the IT department (that is not a development departmental concern, so why would it appear on the invoice). This information is not immediately available and can sometimes only manifest itself in the departmental accounts sometime in the future.

A savvy business manager will figure this out. As a result, will they send a supervisory/managerial level employee if their department is fully utilised? I doubt it. They will send an underling, given them a responsible role and ship the closer to minimum wage employee to the IT department for the duration. It has the benefits to the business of:

• It costs the department little
• Their loss of the employee for a while doesn’t have a knock on effect on a large amount of other employees (contrast this with sending a manager)
• They see it as providing the best knowledge from the individual who is closest to the system

All this may increase the value for money that the business department can provide in the eyes of the departmental manager. The only issue is this may be interpreted as not being ‘bought in’ to the idea of agile development.

7. Face-to-face conversation is the best form of communication (co-location)

This is a doosie! Mainly because it is categorically false depending on your definition. There is no psychological basis to this at all. I have a few friends who are in the field of psychology and the figures for written and aural/oral communication are abysmal. It averages out at between 5 and 10% of what you read being accurately portrayed and you remember about 10% of what you hear. So you can see there is not much in it at all.

The benefit of the written communication however, especially if you pull in elements from NLP, is that if you can present the deliverable documents in the format most suited to the mindset of the intended recipients, then you have a consistent document, that a stakeholder (which includes a developer) understands fully, can refer to time and time again and get the same consistent answer. If someone has a conversation with someone else, something might be missed (especially if it is not first-hand), the quality of communication is generally very low and more often than not, highly inconsistent.

For an example of miscommunication, consider saying the word ‘services’. What do the following people think?

• Software developer
• Enterprise Architect
• Bank Manager
• Regional Manager of a car repair chain
• Hotel Manager
• Restaurant Manager

More and more modern methods out there acknowledge the importance of documenting the domain specific knowledge and terms in some glossary. So much so methods all the way up to TOGAF have formally included requirements for these business specific definitions in their standards. So even with the advent of agile methods, I am very glad to say that other areas of the field still maintain the use of a lot of the necessary documents.

This is categorically the worst manifesto principle. Writing documents last is fine for the development team (and they don’t want to write it anyway), but is no use if that document set is needed by two separate teams (such as how architectural work is delivered).

Also, I don’t see the sense in say, two teams developing code, then refactoring it and writing up their documentation (which never happens in practise) only to realise they have the same structures in it. That is a waste of development effort and run completely contrary to the lean principles modern agilists espouse.

Additionally, what happens if a developer leaves? This has two problems.

• The first is the loss of that tribal knowledge in the developers head
• The second, is that even if the knowledge is not lost, it requires a transfer to the new individual. If this means another developer sitting down with them and transferring that tribal knowledge, then this loses the seasoned developer’s productivity for that time.

If a diagram, which taken an hour to draw can save you 20 hours of refactoring, plus it mitigates against the risk of someone leaving, then I can’t justify the reliance on refactoring work and knowledge transfer, sorry.

Additionally, how many developers go to talk to the CIO directly and explain the project to them in terms they understand? Sometimes such communications can't happen, so a series of intermediaries need to summarise relay that information 'upwards'.

8. Projects are built around motivated individuals, who should be trusted

In an ideal world yes, people should enjoy what they do and desire to do a good job.

However, in reality, there are a few things wrong with this. This includes that developers don’t know every skill! Also, they may not have domain knowledge and trusting the team to be able to do a good job is impossible if they are not, say, mathematically minded when working for a finance organisation (you won’t believe the numbers on that one, much to my dismay), or have no idea how to optimise code/database queries.

I tend to sometimes chuckle to myself about the irony of developers ‘who should be trusted’, then getting told to do agile methods because they don’t trust that developers can produce good code by themselves.

9. Continuous attention to technical excellence and good design

You can’t do design! That is documenting up front :-D

Joking aside (because you have whiteboards and cameras don’t you?), technical excellence takes many forms and is not just limited to cutting code.

However, becoming ‘technically excellent’ in one domain often means you cannot transfer into another domain without a significant amount of effort. A more generalist approach would help with that transition, especially if the organisation is struggling to recruit into particular roles. The supply-demand curves for devs, QAs and BAs is never the same in the industry all the time every time. Having a more generalist approach allows for an appreciation of the other roles in the team (which fosters better communication) and allows the vacancies to be filled easier when supply is higher in one domain than another. I can certainly advocate ‘Role based’ responsibilities being fostered, but detailed, specialist jobs I have a problem with from both a recruitment and intra-communication perspective.

Now, I appreciate the agile manifesto says nothing about the above. However, I have met some people who take the above to mean that people will become excellent and dedicated to their sub-field alone, which is fine, but the worry is that they become so specialist, that the ability to bridge the communication gap to their team-mates is severely diminished and given we have no documents, quality of development dynamics will suffer.

10. Simplicity

I don’t have a problem with this in theory. I have a problem with the way it has been applied in industry.

Simplicity often gets tagged by the KISS principle (Keep It Simple Stupid). However, there is a much better rule used in almost every other engineering discipline and that is Occam’s Razor.

Basically it infers “All things being equal, pick the simplest thing that works”

Pay particular attention to the words at the beginning of that statement, “All things being equal” and let’s take a look at a classic example you can probably remember from various courses you may have done over the years (especially in higher education), the requirement to sort a list of items using bubble-sort and quicksort.

Functionally, both bubble and quicksort will sort numbers into the right order. So they are both equal in that respect. KISSers will then state “OK, I am going to use bubble-sort”.

So they make it happen. The list then eventually contains over 1,000 items. It slows to a crawl! They then have to revisit this decision, identify new options and re-implement the solution, yielding more waste. Why was that? [rhetorical question].

A RAZORite will have asked questions such as:

• What is the algorithmic complexity of each solution?
• How many exchanges do I need to do?
• How much memory is available to me?
• How well will this scale when more items are added?
• What is the performance profile of this algorithm?

When they have found that all things were NOT actually equal, they would have chosen an appropriate solution out of the sorting repertoire immediately. It is the old trade-off analysis again.

In general, I have never seen an agile method deliver well optimised, performant code first time or even explicitly consider it in any part of the initial design. Fast teams do no thinking (and Katas etc. foster that mind-set, encouraging the developer into autopilot) and scoff at attempts to get them to do so. I tend to safely story my ‘I told you so!’ cards for such occasions when the non-functional requirements have not been implemented or thought about from the offset (authentication and authorization being a prime example).

11. Self-organizing teams

This is fine. No autocratic dictate of who should do what when. The team decides its own protocol.

The difficulty comes when you scale it upwards. The bigger the team, the less organised it becomes. Also, in order to foster increasing levels of self-organisation, the team have to develop the foundations to support agility in ever increasing teams. This means some of the foundations will have to be set and this can include documentation, metrics, Kanban strategy and the like.

However, in reality, teams stick too rigidly to the mistaken belief that agile methods mean none of that. The manifesto only specified there is a ‘preference’ for one thing over another. The misinterpretation of this as ‘don’t document’, when it actually says “working software over comprehensive documentation” (Which I joke is how it is read by KISSers and not RAZORites, see above ;-) tends to mean agile teams lose the knowledge needed to be communicated (intra or inter-team) either over time or through too quick a team change.

In order to facilitate team decision making, metrics are an important part of the toolkit. Just like a CEO needs the business intelligence to support their decision making, an agile team need the metrics to be able to plan, optimise and ultimately, become predictable.

I remember sending an e-mail to someone at a client of mine stating that there should already have been a collection of some metrics (story points) and that they had a calendar of events that happened (including things like holidays, staff leave, high priority ‘red flag’ issues etc.) and they should correlate these events to the effects they had on the metrics. This can been done by using a Taguchi methods or statistical factor analysis to find the actual relevant variables affecting the velocity. This is a standard method in quality engineering in manufacturing industries and seeing that Kanban is routed there too, it is a natural transition to accept this in the software world.

12. Regular adaptation to changing circumstances

This is good. I don’t mind it within projects at all and it is a holy grail.

However, if an organisation is shifting priorities, then the decisions on delivery of software is somewhat taken out of the hands of the developers. Note, they will always be delivering software, it is continuous integration after all, but if the software is not complete to deliver the value at every stage, for example, delivering a story as a release or iteration, which only covers 60% of the value in the prioritised release backlog, which doesn’t deliver the total (or even sufficient) business value before losing business priority to a much more important project, such as a change in the regulatory landscape (which requires prompt action), then as well as delivering a value only equivalent to what has been realised so far, if the project is a migration one, you may have part migrated the project, resulting in a split of functional concerns at an architectural level.

It is often hard to find that magical 80% point suggested by Pareto’s in some projects and even harder to implement them in an every changing landscape if you don’t have tools and techniques to support it and metrics are a vital way of doing that.

From an agile perspective, a technical debt exercise of that magnitude is incredible difficult to put right later if other systems have come into play on those connectors separately later on. Trying to fish out the new systems tacked on to old ones and the others which were tacked on to the new is a difficult exercise, that costs more at the time that you do it (because there is more in place to fish out). It requires governance which is somewhat at a tangent with agile philosophy (people have to be trusted and people over documentation) or at best, is at a much higher organisational level.

Summary

Agile methods work best in environments where there used to be chaos. Where established structured method exist and work for the organisation, purely agile methods are a retrograde step. However, in reality, there is very little in the way of adequate structured methods in organisations.

Through time, we have seen agile methods meshed with other methods (‘agile’-‘waterfall’-‘agile’), placed as part of a bigger strategy (such as TOGAF-Agile) or agile teams seamed together (Scrum-of-Scrums). I note that Agile is never compared to RUP directly, which delivers value in the form of use cases or groups of use cases in iterative cycles. Anyway, these seem to be the small steps that are needed on the journey to becoming agile and indeed, some organisations choose to stay in some of those intermediate states.

Agile methods are not scientifically proven. I still maintain that it predisposes itself to developers who are really just programmers, but again, this is not always a bad thing. Indeed, that vast majority, some 80% to 90% of industrial developers here in the UK are not engineers, nor have they an engineering mind-set. It is all about the functional elements of the code within the component boundary. Architecture is not their concern, and non-functional requirements they consider secondary. So to introduce some quality in the functional aspects, where previously there was none is a good thing. But it is certainly not a catch all and should not be a total substitute for a working structured practise in isolation, at least immediately.