One can differentiate between how precise an estimate is and how accurate it is. Precision describes the estimation range. Accuracy how close the estimate is to the actual outcome.
If I tell you I was born at 13:37 on August 12, that piece of information is very precise. On the other hand, if I say that my birthday is in January, that's less precise. The latter is more accurate though, as I was indeed born in January.
If the outcome is within the range of the estimate it can be said to be accurate.
It helps understanding the difference between accuracy and precision, and to think about if your own estimates are accurate, precise, or both. Or neither.
In this post you will learn:
Why accuracy and precision matter
Key differences between the types of estimates
Why Accuracy and Precision Matter
Imagine you are a stakeholder of a big software development project. Customers impatiently wait for the product to ship. You need to communicate something, you need answers!
The development organization, feeling the pressure, wants to help. They set out to plan the release and give you numbers. The intuitive reaction is often to analyze in detail, identify all work, and estimate it. The goal is clear: produce an accurate and precise plan.
A roadmap is developed, figures are provided. Glossy slides presented on big screens in conference rooms. Looks like they really did a rigorous job on this one! Surely this is thorough enough and the numbers trustworthy. It looks very... ambitious.
At this point warning bells 🔔 should go off and it's time to ask some important questions:
How accurate are these estimates really?
What's the teams' confidence in the plan?
The thing is that estimates are worth nothing - however precise - unless they are accurate. Accuracy always trumps precision! But the inherent complexity of software development makes it impossible to give precise and accurate estimates upfront. Not because of lack of estimation or analysis skills. Simply because complex systems cannot be predicted far in advance.
It is impossible to give precise and accurate estimates upfront. Not because of lack of estimation or analysis skills. Because of software development's inherent complex nature.
High Precision - High Accuracy
The ideal situation: something both precise and accurate. It means you can be confident the estimate is more or less correct, and will not deviate much from the predicted value. An example of a high precision-high accuracy estimate could be "The team will complete this epic in 53 ideal man days".
High precision-high accuracy estimates are great, but caution is advised. It's rarely realistic for estimates in software development to fall into this category, and it's much more likely that what you really look at is the next variant.
High Precision - Low Accuracy
Many times estimates are given disguised as high precision-high accuracy ones, but they are in reality high precision-low accuracy. These are especially devious, as they give the illusion of safety and precision, whereas in reality they are almost guaranteed to miss the target. More often than not by a lot!
High precision-low accuracy estimates are characterized by detail level data expressed in narrow ranges and precise units (for example hours), but the truth is not in the estimation range. If asked teams' usually express low confidence in estimates like this.
High precision-low accuracy estimates are toxic. They foster distrust, stress and poor quality.
High precision-low accuracy estimates are toxic. They foster distrust, stress and poor quality.
Sadly they are overrepresented in software development. I believe this happens for several reasons:
Engineers and project managers alike revel in details, and are often over confident in their ability to analyze and plan.
Stakeholders like exact numbers because they want to know exactly what they get and by when. They want to believe in the plans provided.
Non-technical stakeholders lack understanding - and willingness to learn - about software development and its inherent complex nature. (A telltale sign is when development organizations are compared with "the factory".)
But again, accuracy always trumps precision, and the trick is to do as precise estimates as possible without sacrificing accuracy. This means that you often need to settle for low precision-high accuracy estimates.
Low Precision - High Accuracy
This is typically the type of estimate you should aim for. Low precision-high accuracy estimates are characterized by ranges and appropriate units of size. An example of a low precision-high accuracy estimate could be "The team will complete this epic in 10 workweeks ± 2 weeks".
Low Precision - Low Accuracy
Sometimes both precision and accuracy is low, and an estimate is more or less pointless. For example if you switch to new and/or unproven technology, if requirements are unknown or if there are other big risks.
In this case, the only course of action is to find ways to achieve low precision-high accuracy estimates before communicating anything externally.
Do a timeboxed tracer bullet or spike to learn more.
Just start working, deliberately selecting the first items to enable more accurate estimations and to reduce risks.
Final Thoughts
Make sure you understand what type of estimates you are dealing with, and communicate accordingly. Make sure recipients get in the right state of mind, and don't put misplaced trust in low accuracy estimates.
As development organization:
Only communicate accurate data.
Time might be relevant in the end, but estimate in relative effort - and derive calendar time.
Choose appropriate units to reflect estimate precision. For example, consider using weeks or months instead of hours.
As stakeholder:
Challenge estimates that seem too good to be true (read: too precise to be accurate).
Make it clear you value accuracy over precision.
Don't push the organization to reduce estimates because of business needs. Separate business risk from estimations, and take accountability for risks you take.
In the next post I will talk about how to break down big projects and comfortably create accurate initial estimates.
Comments