How Not To Lie About Estimates

5 steps to more responsible estimations

Estimations are hard. There’s no debate about it. From “how long will it take us to build a new product?” to “how much will it cost?”, estimations are part of the daily grind when it comes to software development. More often than not, the pressures of time, chaos, deadlines, commitments, and management dictate the opportunity and ability to provide an estimate that is honest, credible and accurate. Regardless of these pressures, one who makes an estimate is responsible and accountable for making one that is honest, credible and accurate.

The unseen responsibility

There’s a pretty good chance that you’ve made a “bad” estimate if you’ve helped develop software. This doesn’t make you a bad person, but it does make you responsible and accountable for understanding what went wrong and being self-aware of the quality of the estimates that were provided. For example, do you often overestimate, underestimate or estimate “just about right”? If so, why is that and how can it be improved? There are also some things that we cannot control such as an estimate that was transformed into a commitment. This is where how we communicate plays an important role because if we clearly understood the purpose and intent of the estimate, the way we communicated the estimate would most likely be quite different than just a number.

Our human mind fails us

We love “lying” about numbers. It is because of the cognitive bias that is built into our brains. In the context of software estimations, our human minds take mental short cuts which cause our estimates to be driven by emotion more often than credible data. Statistics Canada mentioned that “people and organisations do manipulate information for their own uses.” This can be seen in the theatre of politics where people present project deliveries sooner than the project can realistically be delivered by. The reality is that we’re not going to be able to stop the “lies”, whether it was intentional or not, but we do have control over our individual responsibility to detect and provide accurate estimates.

So, what does it mean to detect and provide accurate estimates?

5 Steps To Estimating Responsibly

1. Understand the purpose

When providing an estimate, there is a purpose. The purpose is to provide some information to quantify uncertainty so that a decision can be made. In the case of estimates, the decision-making is usually not done by the person who provides the estimate. And the person providing the estimate usually isn’t the person who needs the estimate for a decision. So in order for the provided estimate to be relevant and useful for the decision maker (eg. manager), the purpose needs to be understood. The purpose serves as the foundation and necessary context of each estimation.

Knowledge of the purpose enables one to determine the method required to produce the appropriate estimation. In turn, the decision made is impacted by the quality of the estimate. Without an understanding of the purpose, it is easy to become distracted by the noise of overwhelming information and uncertainty, making it more difficult to select the appropriate estimation method required. As a result, the estimation is no better than a guess. For example, the estimation method and rigour required for identifying a feasible investment amount of a new product is very different to the estimation method and rigour required for additional work that arose from the unknowns of an existing project. The main reason for this is that the risk and size of potential loss is much higher and larger respectively in the former than the latter case. This example also highlights a common misconception between methods and techniques. When it comes to estimations, there are many different techniques (eg. “tools” or technical skills that can be applied to achieve a desired result):

• Analogous Estimation
• Top down estimation
• Bottom-up estimation / Work Breakdown Structure (WBS)
• Three Point Estimation
• Function Point
• Delphi / Wideband Delphi Technique

On the other hand, a method is a procedure, usually including a logical, systematic or established plan, used to solve a problem. Depending on the problem and level of risk associated, we need to employ different methods fit for the purpose of our estimation.

2. Articulate assumptions

When we make estimates, assumptions are made because adequate information is often not available when we want to make a decision. When it comes to software development estimations, here are some common assumptions that we make (perhaps without even realising):

• The order of work is ideal
• Engineers/developers understand exactly what needs to be done
• Engineers/developers know the benefit of the card/work to be delivered
• No wait times or delays between tasks
• No rework
• No hand-offs
• Showcases and status updates do not go over the allocated time
• Etc.

It is worth noting that the number of assumptions is proportional to our level of uncertainty. Because of this, there is another type of assumption which we don’t often consciously recognise. They are the “unknown unknowns”. In other words, these are the assumptions that we cannot “imagine”. For example, when we have a validated idea with no requirements, there are significantly more assumptions compared to when there is a detailed design specification of the software. We can see this from the cone of uncertainty diagram below.

Cone of Uncertainty (Barry Boehm 1981, p. 311)

Thirty-seven years ago, the cone of uncertainty was a result of empirical research work by Barry Boehm. He found that no matter how hard we try, our estimations made before the requirements are defined will be +/-400%. In the beginning of a software development product, our uncertainty is at it’s highest and the only way to get an estimate to precisely reflect the actual, exact time a project will take is to just do the project. This is still true today, in 2018, and one of the reasons why other methodologies, such as Agile, Lean, Scrum, Extreme Programming (XP) and more, have emerged over the last few decades.

We make assumptions based on our experience of what goes on during the software development process and lifecycle. This is acceptable so long as we clearly communicate these assumptions to the managers and our team members so that they have an accurate understanding of what the estimate reflects. By doing this, a shared understanding is created and some discussions could arise to strengthen an estimate. After all, collective knowledge is more powerful than knowledge of a single mind.

3. Assess risk, money and time available

Before diving into this one, it’s worth clarifying that I’m referring to the risk, money and time available to complete one or more activities (eg. surveys, prototyping, etc) to collect adequate information for an estimate. Boehm referred to such activities as “buying information” to reduce our uncertainty and therefore risk. Given enough time and money, we could provide an almost perfect estimate. Unfortunately, time and money is not unlimited. This combined with the pressures, expectations and commercial viability of developing software, estimations are often made in a hurry without much thought to it’s quality or if the appropriate method was applied. It’s ironic because the intent of estimations is to quantify and reduce our uncertainty about a particular initiative or effort. In other words, estimations help us manage the risk associated with delivering successful software. So why is it not perceived important enough to ensure our estimation method and techniques are robust?

Firstly, quantifying risk is not part of the daily software development practice and therefore it is hard to “imagine” because “reality is based on perception”. Secondly, having the capabilities to produce high-quality estimate in a short timeframe is not perceived as being important but only that an estimate was provided. Although the fast-paced environments of software development could benefit a great deal from quicker and higher quality estimates. The obvious would be more accurate estimates leading to increased confidence. Recall earlier that understanding the purpose was the first suggested step to providing an appropriate estimate. The purpose is also vital for knowing the appropriate estimation method to apply given time available, money and risk factors (eg. How much money is worth investing to reduce our uncertainty around successfully delivering project X? How much confidence do we desire in our estimate given an investment of $3 million into this project?). Awareness about the level of confidence of the estimate provided is also important.

4. Communicate estimates as a range

Estimates are often communicated as a date or a single value. Whilst how we ask for the estimate is a contributing factor, we will not focus on the appropriate way to ask for an estimate here. For now, we will focus on how one should present and hence communicate an estimate.

When an engineer presents an estimate as a date or single estimate value, may give the impression that the estimate is highly accurate when it likely is not. Single value estimations also makes it easier to be transformed into a promise, whether that was intentional or not. And in doing so, the engineer has taken on the increased risk of delivering by that date or within that exact time. By definition, the engineer has taken it upon them self to provide the information and decide on when the work could be ready for launch, rather than just providing the information. It is the job of the manager to make decisions, as the developer is likely missing some information. Note that there is nothing wrong with an engineer wanting to make a commitment or promise but doing so with an exact date or timeframe is high risk. It means that one must be 99.9% confidence but the chances of this is very slim. An engineer’s main responsibility is to come up with a honest, credible and accurate estimate. To do this, they should provide an estimate as a range with that they are 90% confident about. “A reasonable probability is the only certainty.” Additionally, ranges enable an engineer to share more information such as different ranges with the respective probabilities of delivering within those different time ranges (eg. 90% chance that it can be done in 7–9 days or 95% chance that it can be done in 6–10 days). With this approach, the manager (or the “decider”) can now make truly informed decisions based on the risk that they are willing to take on. Some of the questions might include:

• Which features should we put into the software product in the first version?
• What order should we develop the features?
• When do we need to handle the communications to existing customers about the upcoming changes?

As you can see, the manager has lots to coordinate and the better the estimates provided, the better chance of not having a negative ripple effect.

5. Present duration, not the effort required

When asking an engineer for an estimate, one often receives an estimate for the effort it will take the engineer to complete the work or task. However, this estimate doesn’t account for the interruptions of an engineer’s other daily activities such as meetings, coffee breaks, ad-hoc defects and/or customer issues, etc. Without these factors, an estimate is meaningless to the manager or stakeholder because it does not reflect the “common conversation”, specifically the day-to-day realities of delivering working software to the customer. It is also not be helpful for the decisions that a product manager needs to make. This might include the following questions:

• How long before this feature is ready for marketing?
• How long before the next chunk of product development work can begin?
• Do I need additional resources to deliver within the expected timeframe?
• How long before the new customer acquisition strategy needs to be launched?
• How much could we deliver for the next version of the product?

As you can see, all the questions above relate to duration. Stakeholders, investors and managers make decisions based on the duration something takes to complete or execute as well. Hence, communicating estimates that can be understood and utilised to make business decisions is crucial. Additionally, clear communication of estimations are directly related to stakeholder, investor and manager expectations. “Since expectations are based on what they understood we said, not what we meant, failing to communicate our meaning clearly is a major risk, particularly if misunderstandings by senior stakeholders lead to unrealistic expectations are that unlikely to be fulfilled!” explains Dr Lynda Bourne.

Measure Probabilistically Over Guessing Badly

Uncertainty limits our ability to make accurate estimates. But when estimations are done right, they are useful and valuable. The intent of estimations underpins all software development efforts from the moment an idea is born to when we’re madly pumping out features to satisfy our quickly growing customer base. A careless, ignorant estimate will be the difference between a definite failure and fighting chance to a timely and successful product or project delivery.

Honest, credible and accurate estimations are a much harder task than we perceived them to be, especially if you’ve not done them before or never received formal training on how to do estimations correctly, eg. calibration training. Who knew estimation training was a thing right? It is also naive to think that estimates was just sharing an educated guess of numbers or a number from applying a popular technique (eg. planning poker or t-shirt sizes) —which have their issues but will save this for another article; in the mean time, Kiryl offers some great insights on why he stopped using planning poker. Estimations frequently get morphed into promises that were not intended and I touched on how it’s a communication problem between the person who needs the information to make a decision and the person who is providing the estimate. Therefore, it’s important for the person providing the estimate to always thoroughly understand the purpose of an estimate before they come up with the estimate. Then they should articulate the assumptions made, balance the money and time available to arrive at a credible estimate, communicate the estimate as a range with probabilities and ensure that the estimate reflects duration rather than the effort required. Using these 5 simple steps, as a guide, will help anyone create an estimate that is useful.

Quote Daily

Remember, those who provide estimates are accountable and responsible for the information they provide as it directly impacts the decision to be made. There are different estimation methods that are more suitable that others and the most suitable one needs to be applied in order to produce an honest, credible and accurate estimate. Don’t use a screwdriver when a hammer is needed;)

I’d like to leave you with one last thought. Given the definition of forecasting, why do we perceive the activity of estimating as requiring less “know-how”? When it comes to software development, what is the difference between estimations and forecasting in relation to reducing uncertainty?

Learned something? Hold that 👏 to say “thanks!” and help others find this article.

Do you want more? Follow me or read more here.