Why Estimations Are Regularly Off

Whatever they say, estimating is an important skill for both work and everyday life. From time to time, finding exact answers and providing precise info happen to be quite impossible whereas quality estimation can help you come up with the closest to real figures.

While handling a certain issue, it is always useful to look back at your initial estimates to analyze and see if you are on the right track. As long as you have an estimate at hand, you’ll always have an understanding of what is still left to do.

However, estimation is often tricky and not as simple as it seems. Let me first give you a great life example. I’d like to refer to Michael Wolfe and his answer to one of the questions on Quora.

In the example, he describes a hike on the coast from San Francisco -> Los Angeles to visit friends in Newport Beach. The route seems about 400 miles long; if walking 4 miles/hr for 10 hours/day, it’s possible to reach the destination in 10 days. It sounds logical to call the friends and book a dinner for next Sunday night after you roll in triumphantly at 6 p.m that day. Estimation rocks, right?

Next, the author starts mentioning a variety of things and everyday events that are constantly moving the deadline (the arrival date) further during the trip: twists and turns on the coast, sand that is difficult to walk on, stairs, angry sea lions, an inaccurate map, oversleeping, blisters, headaches, and fever, as well as frustration and fatigue. As a result, you get to realize that an initially estimated 10-day journey actually turns into a 60-day disaster.

This is a very good example of ill-thought assessments, overconfidence, the interdependence of circumstances, and simple negligence.

Case study

Imagine that you have a set of prioritized tasks to complete. However, every day you face some urgent unplanned tasks to be done right away and interfering with accomplishing the major goals.

What to do?

You can carelessly accept new tasks, switch between them non-stop, and at the end of the day be horrified and overwhelmed by your giant backlog. It will result in numerous missed deadlines, no doubt.

Alternatively, you can stick to your initial plan & refuse to do anything else saying “I already have things to do”.

Would you call both these attitudes professional? Sure enough, you can’t win in either of the scenarios. Instead, you can select a work-oriented approach.

Evaluate your current workload & fit the most urgent tasks into your plan

Always stay aware of your current workload and never schedule your tasks to be completed back-to-back. When estimating time for getting things done, remember that there should always be room for unforeseen circumstances, and in such a case you will be ahead of the game and keep up the pace.

Negotiate with your lead manager if urgencies require overtime

Of course, it may happen that you’re really overloaded. Extra payment for overtime may be a reasonable option for both parties. But it is better to agree on the deal in advance.

Change priorities to fix the critical issues first

To be truly effective, you need to define activities that are of the highest value and focus on those tasks. If you’re requested to do something unplanned, find out what is more important: these new things or your current work. It may happen that a new task is lower in priority.

With the right approach, you will always be able to get the most important and most pressing issues handled.

Review your plan to delegate some tasks to other team members

The worst possible outcome is failing to do the job but a man can do no more than he can. If you feel you don’t have enough time to do something yourself, delegate it. In most cases, you can always reschedule things or smartly allocate your duties. The main thing is to do it on time.

That’s why it’s so important to provide accurate estimates. When you’re aware of time limits, you’ll carefully and thoughtfully choose the best fitting options.

Why do people tend to underestimate?

Dan Milstein has a great article called “Coding, Fast and Slow: Developers and the Psychology of Overconfidence“. He addresses the problem from the software engineering perspective though I’m sure many of the lessons learned can apply to other fields as well.

He also mentions a great book by Daniel Kahneman Thinking, Fast and Slow. Kahneman, a winner of the Nobel Prize in economics, describes our thinking process as consisting of two systems that determine the way we think.

The book perfectly describes the impact of overconfidence on corporate strategies, the difficulties of predicting what will make us happy in the future, the profound effect of cognitive biases on everything from playing the stock market to planning your next vacation — each of these can be understood only by knowing how the two systems shape our judgments and decisions.

Another point is that sometimes people transform something very uncertain (a feature or story) into something really precise. Here is a real example:

Client: “I need a report on the number of hours my employees track each month”.

Developer:

1. SELECT a month, SUM (hours) FROM the database, and GROUP BY employee. 5 minutes
2. Display it on a separate page. Another 15 minutes.
3. Testing it. Oh yeah, check-in, deployment, release notes…
4. Uh, looks like it should take about 1 hour. No, to be safe: 2 hours.

Two hours later (or whenever), the client enters user testing and says “no, no, no, it’s completely useless”.

“Why?”

“Well, it is obvious that I need filtering to see a report for the needed month, and I want it to be broken down by employee title, and I want a chart that shows clearly who reports more and fewer hours, on what day, and then I need colors for sorting, and …”

• Always discuss expectations. Do not be lazy and explain to the client that you will start from something simple to lay the basis, and all other enhancements can be added later
• Always mention things that you think haven’t been discussed properly
• Draw more pictures (frames), diagrams; a picture is worth 1000 words

Have you ever heard about Hofstadter’s Law? It reads as follows: “It always takes longer than you expect, even when you take into account Hofstadter’s Law”.

In other words, even if you know a project will overrun and you fit that assumption into your planning, it’ll simply overrun your newly estimated deadline too. This is also referred to as the “Planning Fallacy.”

According to researches, the Planning Fallacy can be determined by the three basic biases:

• We fail to consider past experiences.
• We consider only the best possible outcome.
• We focus on the overall task, not on sub-components.

To avoid the planning fallacy, be disciplined, always question all key assumptions, don’t blindly accept the best-case scenario as the most likely one. Do research, find actual and unbiased information on similar situations in order to clearly articulate the worst, the best, and the most likely case scenarios.

Forewarned is forearmed.

Finally

Being a good estimator comes with years of experience, practice & knowledge.

After you set up your personal estimates, you should keep up with the deadline. If you miss the deadline, it only means your estimates have been wrong & in this case, you not only fail the task but also let down the whole team (or just other people) relying on you.

Besides, having a reasonable expectation of your potential progress will highly increase your confidence. If you have a tangible goal, you’re more likely to feel optimistic while proceeding and seeing real progress being made. Your initial achievements will greatly contribute to your motivation!

And of course, if you’ve never tracked the time you spend on certain tasks and activities, you will hardly manage to provide clear & exact estimates right away.

How about you? Do you believe estimation is crucial for working properly? And how do you estimate? I’ll be happy to hear from you!