Why project estimations are always off

by Yuri Mediakov, PM at BraveGeeks Team

Anyone who works in software development is constantly being asked to provide estimates for projects. An estimate usually has to be given for both work hours and calendar days. At this stage, the project description consists of a small text document and, in a best-case scenario, UI mockups.

It’s a very logical and understandable request: a business needs to estimate development costs in order to make a budget, construct a plan, and estimate the amount of investment required and the timeline for getting it back. For a development team, however, precise project estimates are difficult to come up with.

But why is it so difficult to provide an accurate estimate for a future project? At BraveGeeks, we provide custom software development, so we have to estimate project costs frequently. And we often explain to clients why the estimate is an approximation and how to make the best use of it.

I’d like to share some thoughts on this subject. This probably won’t be anything new for developers, but it might be useful for negotiating with internal and external clients. If a client understands the assumptions that an estimate is based on, then they’ll use it properly, which will ease tension as they work with your team in the future.

I want to make clear from the outset: I’m talking about software development at small and medium-sized companies. Large companies also have difficulties with project estimates, but the reasons why are often different.

Screenshot (Movie: In Time, 2011)

Technology develops very quickly

Let’s say there’s a hypothetical situation in which we’re asked to work on a project that’s completely identical to something we did a year earlier. It might seem like it would be very easy to estimate a project in a situation like this. After all, we already know all the ins and outs of the project, and the code is already written. But that’s not entirely true.

In the world of IT, everything changes rapidly. Since we completed the project a year ago:

• the library versions that we used in the project have been replaced with new ones, and the old ones have stopped working;
• new devices have come out: a couple of new iPhone models are now available, as well as a whole bunch of new Android phones;
• the operating systems that the project will run on have been updated;
• there’s a dozen new browser versions.

Most likely, even trying to build the old project from the source won’t be successful on the first try.

In addition:

• the browser or operating system that the client wants to use (but hasn’t told us about yet) now has new features;
• integrations with third-party services used in the old project have also stopped working;
• the old project’s UX/UI is probably out of date and needs to be updated;

And don’t forget that we’re still just talking about a hypothetical clone of a project from the previous year.

Software development requires creativity

An important note: naturally, the situation will be different in cases when a single type of software is being developed like it’s on a conveyor belt.

But aren’t you monitoring all the updates and already know about all the changes that have happened when the project begins? Why don’t you just make allowances for these changes in the project estimate? The old project cost X, so just add cost Y to account for these changes, and you’ll have the final estimate.

On a well-oiled factory production line, we can tell you how many products will be released per day, because the conveyor produces the same exact products day after day (actually, things are more complicated at a factory too, but that’s beside the point). But even in a factory, the conveyor can malfunction and a certain percentage of goods come out defective.

Unlike the conveyor belt, developing custom software is more like setting up a new production line, since they are both processes with a high degree of indeterminacy. When a project begins, we know very little about it.

From a technology standpoint, we can guess at the tech stack, basic frameworks, and the platforms we’ll be developing for it. What kind of developers we’ll need and how many. We can anticipate what kinds of basic challenges we’ll face. But that’s all.

From a UI/UX standpoint, we know even less. Even if we have a good understanding of the project’s goals, we’ll still have to study the target audience and its needs. We’ll have to figure out how to satisfy those needs by conducting a series of experiments to see if the design is suitable or if it needs to be revised.

At the very beginning of the project, we really know very little about how it will turn out in the end and what kind of problems we’ll face.

Humans work on projects, not robots

No matter how awesome your coworkers are, they’re still just humans.

Perfect developer (Dancing Robot by Arkady Zaifman)

They have their good and bad days. Sometimes code that was written today will need to be rewritten tomorrow. On some days, people make great decisions, and on others, very bad ones. Sometimes they’re lazy, and sometimes they make mistakes. And often it takes some time to deal with the consequences of those mistakes.

Coworkers also get sick. Replacing one developer with another during illness or vacation is usually not cost-effective: the productivity of the “replacement” worker is very low at first. Software development is a fairly individualized process, even when all accepted coding standards are followed meticulously.

Coworkers sometimes change jobs. In my experience, a new developer joining a project will need at least 3–4 months to start working at the same level of efficiency as the previous one.

It is practically impossible to foresee this kind of situations when planning a single project.

The scope of the project isn’t set in stone

During the development process, the scope of a project can and will change. The market situation changes, and additional marketing research is carried out. The client decides that the project needs to be done a little differently. And this is completely normal: no one wants to develop a project that no one will be able to use later on.

But, in contrast, development teams want to set the scope of their work at the start of the project. This is because of the need to make the project estimate as accurate as possible, among other reasons. Agile Methodologies came about in order to formalize the procedure for constant changes to the scope and the estimates for those changes.

But using Agile Methodologies doesn’t get rid of the need to estimate a project before work begins. The client is rarely happy with an approach of “we’ll work on it for a little while and then we’ll be able to estimate the rest of it,” which is characteristic of Scrum-alike methodologies.

Screenshot (Movie: In Time, 2011)

But then why are some people able to stick to an estimate, and others aren’t?

I think the goal of creating a project estimate before work begins and then carrying it out on schedule and on the budget without changing the scope is not very realistic. We have to understand what would make the most sense to change: the deadline, the budget, or the scope.

Postponing deadlines results in development that is impossible to complete: there is always something that needs finishing touches. Changing the budget usually means increasing the size of the team working on the project, but this slows the project down more often than it speeds it up.

So, when something doesn’t go right (which happens pretty often), I think the development team should change the scope of the project. This approach is called Fixed Budget, Fixed Dates, Flexible Scope.

Changing the scope is a difficult process that requires a strong will and collaboration with the client. It is important to decrease the amount of work in such a way that the project’s value to the business is not affected.

To work according to this model, a team must meet two criteria:

• the team must understand the goals of the project and its value,
• and the team must want to work together with the client to achieve these goals.

These things aren’t easy. To understand a project well, you have to spend a lot of time and energy on understanding your client’s business, the project idea, and the value it offers to users.

If you’re interested in what our process for examining a project before it starts looks like and how we plan a project and adjust the scope, let us know in the comments and I’ll be happy to share the details in our next article.

I’d also like to hear your thoughts about the nature of project estimates and the reasons why they’re imprecise.