Jory MacKay on November 03, 2020 The Ultimate Guide to Estimating Projects: How to use time tracking (and experience) to set better estimates
Everyone’s experienced a “quick” task turning into a day-long endeavor. Despite our best intentions, human beings are terrible at estimating how long a project or task will take to complete. But as a project manager, it’s your job to properly estimate a project’s time and cost so that your team knows what they’re working towards (and you know when you’re going off the rails!).
Most major project management issues can be traced back to poor time estimates.
Project running over on time and cost? Poor estimating.
Team overloaded with extra work? Poor estimating.
Stressed out because you’re missing milestones? You guessed it, poor estimating.
Estimating a project is a skill that you learn over time and is heavily influenced by experience.
The more you do it, the better equipped you’ll be to estimate future versions of that project. But what if you don’t have the necessary experience? Or if you’re running a new project or just started a new job?
Luckily, there are some specific tools, tricks, and techniques you can use to help estimate how long (and how much) a project will take to complete.
Why is it so hard to properly estimate a project? Blame “The Planning Fallacy”
Estimating requires a human touch. And unfortunately, humans are pretty susceptible to biases and cognitive errors, including when it comes to planning for the future. The worst of which, at least when it comes to project estimating, is the planning fallacy.
The Planning Fallacy is our bias towards being overly optimistic when it comes to estimating how much time a task will take to complete. In other words, we assume pretty much everything will take less time than it actually does.
The Planning Fallacy has been blamed for everything from late term papers to billions of dollars of unexpected costs on airports, opera houses, and other development projects.
According to Daniel Kahneman and Amos Tversky (who came up with the term), the Planning Fallacy happens because when we ignore historical evidence about tasks and instead assume a best-case scenario for future tasks—no matter how unlikely that is.
Think of it like this: You're assigned with finding a new widget supplier for your company. It seems like a simple enough task, and you've done it plenty of times before. If all the potential suppliers have the information you need on their website, it shouldn't take you more than a couple of minutes to assess each supplier. Double that to be safe, and you can easily get the task done before lunch with time to spare...
Except half the suppliers don't have any of the information you need on their site. Your easy morning of browsing websites turns into a day of filling in forms and looking for contact details.
When you finally manage to find the suppliers’ phone number, it takes forever to find out the person you need to speak to won't be in until next week. Your four-hour job ends up taking a week. Still, the next time you're asked to source a new supplier, it's a simple job, and you should be done by lunchtime. Right?
The planning fallacy doesn’t just impact project timelines. It can also lead to overrunning budgets and resources, or under-delivering on promised performance and benefits.
Estimating a project is a skill that you learn over time, not something you just have. It heavily depends on your experience.
The essential elements of a project estimate process
To understand how to free your mind from the planning fallacy and properly estimate your project, you need to break the process down into a few key parts. Here’s how veteran project manager Werner Meyer describes it.
First, there’s the estimation object. This is what you’re going to be estimating. So, in our case, this will be a task, milestone, or project.
Next, comes the attributes for each of these objects. This is how you’re going to complete the task/milestone/project. This could include duration, budget, resources, and so on.
For each of these attributes, there's an unknown quantity. In other words, the thing that you’re going to estimate. This could be the length of time a task should take or the cost of hitting a milestone.
Finally, we have what’s called the estimation event. This is the moment where we take all these objects, attributes, and unknown quantities and come up with an estimate for how long a project will take to complete or how much it’s going to cost.
To deal with the estimation event, we look at a number of different internal and external influences that will determine what we think the unknown quantity should be. More specifically, those influences include:
- Internal: Personal biases, experience, personal interest, etc...
- External: Risks, resources, team, etc… (for example, using a new technology
That’s a lot to deal with at once. But the true extent of the issue becomes even clearer when you see it in practice.
Applying this estimating model to a website redesign, here's what it looks like:
- Estimation object: Website redesign
- Attributes: Duration, technology, team size
- Unknown quantity: Hours required to redesign, cost of technology used in the redesign, number of team members available to work on each task
- Estimation event: Time and cost of redesigning the website, taking into consideration:
- Internal influences: Optimism bias (everything will go smoothly without any complicating factors), Experience (time taken and cost for previous redesigns)
- External influences: Existing time and/or team commitments, Complications arising from choosing/installing new technologies
While this model won’t tell you exactly what your estimate should be, it does give you a framework to think through the specific issues and biases that could influence you in the wrong way.
It’s a start. But to properly estimate, we need to dig deeper.
To make more accurate project estimates, ask these 5 questions
With your project estimating process in place, it’s time to dig into the details.
The more accurate and objective your information is, the better chance you have of making an accurate estimate. As you get to your estimation event, ask these questions to push yourself further and not fall victim to the internal influences that want to skew your vision.
1. What are your team’s expertise and responsibilities?
When you start thinking about accurate estimates and timelines for your next project, it's easy to get the spreadsheet out and focus on the formulas and figures. Reality, however, doesn't fit neatly into a spreadsheet.
Project planning and estimating should always start with people, not numbers.
This means you can't hide away in a tower when you're planning your project. You need to know your team, to work alongside them, and understand exactly what's involved in the day-to-day job. You need the knowledge that comes from getting your hands dirty and working alongside your team.
The more you know about the individuals on your team, their work, and their processes, the more accurately you can estimate their future work.
If there's anything you're not sure about, don't be afraid to ask your team. You don't have to know everything as a project manager—in fact, being prepared to ask questions and learn from the others on the team will make you far better at coming up with accurate estimates.
Project planning and estimating should always start with people, not numbers.
2. What is your team/company’s project management process?
If you're reading this post, chances are you have a specific process for managing projects, something that's more detailed than 'start project, finish project.'
We've written before about the different project management processes out there and how to choose the right one for you but, for the purposes of accurate project estimates, we’re not so worried about which process you use. Instead, it’s more important that, whatever process you choose, you understand it. Become a student of the process, familiar with all its intricacies, every strength and weakness.
You should especially learn how the process works in your business. You can learn a lot from a book, but you can learn even more by seeing what the process looks like in practice, in the wild, for your team. The way your team uses the process will most likely have its own unique quirks, and understanding those quirks will help you make far more accurate estimates.
Working on a team without a specified project management process? If so, it's even more important to understand how your team plans projects. Even if it's a loose plan, one that changes regularly, you need to know how present projects are carried out before you can estimate future ones.
3. How long have these projects taken in the past? What roadblocks did you/your team hit?
The best way to get an accurate estimate for a future project is to look at similar past projects.
If it took you an average of four hours to write up your quarterly report in the past, chances are it'll take four hours next time too. That's also true for any problems you run into—if they were an issue in the past, they'll likely be an issue in the future.
Remember the planning fallacy though. We have an uncanny ability to ignore the lessons from past projects. Our rose-tinted view of the past downplays any problems as unlikely or unimportant, leaving us far more optimistic than we have any reason to be.
To overcome this, you need to have some solid data on your past projects. If you've carefully recorded the time it took to perform a task, then that becomes a fact, rather than an opinion. If you keep track of the roadblocks as they appear, they become harder to ignore in the future.
By using a time tracker (such as the one found in Planio), you'll build up historical data. Then, when you're trying to estimate how long that new website redesign is going to take, you don't have to go by your gut—you can make an estimate based on the facts.
4. What exactly are you building?
If you're going to come up with an estimate for a project, you need to know what you're planning to achieve.
Of course, no-one's going to create a project with absolutely no idea of the intended outcome.
Still, for the best results, you need more than a general idea of what you're planning to achieve—you need a crystal clear image. It's the difference between planning a trip to New York and planning one to the top of the Empire State building. One is a general idea, with no clearly defined outcome. The other trip leaves no room for doubt. The details make a big difference.
Most projects will require either an SOW or project proposal before they're green-lit, which makes these a great opportunity to get your project goals down in detail.
Move beyond what and also think about the why, the reasons behind your project. If you're planning on redesigning your site, what are you hoping that'll achieve? Are you trying to launch a new service? Make your site more mobile-friendly?
From this high-level view of the project, drill down into the specific deliverables. How will you know when you've succeeded? What will it look like?
For your site redesign, this might mean a seven-page site built on the Wordpress platform with eCommerce capabilities. By the time you're done, you should have a scope that spells out every detail of the project, with no room for misinterpretation.
5. What steps do you need to take to hit milestones, goals, and OKRs?
Once you know your destination, you can start to plan your route. This means breaking down every part of the project into detailed tasks, step-by-step instructions that can be followed to the letter.
No matter how complex your project is, it can be broken down into smaller separate tasks. Simple projects mean fewer steps, while more complex projects mean you'll need to take more steps. Making a cup of coffee might take six steps to complete. Landing on the moon might take sixty thousand. Either way, identifying those individual tasks makes completing the project—and estimating the time it'll take easier—a lot easier.
Task management starts by capturing every single step that needs to be carried out to reach your objective. Brainstorm all the tasks, ideally with your team, and make sure there are no gaps. Once you're satisfied you've identified every step, you can then start filling in the details.
Remember to record each step with all the necessary information and in the right order. If your project is making a cup of coffee and your first step is to take the coffee out of the cupboard, make sure you have all the information you need to carry out that task—you wouldn't want anyone mistaking the decaf for the proper stuff!
If your steps contain enough details so they can be followed by anyone, even without prior knowledge of the project, then you've done your job.
Most major project management issues can be traced back to poor time estimates.
Need help breaking your project down into actionable tasks? Check out our in-depth Guide to Task Management.
Time tracking: The secret weapon for better project estimating
While asking all these questions will set you up for estimating success, there’s no substitute for cold, hard data and experience.
We've already touched on the importance of time tracking; if you don't accurately know how long a project has taken in the past, the planning fallacy will be stronger and any future estimates will suffer. However, having solid data on the time similar projects have taken means you don't have to make a guess.
The easiest way to collect that data? Time tracking.
Before getting started, make sure you've uploaded the details for all the tasks into your chosen time tracking tool. For example, by keeping all your project details in Planio, time tracking is readily available on every page, just by clicking on the clock icon.
Click when you start a task, click again when you're finished.
Finishing the task also lets you log additional information that can be useful for future estimates, such as activity details and comments regarding any roadblocks.
Whatever time tracker you choose, the best one for you is the one you'll actually use. The best time tracker app in the universe won't do you any good if it just sits unused on your desktop or phone.
That's why it’s so powerful to use the time tracker built into your project management tool. Keeping everything for your project—including time tracking—in one place means you're more likely to use the tools and get the full benefit.
Once you've tracked the time for your project, it's important to do something with that raw data.
It's not enough to just know how much time you've spent on the project. You need to be able to filter that data by task, team member, or milestone. This is the kind of information that leads to more accurate estimates.
For example, in Planio you can quickly see your estimated time next to the actual time you spent on a task or issue. This is a fantastic way to test your estimates and make more accurate ones in the future.
Not only that but seeing your progress and the time you’ve spent on tasks can help motivate your team.
According to Harvard professor Teresa Amabile, making progress in meaningful work comes with a whole host of other additional benefits. She has dubbed this the Progress Principle, and it includes boosting emotions, motivations, and perceptions during the workday.
However, tracking the time you've spent deleting spam emails isn’t going to put a spring in your step. Instead, track those tasks that actually matter, the ones that move you and your team toward completing your next big project, and you'll have a happier (and more efficient) team.
3 tricks that will improve your project estimating
Alright, at this point we understand the framework for estimating anything from time to budget. We know about the Planning Fallacy and other influences and biases that impact how we estimate projects.
Lastly, let’s look at a few tricks that the best project managers use for better estimating.
1. Reference class forecasting
When we have to come up with an estimate, our tendency is to go with our gut. Unfortunately, this is also the most unreliable method. When we remember the planning fallacy, it's clear that the best way to come up with an accurate estimate is to use previous projects as our source.
Reference class forecasting means using similar past projects to develop more accurate project estimates. Rather than looking inside for the answers, we can produce estimates based on established facts.
The bottom line? The time a similar project took is a much more reliable indicator than whatever hunch you have.
2. Estimate reviews
To ensure that undue optimism isn't affecting your estimates, consider having your estimate reviewed by an independent party.
An expert who's familiar with the project type—but who lacks the bias that comes from direct involvement—is much more likely to give an accurate estimate. As they're removed from the project, with no personal stake in the outcome, they're able to dispassionately consider all the different factors that can affect your timeline.
However, it's important that any reviewer isn't incentivized, even unintentionally, to provide an optimistic estimate ("By the way, if we get this project completed within three months, that’d be great. There's a bonus for everyone involved too"), as this will lead to the expert bringing their own biases into play.
3. Unpacking tasks
You should never try estimating how long the whole project will take as a single entity. Instead, you should estimate how long the individual tasks will take. Not only will unpacking the project into its constituent parts make it easier to accurately estimate, but it also reinforces the true size of the project.
Breaking your project into its constituent parts not only makes it easier to accurately estimate, but also reinforces the true size of the project.
Redesigning a website sounds easy, but when you break it down into the multiple design, copy, and tech tasks you quickly realize just how much work is involved. Even without similar projects to use as a reference, looking at a project through this lens greatly reduces the effect of the planning fallacy.
Remember that planning is just guessing
No matter how experienced we are, accurately estimating projects is a lot harder than it first seems. The planning fallacy and our optimism bias make us blind to potential obstacles, as we fool ourselves into thinking only of the best-case scenario.
Still, the situation isn't hopeless. By asking more specific questions about the project, tracking time spent on previous similar projects, and using tricks designed to reduce our biases, we can realistically estimate how long a project will take and avoid missing deadlines or blowing through your budget.
However, it’s worth remembering that accurately estimating projects is always going to take time. By being willing to learn from our mistakes, we can avoid spending forever trying to “perfectly” estimate a task and get on with carrying out more meaningful work.
