Estimate this!

“Those who do not learn from history are doomed to repeat it.” - George Santayana

We often see this quote carted out when referring to some geopolitical or societal issue. Truth is, it's a great way to illustrate the problem with up-front estimating on software development projects. We've all seen it: The client wants an estimate for how long it will take to build the system. The delivery team is newly formed, with each member having various levels of experience in the technology and or the domain, or worse, there is no team yet and a smaller group (who may not even end up on the project team) has to put together the estimate.

This initial estimate will often be done bottom-up – the project will be decomposed into small units of work (e.g. features) that each get estimated separately. These small estimates are then summed up into the overall estimate. Unfortunately, these estimates are often wrong, sometimes dooming the project to a 'death march' situation.

So what have we learned about estimates?

• Estimating learning #1: Any newly formed team is very bad at estimating at a granular level early on in a project, and pretty good at doing the same thing at the end of a project.
• Estimating learning #2: Any estimate carries with it a set of critical assumptions that are often not communicated.
• Estimating learning #3: Personalities play a big role in early estimates. People who are overacheivers or who are seeking approval from others tend to give very optimistic (and innaccurate early estimates. People who lack confidence, who are conservative or are risk averse tend to give very pessimistic early estimates.
• Estimate learning #4: The past is often the best predictor of the future. If you have metrics that are applicable, then use them. For example, if you built an application of similar scope in the past, this is a good metric to use to check your estimate. Better yet, if you know the velocity of the team that will deliver the new project, you can use it to help develop your estimate.

What's the bottom line? Base your estimates on what you know (no guesses) and make all of your assumptions explicit. This means that, early on in a project, you typically have to avoid bottom-up estimates based on tasks or features, and instead derive estimates based on similar projects whose size is already measured. This initial estimate can then be refined as the project produces more data and more assumptions are validated.