Estimates

[Jason Madison]

A programmer I work with spent 4 hours producing a 14 day estimate for a
project. In the event the project took 15 days and his manager was annoyed
that he had estimated incorrectly.

Is there any research into the ratio between time taken to produce an
estimate and the accuracy of an estimate? Could it be said for example that
if the programmer had spent 8 hours on the estimate he would have been more
likely to have arrived at 15 days?

 

[Guest]

There is definitely no single ingredient to calculating a correct estimate on
solving a given problem that involves any non-trivial amount of complexity.
I've had problems getting my guys/girls to provide me with accurate estimates
for project work and this is what I've done.

I let them develop a detailed project plan with their estimates on times and
then I retain them. When the given piece of work is completed, I then
calculate a ratio of how long they estimated versus how long they actually
took. Without their knowing, I simply multiply their estimated timeframes by
this ratio and use the calculated figure for my 'real' timeframes. As time
goes by, I continually update this ratio and retian the history of it. I am
then able to plot how their ability to estimate work compares to their
ability to meet these estimations. If, for some reason, the delivery of a
piece of work is outside of some threshold that I use to indicate a problem,
I then look at the detailed estimates and investigate where (and why) a
particular task took longer than expected.

Together we then work on how we can provide better future estimates and then
discuss this with all developers. It has to be something that is ongoing and
unfortunately, there will always be situations where you are unable to really
estimate accurately as there are always issues coming from left of field that
noone expects.

Also, the ability to accurately estimate tasks improves with the experience
of a given developer, working within a given environment. By using this
method, I can maintain realistic figures for all of my workers, irrespective
of their ability to estimate accurately. I can also pair up people whom sit
at different ends of the spectrum and it typically improves both peoples'
ability to provide workable timeframes.

Hope this helps.

-Eric

 

[John Murray]

For the essence of the question, I don't know of any research out there
that addresses the question that you ask. But just like programmers
produce at vastly different speeds, it's not hard to see that those
doing the estimation also would produce at vastly different speeds.
Additionally, the estimate was only off by less than 10% which in my
experience is highly accurate. My guess, based on the limited
information, is that it's the manager that has the problem and not the
estimator.

What this post doesn't provide is the cause of the under-delivery.
Where people out (sick etc), did the plan estimate people working
death-march hours? Was there scope-creep? Was the person who did the
estimate the only person who worked on the project? Perhaps the
estimator should have padded the estimate to account for unanticipated
problems?

If you are concerned with estimation, I would suggest that you look at
some of the more formal estimation methodologies including function
point analysis.

John

 

[Jeff]

My opinion is that being [off by only one day on a 15-day project] is either
a lucky coincidence or the estimator is incredibly knowledgeable about all
possible relevant factors and didn't have the requirements change on
him/her. The manager is completely out of line to be unhappy about that.

Think about it. It's impossible to accurately estimate. Alternatively you
could say that all estimates are perfectly accurate and then reality is or
goes wrong. Okay I just lost a bunch of readers with that. Put another way,
let's walk up to that manager out of context (e.g.., on the golf course or
in the super market) and tell him/her that we can perfectly (or at least
accurately) predict the future. Period. In essence we are claiming god-like
powers to *know* the future. The manager would simply not believe us. So,
why then does the manager get to work and expect his/her programmers to be
able to have god-like powers to know the future perfectly? That's nuts.

In the manager's defense in relation to my statements, one could say that
the manager wants *reasonably close* estimates and not *perfection*. Fine;
then what is the definition of reasonably close? Within 10% is pretty good -
especially for a software project. Heck, with above-90% accuracy that
programmer should go to Vegas.

So what can be done to make estimates better or worse? More accurate info
and more complete info (recognizing that it will never be 100% accurate nor
100% complete). The more the estimator knows, then the more accurate the
estimate can be. That should be obvious to management - but often times they
are blinded from such rational thought by their own deadlines and promises
to their superiors; often times made on behalf of unknowing programmers.
They *want* for the estimates to become reality and when it doesn't happen
they forget that it was an *estimate* based only (and possibly only) on
limited knowledge. They often rant and rave about the deadline being missed
and then point fingers to the programmer or estimator when reality comes to
bear. Remember this is the same manager who would simply not believe you if
you walked up and claimed to perfectly know the future.

"Reality" [that comes to bear] includes practically unlimited factors that
can render the original estimate invalid. Most often it is a combination of
project scope creep mixed with imperfectly understood requirements. Even
when the scope is nailed down and requirements very clear, the existing
system within which they are to be implemented may be poorly or at least
incompletely understood. Example: "we just need to add one new field to the
TPS report" and so the developer estimates 4 hours to add the new column to
the database, create a trigger to populate the new column, update the SQL to
retrieve the value, place and format it in the design-time report, and test
everything. Sounds so simple. But then the developer subsequently finds that
the table already has the max allowed number of columns; so a new table has
to be added to the db. The query then becomes more complicated because we
now have to join a new table; at this time additional refactoring of the db
or application may become necessary. That original 4 hour estimate is out
the window. Then there is a manager somewhere ranting that the estimate was
off. Again, this is same guy/gal who would not believe you if you walked up
and claimed to perfectly know the future.

Everything would be much better with managers would understand that
estimators are incapable of (1) predicting the future perfectly; (2)
understanding the requirements perfectly; (3) understanding beforehand what
all of the relevant unknowns will be; and (4) perfectly understanding all of
the "knowns".

Your developer is either brilliant or lucky or both to have gotten a 15-day
project off by only one day.

-IMO

 

[Guest]

hmm,
what would the manager have said if your colleague had completed in 13 days?
would they have been angry - but consistent - that would have left your
colleague understandably annoyed?
or would they have been pleased - and inconsistent - and left your colleague
reasonably happy?

either seems wrong to me - the response i would expect was congratulations
that the estimate was so close

when i produce an estimate i make it clear that the estimate is within 20%,
40% or whatever, dependent on the knowledge i need to aquirre

While i know and understand Erics approach the problem is that the wily
developer who finishes say 3 days early wont let the manager know that they
have finsihed until 1 day early - looks good but doesnt drop the time they
have for the next project


cheers

guy

 

[Nick Malik [Microsoft]]

A programmer I work with spent 4 hours producing a 14 day estimate for a
project. In the event the project took 15 days and his manager was annoyed
that he had estimated incorrectly.

Is there any research into the ratio between time taken to produce an
estimate and the accuracy of an estimate? Could it be said for example
that if the programmer had spent 8 hours on the estimate he would have
been more likely to have arrived at 15 days?

Hi Jason,

First off, your friend's manager needs to get a job in a profession that he
understands.
Secondly, your friend needs to get into the habit of stating, in his
estimates, that they may be off by as much as 25%. Rolling in at under 7% is
super.

Third, if he is spending four hours producing an estimate for a very small
amount of work, then the manager may want to reconsider the need for the
estimate at all. Given that estimates are pretty well useless for most
short projects, I'd suggest that you can improve the overall ability to meet
the customer's needs by doing less estimating, not more, and spend more time
actually writing code.

This last point is a bit controversial. However, if you want proof that
sometimes, the best thing to do is to kill the estimates, read this paper
from David Anderson:
http://www.agilemanagement.net/Articles/Papers/TOCICOBarcelona.html where he
took a small team, killed the estimates, changed a few of the management
policies, and whittled away a long backlog of small changes (like the one
you suggest) to zero in nine months with tremendous increases in
productivity and developer satisfaction.

--
--- Nick Malik [Microsoft]
MCSD, CFPS, Certified Scrummaster
http://blogs.msdn.com/nickmalik

Disclaimer: Opinions expressed in this forum are my own, and not
representative of my employer.
I do not answer questions on behalf of my employer. I'm just a
programmer helping programmers.
--