The Fallacy of Work Estimate Tracking

An estimate is only an estimate. Let's start there. It's guaranteed to be wrong, and you'll adjust either upward or downward as you gain more information. The only time you know exactly how long a task will take you is when you're done with it.

That's not what this post is about. My current environment is a bit challenged when it comes to sharing information. There are many aspects to this, but the one that's on my mind right now is task estimates.

I come from a TFS environment. I'm used to being given a list of scenarios, and then I fill in the details as far as what tasks need to be accomplished to mark those scenarios as implented. Each task gets an estimated number of hours, and as you begin getting your hands dirty you can update the remaining number of hours.

• Scenario A
• Task 1: Estimated = 8 hrs, Remaining = 5
  
• Task 2: Estimated = 12 hrs, Remaining = 2
• Scenario B
• Task 3: Estimated = 2 hrs, Remaining = 0

And from here, there might be some web-based reporting that indicates to my manager that I'm (5 + 2 = 7) / (8 + 12 = 20) = 35% complete with Scenario A and 100% complete with Scenario B.

And therein lies the problem. This says nothing of feature completeness, it only speaks of time remaining. Scenario A might be 95% complete as far as the feature itself, but you wouldn't know that by seeing the 7 remaining hours.

As I mentioned before, my current environment is a bit challenged, so I've resorted to using an Excel spreadsheet to track my work. Each Scenario is broken into subtasks. Each subtask has 3 different dates associated with it:

• The date the logic was implemented, including FxCop and StyleCop validations. I think it should work.
• The date I verified the logic in a test scenario. I've confirmed that it works.
• The date the code was reviewed. Someone else agrees on this implementation.

Once all three of these dates have values, the code can be considered ready for checkin.

I started out thinking, Well, I have all of this information; I should be able to figure out what % complete I am. If I take the COUNT() of subtasks that have dates in the "logic implemented" column and divide by the total number, I know how many I've coded up. But as we all know, what you think should work and what actually works are two completely different things. Otherwise we wouldn't ever need to debug. So I brought in the other two columns in a weighted average:

• Logic: 50%
• Test: 25%
• Review: 25%

I'm comfortable with this. Most of my tests and most of my reviews are going to work right off the bat, so 25% seems like a bit high of a weight to assign. However, those cases that fail will potentially require more rework, so I think it balances out. In my previous incarnation, we padded our estimates to account for test and review, but they were never explicit checkmarks on a list.

If I've done my Scenario decomposition correctly, each of those Subtasks should be represented by a bit. It's either done or it isn't. As I get into the code and see what the solution actually entails, I should be adding subtasks for logical paths that weren't considered in my original estimate, not tweaking the "Hours Remaining."

Now we're back to the original problem. I've indicated what % of the feature is complete, but now I have to link this back to a time estimate. The most obvious way of doing this is to just multiply the % Complete by the original baseline estimate for the scenario. But this is incorrect, and possibly very very wrong. The last 10% of implementaiton might take 90% of the total time. Furthermore, times can vary dev to dev. What takes me 2 days might take someone else under an hour.

I think ultimately the problem here is that I'm coming into this project backwards. The scenarios and estimates were already made. Had I made them myself, I would have been able to add times to each subtask and the total estimate for the scenario would have been the sum of all those subtasks. There's still something quite valuable to be taken away from this mental exercise, however:

The most granular level of estimation, what I'm calling here the SubTask, is either implemented or it isn't. It should never have a Time Remaining associated with it because if there's any time remaining, it's not done by definition. Baseline it, fine. Track actual, fine. But if you ever find yourself tempted to give it a partial completeness, you need to decompose it.

This serves gives you several benefits:

• Management can see exactly how involved the assignment was. When you're presented with "Why's it taking you so long? You're way over your estimates." You can say, "Because I encountered these 15 other execution paths that need to be accounted for," and provide the list.
• These extra execution paths do not fall through the cracks. You and your reviewer will have the list to consider for new implications, and test will have the list.
• You're tracking what actually needs to be done, and not obscuring the work under the umbrella of an hourly estimate.

One of the features I'm most excited about in TFS 2010 is hierarchical work item tracking. I'm hoping my Excel-sheet tracking days are numbered, and soon I'll be using the tool designed by the experts for this job.