20

Although it is a commonly held opinion that planning poker improves the accuracy of project estimations (a small sample of which demonstrated on this question), has any defined research been done on the subject?

More specifically, I am looking for non-circumstantial information showing that planning poker would be an improvement over traditional estimation techniques.

Community
  • 1
WLPhoenix
  • 309
  • 1
  • 5
  • 4
    i don't see how research like that would be useful at all, estimation is highly dependent on what you are estimating and experience estimating it regardless of how you do it. anything a study found would be extremely unlikely to be reproducible in your setting – Ryathal Jan 16 '13 at 15:10
  • 3
    @Ryathal: I don't see why this would be inherently un-researchable. Pick two randomized groups of programming teams; measure their relative estimation efficiency (both groups doing the same projects) using traditional estimation techniques, this is your baseline. Then have one group switch to planning poker, while the other one keeps using traditional techniques. Have them both deliver the same projects. Measure again, and correct for differences found in the baseline. It's not double-blind, but still meaningful. – tdammers Jan 16 '13 at 15:15
  • Research can be much more effective at convincing management than group consensus, especially if it's an early foray into agile territory. – WLPhoenix Jan 16 '13 at 15:16
  • I suspect the answer is that experience has taught us that any estimation technique is flawed and this one at least has the decency to be obvious about it. But, very good question. – pdr Jan 16 '13 at 15:27
  • @tdammers my objection isn't to the inability to do the research, its to the research providing meaningful results. – Ryathal Jan 16 '13 at 15:28
  • 2
    **[resource requests are not quite welcome at Programmers](http://meta.programmers.stackexchange.com/q/5454/31260 "as discussed eg here")**. As far as I understand, one would rather present an **underlying problem** instead - a problem that was intended to be solved with particular resource requested – gnat Jan 16 '13 at 17:39
  • This could be done on any team as a [Single Case Design](http://en.wikipedia.org/wiki/Single-subject_design) study. – JeffO Jan 16 '13 at 18:36
  • 1
    Planning poker is an example of a delphi estimation technique, and is a variant of wide-band delphi. There is much research on wide-band delphi that applies to planning poker. – Michael Jan 23 '13 at 01:11
  • @Michael If you wrote that up as an answer and provided some links I would accept that. – WLPhoenix Jan 23 '13 at 15:57

3 Answers3

6

Google Scholar turns up some papers

You might find the following papers useful, but they are behind a paywall and may be a little dated now:

You might also want to consider A Case Study on Agile Estimating and Planning using Scrum 2011 (free PDF) starting around page 123.

Gary
  • 24,420
  • 9
  • 63
  • 108
1

A story from my experience.

When our team started - we were using planning poker by Mountain Goat. Since our team was distributed among 2 cities, we were short on options for estimation methods. As for the tool itself — it was Planning Poker 3.0 JIRA Add-On

Thus, as our team was growing (in size of members and in terms of experience), we found out that the planning poker does not fulfil our needs anymore. Due to the scattering factor of team members, due to issues from different projects that have to be estimated during the same meeting, due to inefficiency for estimating large piles of stories - we switched from this method to another - the Team Estimation Game by Steve Bockman.

This method worked well for us, though the preparation time for a session increased (we didn't have a proper tool with the "online multiplayer" solution) - our scrummaster used Google Docs for preparing an estimation session.

Vitalii Zurian
  • 111
  • 4
-2

Measurements so far based on same set of staff - (unscientific measures): Planning poker - variance off by 24-78% from actual time it took to complete 113 tasks. Historical, data analysis techniques off by as much as 200%. On average, however within 30% of each estimate it took to complete 113 tasks. Commentary- I am biased I admit after the above was observed. The value i would see if there is a discussion for rationalization of each estimate. if you have a team member who thinks through carefully each step of his estimate, and also carries out his estimate within a small variance, then that person(s) would add value - however if it is a quick guess on each estimate this is a folly and a waste of time imo. completely breaking down the problem into small bits would probably be the best estimate of time expended - coupled with a long trail of historical data.

mrskeptical
  • 1
  • 1