6

One of the projects I work on is measured by number of issues. Management has given arbitrary (as far as I can tell) measurements that we need to have no more than X number of bugs and Y number of features in our issue tracking system.

This seems like a misguided approach, but I can't seem to find any writings on the subject.

Thomas Owens
  • 79,623
  • 18
  • 192
  • 283
Asa Ayers
  • 526
  • 3
  • 6
  • 8
    How to spot a bad idea: No one else has heard of it or written anything about it. – S.Lott Jun 27 '11 at 21:46
  • 1
    Most things that are easily quantifiable are bad metrics because they are oversimplified or can easily be gamed to perform well against the specific measurement. – unholysampler Jun 27 '11 at 22:00
  • 1
    I'd like to know their definition of a feature. – JeffO Jun 27 '11 at 22:16
  • I don't think the definitions matter much, but generally if we want it to do something it isn't currently doing it's a feature. If it does something unexpected or fails to perform an expected action, it's a bug. – Asa Ayers Jun 27 '11 at 22:35

6 Answers6

8

No, such a metrics are absolutely counter-productive. With this kind of metric, people spend time gaming the metric, rather than thinking how to do their job well.

I've actually seen a situation, where there was contract on maximum number of outstanding bugs. The effect was, that most of bug reports were immediately closed with bogus resolution.

vartec
  • 20,760
  • 1
  • 52
  • 98
5

Not all issues are created equal. Some are trivial changes and some are multi-developer multi-week projects. A simple issue count doesn't take these factors into account.

Asa Ayers
  • 526
  • 3
  • 6
  • Why answer your own question....? – Tyanna Jun 27 '11 at 22:04
  • 3
    Because there are multiple answers and pieces to this question. I thought about including this up top, but realized that it really belongs as an answer. – Asa Ayers Jun 27 '11 at 22:05
  • 1
    Answering one's own question has always been OK _as long as it really is an answer_. – Donal Fellows Sep 03 '11 at 15:01
  • A classic example of an awkward issue is “My code is too slow when running in your interpreter. Please make it compile to automatically-parallelized machine code.” That's potentially years of work… – Donal Fellows Sep 03 '11 at 15:06
4

This is a very silly thing to do. This will discourage people from reporting problems, so the problems will be ignored, and everyone will pretend things are fine when actually they are bad and getting worse by the day. Then you will be working around the mess. When management decides to punish people for telling the truth, it is time to go.

kevin cline
  • 33,608
  • 3
  • 71
  • 142
2

Tracking just the number of bugs and feature requests doesn't strike me as a good idea. To me, it's dependent on people reporting issues. If there simply is no need for a new feature by an end user or a bug found either in the field or by QA, then there won't be any tracked bugs. It might be interesting to know how many bugs or feature requests are in the pipeline to try to estimate the amount of work. Also, the phrasing "no more than X bugs" or "no more than Y features" bothers me. What happens if there are X bugs and another one is discovered? Or Y features and another user or client asks for something else? They shouldn't be ignored until bugs are closed or features completed.

Instead, I would use bug reports and feature requests as measurements for use in metrics. For example, tracking reported bugs over time. If internal testing is doing their job, the number of bugs reported after a release should not spike. Another example would be tracking the time it takes to fix a bug or create a feature as this would give some insight into the maintainability and extensibility of the system.

I was looking for some material cited in a product quality course that I took. Although I couldn't find it, I did find these notes on change and defect metrics. It might be interesting and give you some insight on how you can use data about feature requests and bug reports to give you more reliable and valid information about a project.

Thomas Owens
  • 79,623
  • 18
  • 192
  • 283
2

I worked at a company once where management had a very elaborate system in place for tracking progress and productivity - almost entirely based on change control and bug reports items. It was a dismal failure:

eg. You might have a programmer who had to deal with a very obscure and difficult to reproduce bug, and as such just resolved just "1 item" all week. On the flipside, someone else might have been working on relatively easy items, and resolved "20 items" in the same week. Superficially this looks like the second programmer was "20 times more productive".

Now, to be fair, none of this was actually directly used to reward or punish people per se. Our management was technical enough to understand that the above scenario can happen. But still, given that the metrics are in place, it just tended to psychologically look and feel bad when you spent forever on something. And certain products and codebases at the company tended to generate more nasty bugs than others - or the work was more predictable than on others. So in the end, all this did was cause people to get demotivated when their "numbers" didn't look as good as other groups' or programmers' numbers.

Bobby Tables
  • 20,516
  • 7
  • 54
  • 79
0

Number of issues and open/close trends in a tracking system can be a reasonably good indicator of status of the project, when taken in context of historic data for the same team or company.

However, issuing a directive to limit the number of issues only invites abuse. People will game the system, even without dishonest intentions. Bugs will go unreported or resolved improperly, features will be skipped or undocumented, friction between test/dev teams will increase.

dbkk
  • 2,106
  • 17
  • 16