10

The upper management at our company has laid out a goal for our software team to be “15% more productive” over the next year. Measuring productivity in a software development environment is very subjective, but we are still required to come up with a set of metrics.

What sorts of data can we capture that would measure our team’s productivity?

markyd13
  • 211
  • 1
  • 2
  • 5
  • 7
    Sell more software. – JeffO Mar 28 '13 at 19:17
  • 17
    Measure it in Days Since the End of July 2006. You should get about a 15% increase in the next year, and it's as valid a measure as any other you'll find. – pdr Mar 28 '13 at 19:18
  • 1
    @pdr - That's no good-they'll suffer from declining returns: 16.65% in 2013, 14.27% in 2014, and less than 10% in 2017. – Matthew Flynn Mar 28 '13 at 19:41
  • If your team is held to the fire on that requirement, the biggest metric you'll notice is number of engineers that leave – Jimmy Hoffa Mar 28 '13 at 19:52
  • 14
    If management has no way to measure it, how do they know it needs to be improved by 15%, or if it's even feasible to do so? – Karl Bielefeldt Mar 28 '13 at 19:53
  • 1
    [Martin Fowler wrote a good article on this](http://martinfowler.com/bliki/CannotMeasureProductivity.html). If he'd post it here, I'd vote it up :-) – Dan Pichelman Mar 28 '13 at 19:54
  • Unless I'm seriously mistaken, it it the task _and_ responsibility of (uhmm) "upper management" to actually define this kind of thing. They get a helluva lot of money more than anyone else, and, as far as I'm concerned, are supposed to do something for that buck. This includes a vision on what is actually expected. – Thomas Mar 28 '13 at 20:16
  • They actually want you to work harder/longer hours – ozz Mar 28 '13 at 20:20
  • 1
    Fire 15% of the developers and expect the rest to do the same amount of work. If the rest can't do the same amount of work, fire some more. Sooner or later you'll be down to the really productive ones. (note to the sarcasm-impaired: this is what's known as "sarcasm") – parsifal Mar 28 '13 at 20:22
  • 4
    Just use LOC, it's the one you can easiest fake. – thorsten müller Mar 28 '13 at 20:31
  • 3
    I would use total LOC added or removed to calculate the Iterative Refactoring Factor. – JeffO Mar 28 '13 at 21:12
  • @JeffO lolz!... – ozz Mar 29 '13 at 09:11
  • As a developer, the reason I was against any metrics when I was younger, because I wanted to keep what I do as black art, so I would always have a way out by saying things like "it is compiling" when the project/task was behind schedule. Look at most non-answers in this thread. Basically trying to find any excuse against any metrics, because metrics usually make Devs accountable. There are definitely some metrics you can use, but it is not exact science. It can vary by team, projects, etc.... See my full answer below. – Mag20 Mar 29 '13 at 11:09
  • @Mag20: As noted in my answer, visibility is definitely a thing to strive for, but it's not measured in numbers, it's done using standups and retrospectives and work boards. If developers are hiding something or, worse, if developers are working hard and managers THINK they're hiding something, visibility needs to be improved. – pdr Mar 29 '13 at 11:40
  • Yes we have retrospectives and work-boards. But what used to happen is: - Business Person: "So, Peter how its going?" -Peter(Developer): "It is compiling..." -Business Person: "Sounds like everything is hunky dory, just make sure you don't slip this project 2 weeks, like you did last 3 times." – Mag20 Mar 29 '13 at 11:44
  • @Mag20: Which is why we do these things at regular intervals and not just taking a snapshot in time ("what happened yesterday / last week?" vs "what's happening right now?") You can't claim something was compiling all day every day. And then, if a project slips, the reasons should come out at retrospective and be fixed. If they're not fixed, you expect the same delays to be repeated. (The business can make that decision.) If they are then said developer needs to have a different reason every time. (Which they might, validly, but probably won't, invalidly.) – pdr Mar 29 '13 at 11:48
  • Developers can and do claim it is compiling every day or they are fixing "Null exception", or they are almost almost done, just need to one more fix. Not sure exactly why, but if asked, Devs will paint you the most positive picture which has almost 0% chance of happening. Until we introduced metrics, our projects slipped almost every time. How exactly can a business person actual verify anything developer says? Most of the time Devs use tech jargon, that PMs just ignore. – Mag20 Mar 29 '13 at 11:53
  • @Mag20: Confirmation by peer. That's the point of a daily standup meeting and iteration review. If your entire team are slackers, you're screwed anyway. Start another company and hire better next time. But if one developer is a hard worker and the rest aren't, that one will eventually start asking questions. (Much more likely, you will have just one slacker, who is doomed.) – pdr Mar 29 '13 at 13:44

6 Answers6

30

I try very hard not to write non-answers on this site but I do believe that, in this case, I have to. It's the only right answer. But I'll try to help you out with more than a quip and a "you can't."

In all seriousness, there is no valid measure of developer productivity. I know this is hard for managers to cope with, but it's a fact. Refer them to a few links from people very experienced in the field. For a couple of examples:

Martin Fowler

So not just is business value hard to measure, there's a time lag too. So maybe you can't measure the productivity of a team until a few years after a release of the software they were building.

I can see why measuring productivity is so seductive. If we could do it we could assess software much more easily and objectively than we can now. But false measures only make things worse. This is somewhere I think we have to admit to our ignorance.

Joel Spolsky

Let's start with plain old productivity. It's rather hard to measure programmer productivity; almost any metric you can come up with (lines of debugged code, function points, number of command-line arguments) is trivial to game, and it's very hard to get concrete data on large projects because it's very rare for two programmers to be told to do the same thing.

Also ask them who is responsible for that increase. What measures are they allowed to take?

It is my experience that managers set these goals because they have zero clue what goals to set with respect to development teams. Maybe you can help them out with that.

Explain to them that you (or the team) want to take your targets seriously, but they have to be SMART or they're meaningless. Suggest to them some targets which are SMART. do you have a build/CI server? If not, setting one up is a SMART goal. If so, do you have some way of displaying quality statistics? If not, setting that up is also a SMART goal.

If so, then you have something that's very measurable: code quality. Maybe bringing your technical-debt rating down is a SMART goal, which will in turn improve productivity, unless they're assuming that people are slacking off, in which case you have an entirely different problem to solve: visibility.

Help them to give you targets you can actually achieve. There's no satisfaction in having goals that cannot be proven or disproven a year from now, or where you'll be wasting time gaming the system rather than improving it.

Community
  • 1
pdr
  • 53,387
  • 14
  • 137
  • 224
  • http://pm.stackexchange.com/questions/8968/how-can-i-track-the-work-done-by-a-remote-team-of-coders/ – psr Mar 28 '13 at 20:25
  • 1
    The answer is not to come up with a measurement. – JeffO Mar 28 '13 at 21:13
  • `bringing your technical-debt rating down is a SMART goal` +1 – Kolyunya Mar 29 '13 at 06:36
  • Code quality is subjective. I've seen beautiful code that is crap and disgusting code that is great. Technical debt is also subjective.Technical debt is an abstract concept about how far off from "ideal" some code is. "Ideal" is subjective. It is common for some programmers to peer program some code, have it reviewed, think it's nearly perfect, then they have me look it over and I find all kinds of issues. The fact that I can find issues that most others can't proves that not everyone sees the same issues, therefore proving that code quality and debt is subjective. – Bengie Mar 07 '19 at 13:11
12

One of my most productive days was throwing away 1000 lines of code.

    — Ken Thompson, designer of the original Unix operating system

Measuring software productivity isn't really that difficult, although it's somewhat imprecise. Survey your programmers and ask what percentage of their working time is wasted on non-productive tasks, and what those are. Have them focus not on personal productivity issues, but on non-productive assigned work tasks. Some examples are:

  • Waiting for slow source control updates
  • Requirements changing
  • Too many interruptions
  • Compiles are too slow
  • Performing regression testing
  • Marketing cancelling projects after you've put significant work into them
  • Spending too much time doing status updates
  • Spending too much time on unexpected bugs reported from QA or the field
  • Waiting on dependencies from other teams
  • Unraveling spaghetti code

However, due to an observer effect, it's impossible to tie those survey results to any sort of reward or consequence to the programmer, including having management angry that productivity didn't increase 15%. They will game the system even if they suspect that's how the numbers are used, and you end up with a completely useless metric.

What you can do is use the responses to move productivity in the right direction. Pick the worst two or three and put a lot of effort into it. Improvement can easily be measured numerically for many of the individual complaints, and hopefully when you do the survey again next year, some of the complaints will no longer be listed, or at least be reported by fewer of your programmers.

In other words, you can observe movement in the right direction, even if you can't quantify it.

Karl Bielefeldt
  • 146,727
  • 38
  • 279
  • 479
  • 1
    If you wish to add a link to an anecdote about negative lines of code - http://folklore.org/StoryView.py?story=Negative_2000_Lines_Of_Code.txt –  Mar 28 '13 at 21:57
1

Instead of trying to focus on metrics for programmers which as others have suggested are incredibly imprecise, try suggesting non-code things like the following:

  • XGb memory for everyone's PCs (where x is more than you have) and quad core
  • 2 monitors for all (3 if you already get 2)
  • Desks away from loud employees like managers, support, sales that are always on the phone
  • better specs
  • Less context switching between multiple projects, or because of changing specs
  • Less meetings, and better focus/consideration for staff being invited to meetings

Explain in detail why each of these things will actually make a programmer more productive.

These are the types of things that can drive programmer productivity, not trying to set some unachievable targets around LOC, or bug count.

ozz
  • 8,322
  • 2
  • 29
  • 62
  • 4
    Your answer will create or further an antagonistic relationship between the two groups. You have some decent points in suggesting alternatives to improving overall productivity, but your suggested approach is wrong. –  Mar 28 '13 at 20:30
  • 1
    I dont see it as that at all. Maybe "demand" is the wrong word, but this IS exactly the type of thing that will help productivity. I see it more as guiding them in the right direction. – ozz Mar 28 '13 at 20:38
  • To be more clear; your answer would be good if you said "Go to management and point out why it's difficult to measure as they're asking. Suggest instead that they try any of the following ... to improve productivity." That moves towards a collaborative approach instead of antagonistic. It probably wouldn't hurt to link to the Joel Spolsky test as well. http://www.joelonsoftware.com/articles/fog0000000043.html –  Mar 28 '13 at 21:08
  • And if you increase the things that make you more productive by 15%, then productivity must go up 15% as well. It's simple math. – JeffO Mar 28 '13 at 21:14
  • 4
    Ok, I've tried to tone my answer down a little :) – ozz Mar 29 '13 at 07:44
1

I agree with pdr's answer. That's the way to go. However, here's another take.

Most process improvement efforts need to start, as you have, with deciding what to measure and then baselining current operations.

What to use as a baseline?

How about Joel's twelve questions?

If you're not already doing 11 or 12, then you only need to do another 2 and you get your 15% improvement.

Community
  • 1
Peter K.
  • 3,828
  • 1
  • 23
  • 34
-2

In my opinion, for programmers there's only 1 metric that makes sense: the number of developers who leave to do similar work for another company.

If one programmer is unhappy enough to find a job elsewhere, it's very likely that a lot of other programmers are also unhappy and performing badly.

Brendan
  • 3,895
  • 21
  • 21
-2

I think it is difficult to do, but possible, if you use good case management software like FogBugz and keep track of old data. Obviously you can't say that we will improve performance by an exact number like 15%, but you can definitely do something about it. So let's say you have:

5 developers

1 month release cycle

These have to be pretty consistent from release to release, otherwise accounting for those can get messy. Here are some things you can measure and improve.

-- Estimates vs actual dev time.

Similar to Agile velocity. If developers are planning to work whole month let's say 20 days or 100 days for 5 devs this month. They estimated all their tasks and told management that they can fit all the features. But it actually took 120 days instead of 100 days.

the 20 days (4 extra days for 5 devs), developers where under a lot of pressure and wrote a lot of spaghetti code.

So to improve performance - increase estimates for next month by 20% and see how much closer it becomes to actual time. Keep updating it for future releases if it is still way off the mark. The performance improvement will not be 20%, but should be substantial, since there is going to be a lot less stress, bugs, and working weekends.

-- Amount of time spent by developers for fixing bugs found by QA

Let's say for this rollout, it took 5 man/days for QA to test.

You can try to improve this number by having less bugs:

  1. Introduce Dev automated testing
  2. Make sure to allocate time for Refactoring and optimization before QA gets to it
  3. Code reviews

Keep measuring bug fixin time every release and you should see improvements if you introduce the things above.

-- Do some research on bottlenecks to development

You may find that there are other things, like communication between Business Side and developers, that are having a huge negative effect on performance. Many times a project can be delayed because of scope creep or feature requirements not clearly communicated. This usually happens over and over again.

This one is a bit hard to measure, but use can use measurements from 1. for this.

-- Type Faster...

Words per minute is a proven way to measure performance not only for developers but also anyone else using computers.

-- Measuring dev performance ain't easy

As other people already mentioned it is not easy thing to do, so how you communicate it to "upper management" is really important. They need to understand that measuring development is not like measuring sales. There is no hard numbers, but there are definitely some things you can improve. So you can give them a plan with some of the improvements and use measures from step 1. and 2. to demonstrate if there is improvement.

Mag20
  • 3,281
  • 2
  • 23
  • 21
  • 2
    The first measures estimation accuracy. The second estimates quality and encourages perfection to be the enemy of good enough. The third is a good practice but not a measurable metric; he didn't ask how to improve performance, he asked how to measure it. And typing faster would be all well and good for secretaries but most of our bandwidth is spent thinking and communicating, not typing. – pdr Mar 29 '13 at 10:03
  • Estimation accuracy is directly related to productivity. Second one has nothing to do with perfection, but improving code to lessen QA time, which is directly related to productivity. Third one - true, was trying to help a little too hard. Fourth one "Type faster..." was a joke, tough crowd around here. – Mag20 Mar 29 '13 at 10:41
  • type faster? this assumes all written code is equal. What about skill level, different ways of doing the same thing, different design patterns, and does it take into account deleting of code? – ozz Mar 29 '13 at 10:43
  • Not sure why I am getting down-voted. My 1st suggestion is actually the top answer to similar/duplicate question: [link](http://programmers.stackexchange.com/questions/26596/metric-by-which-to-hold-developers-accountable) – Mag20 Mar 29 '13 at 11:00
  • 1
    @Mag20: You really don't see how, if you measure developers by the number of issues that hit the QA team, they will strive for perfection and thus reduce productivity? And your linked answer explains more clearly that a) it's not a measure of productivity, and b) how easily it will be gamed (and thus how useless it really is). – pdr Mar 29 '13 at 11:15
  • Not number of issues but time spent on fixing issues. You can have 10 bugs fixed in same amount of time as 1 bug. If number of time is less - code quality is better, thus productivity is better. As for estimates, yes anything can be gamed. But I would argue that developers would actually strive to do a good job with these, since providing better estimates is good for both dev and business side. The guy in link says it will not increase productivity if deadlines are phoney. So your a) is actually caused by b) above. – Mag20 Mar 29 '13 at 11:17
  • @Mag20: Most bugs can be fixed in very little time. Finding them, on the other hand, can take days. And that has very little to do with code quality, nor is quality a measure of productivity, though it is a factor of productivity. I can have perfect code and yet never deliver a single piece of functionality, thus never introduce any bugs -- zero time spent fixing them, I win! – pdr Mar 29 '13 at 11:27
  • @Mag20: As to the other answer, he actually states that it can't be done and says that this is the closest he can think of. You have stated it's difficult but possible, which is a very different promise. I'll quote Martin Fowler again: False measures only make things worse. That said, I haven't upvoted his answer; I think it's misleading. And I haven't downvoted yours, although I think it's even more misleading. – pdr Mar 29 '13 at 11:31
  • I guess with my next comment we will probably start going in circles. Let's agree to disagree. These metrics have worked for my company so your experience could be different. Thanks for good discussion though. – Mag20 Mar 29 '13 at 11:34