30

Do you know any Key Performance Indicators for Developers? What should be measured and monitored?

Walter
  • 16,158
  • 8
  • 58
  • 95
Raymund
  • 425
  • 1
  • 4
  • 5
  • 27
    When you start measuring metrics, you get people aiming to maximize your metrics, instead of actually maximizing their productivity. If you hire good people, they'll get work done; if you hire bad people, metrics won't help as much as you'd like. – Anon. Jan 06 '11 at 03:55
  • 2
    Even good people need to know what is important though. For example, is keeping the site up 24/7 more important than new features? What about customer-support? There will always be more to do than your devs have time :) – Chris Smith Jan 06 '11 at 04:11
  • 10
    "Not everything that can be counted counts, and not everything that counts can be counted." - Albert Einstein – Jason Baker Jan 06 '11 at 09:36
  • 1
    @Chris Smith - But that's where good management and clear communication of the business priorities comes in. This needs to be communicated in advance. Metrics only measure things after the fact. – CraigTP Jan 06 '11 at 10:59
  • 3
    There's this classic dilbert cartoon (http://dilbert.com/strips/comic/1995-11-13/) that shows the risks with measuring and rewarding in the wrong manner. – Rob Moir Jan 06 '11 at 11:51
  • @CraigTP - and payment usually comes after the fact as well. Shouldn't there be some metrics to determine whether you were paid fairly and the client got value for the work done? Just as there should be metrics to determine what parts of a codebase are unsound and need of repair before new features are added so it can be determined what developers have been doing with their time or if repairs are warranted or the code should be replaced. Most of the fears expressed here are that metrics will be misused, but if we fail to define proper metrics bad ones are all we'll get. – Huperniketes Jan 06 '11 at 16:44
  • 4
    The basic problem with using KPI's for programmers is that's assuming programmers are like engineers when they are infact more like artists/craftsmen. It's like having KPI's for Michelango painting the sixtine chapell. "10 brushstrokes per minute, excellent!" – Homde Jan 06 '11 at 21:16
  • Programmers are Michelangelos? Really? Our work is judged primarily on an aesthetic basis, and not for its performance by a logical calculating machine? In fact our work should be calculating machines: finite-state machines which are nested to varying degrees to produce work in a deterministic order. It is on that basis that KPIs should be determined: how decoupled your FSMs are, and how well integrated they still are to produce the desired work. Craftsmanship is a silly romanticization to justify a lack of engineering discipline. – Huperniketes Jan 07 '11 at 12:48
  • 1
    @Huper: That's a bit idealistic, don't you think? In my observation, once a system gets to a certain size, history and level of integration with other systems, non-deterministic behavior starts to result in a big way. It's the art of the software engineer to deal with that randomness. No? – Paul Nathan Jan 07 '11 at 17:22
  • @Paul, the system cannot be random. It's why even random number generators are difficult and such an intense field of study. What is required is for process states to be well-defined and respected and for control flow between processes to be strictly contained in order to prevent them to be in an unexpected state. There is no randomness in the system. It is the product of prior computations. – Huperniketes Jan 08 '11 at 08:16
  • 1
    @Huper: Sufficient complexity is indistinguishable from randomness, to speak aphoristically. Further, network communication behavior can literally be random. – Paul Nathan Jan 08 '11 at 17:08
  • 1
    @Paul, engineering is a disciplined approach to building systems. Patterns for structuring the system such as layering, nesting, dependency injection, etc. were defined specifically to prevent the system and its components from becoming complex, let alone _that_ complex. Further, while the _sequence_ of network communication might be random to a member of that system, its state should never be. One might experience "random" events traveling from home to work, but it isn't okay to end up elsewhere or late. How is it okay for a _finite_ -state machine to be in an unknown or inconsistent state? – Huperniketes Jan 08 '11 at 18:28

8 Answers8

41

Consider the following truth: you will get exactly what you measure and monitor. With that in mind:

Terrible things to measure

Lines of code - Elegant code has a concise nature to it. Lines of code encourages bloat, copy and paste, or even worse, code for the sake of code.

Time-to-solution - Code done quickly contains lots of bugs.

Bug fixes - This goes with 'time to solution'. Don't reward programmers for writing slopping code, and especially don't reward them for fixing the problems they caused in the first place!

What, IMHO, you should measure

Impact. The only thing that matters is what your developers do. Did you write a tool that improves efficiency by 10%? What about automating a task that used to take 3 hours? What about refactoring that gnarly library so it is now easier to use for everyone on the team?

You should measure what happens after code has been written, and how valuable the contribution is towards your business/company's goals. Note it is possible to have a negative impact.

Chris Smith
  • 5,218
  • 2
  • 26
  • 29
  • 6
    I actually worked for a PHB once who used time estimates to fix bugs as a measure (along with the time taken on development work as well of course) of productivity. Try explaining to someone like that how a 2-line bug fix might have taken all day to track down. – Bobby Tables Jan 06 '11 at 06:12
  • 3
    +1 I love the first line. Reminds me of a quote from [this article](http://thedailywtf.com/Articles/Testing-Fundamentals-.aspx) (and it's The Daily WTF, so let that set the tone): _"When you require developers write more unit tests, you get exactly what you ask for: more unit tests."_ – doppelgreener Jan 06 '11 at 06:42
  • Can I work at your company? <3 – cvsguimaraes Jun 20 '15 at 02:47
33

https://i.stack.imgur.com/a0bTh.jpg

Which door represents your code? Which door represents your team or your company? Why are we in that room? Is this just a normal code review or have we found a stream of horrible problems shortly after going live? Are we debugging in a panic, poring over code that we thought worked? Are customers leaving in droves and managers breathing down our necks..

(Robert C Martin, Clean Code - book that opens with above picture)

gnat
  • 21,442
  • 29
  • 112
  • 288
Homde
  • 11,104
  • 3
  • 40
  • 68
  • 5
    +1. We had a discussion along these lines at my work a couple of years ago: the more comments in the code along the lines of "this is a horrible hack that I have to do because of X", the more productive that developer is. It generally means that they're working on real stuff which integrates with other stuff and often forces them into not being able to do things nicely. Hmmm, it's a bit of a you-had-to-be-there now that I think of it. But basically if you had no frustrations that led to such comments in your code on that team, it meant you weren't doing much (or didn't recognise the badness). – Bobby Tables Jan 06 '11 at 06:19
  • 3
    It's ok IMO to do ugly hacks to go quickly forward, on the condition that you continually go back and refactor them as appropriate, 3 steps forward, 1 step backward :) – Homde Jan 06 '11 at 06:27
  • 1
    It's too bad that a lot of the time "going back" never happens. – Jeremy Jan 07 '11 at 17:48
8

Assuming you've hired someone smart, they should be getting things done. Beyond that, when you measure employees (especially programmers), you get exactly what you measure.

In short, monitor that projects are getting done within the standards of the team.

Austin Salonen
  • 1,061
  • 8
  • 11
3

How about efficiency (work done/hr)?. This can be measured with the pomodoro technique. Take a look at this presentation.

Once you are tracking your pomodoro estimations and the actual executed pomodoros per task, you can then measure the reality factor, which this tool called pomodairo does for you. This reality factor can sum up how good are your estimates and how efficient you are; for evaluating efficiency I'd stick to no more than 2 pomodoro tasks.

I find pomodoros objective (on an individual basis) because each one mean a fixed ammount of focused work and they are a good measure for you to improve, which IMHO is the real purpose of any metric of these sorts that matters.

dukeofgaming
  • 13,943
  • 6
  • 50
  • 77
  • 2
    This would not be a good way to measure productivity, though. For example, under Pomodoro, you're supposed to discard your current pomodoro if you get interrupted. Going to help a coworker shouldn't mean that I'm screwing up my own productivity numbers on paper. – Adam Lear Jan 07 '11 at 16:08
  • 1
    Huh? All this measures is the accuracy of your estimation. – Tom Anderson Jan 07 '11 at 22:05
  • @AnnaLear You can always make smaller pomodoros if your position requires frequent collaboration. – dukeofgaming Dec 31 '11 at 00:11
1

We just went through this at my work. Trying to figure out what our KPIs should be, and then what our KPIs were going to be (since they turned out to be hard to measure).

THE KPI should be a measure of solving the business requirements in a timely fashion. If someone knows a good way to measure this let me know. :)

We ended deciding to use # of features implemented. Our customers sign off on every user facing feature and they are publicly prioritized (so people can fight for theirs to go first). We figured that was going to be a simple way to measure if we are performing well. We will see.....

dietbuddha
  • 8,677
  • 24
  • 36
  • You've got a great start to answering your own question here: a measure of solving the business requirements. So how do you go from defining the requirements to running a system that meets those requirements? – Huperniketes Jan 07 '11 at 13:11
  • 1
    Sounds like a strong incentive to break each "feature" down into pieces as small as possible! – Highly Irregular Jul 03 '13 at 21:23
1

There are almost no business metrics that you can use. You have to treat the programmer as a mathematician or scientist. The only thing that's measured there is the impact or potential impact of the ideas/code. Would you count the number of lines in a proof written a mathematician?

Even if you do use a measurement, it will be treated as an optimization problem by the programmer depending on if it determines their wages. Another point is that programmer productivity varies from day to day, some days there's a lot of lines of code written, other times no new design ideas are thought of, etc.

What are some of the key performance indicators for an executive or manager? Maybe you should try using those for programmers since we also deal with ideas and thought-stuff.

0

For an industrial programmer, $/hr saved (or earned) is, in my opinion, the only meaningful business metric.

Software types are notorious for having a gaming mindset (computer games, pen and paper, puzzles, etc). Don't incentive behavior that is not aligned with the business needs.

E.g., if you consider kLOC/week to be the key indicator, you're going to observe that for the same feature, you're getting many more kLOC.

Example:

int foo() {
   return blah;
}

Can well be turned into:

int
foo()
{
   return 
      blah;
}

For the same feature. That's a 2x LOC increase. If you rank my productivity primarily based on LOC, you better believe I'm going to evaluate whether I want to get that productivity-based bonus/promotion.

Paul Nathan
  • 8,560
  • 1
  • 33
  • 41
  • 1
    Not if you pipe it all through `indent`. – Jé Queue Jan 07 '11 at 18:49
  • 2
    @Xepoch - believe me when I tell you - there's more than one way to skin that cat, and lots of them won't be affected by a run through indent. The point is: measure KLOC, you'll get more KLOC, not more solved problems. – Michael Kohne Oct 01 '12 at 17:59
  • the management doesn't know how to use `indent`. They understand a table with numbers: 6000: wow!! 3000: boo – nonopolarity Nov 23 '16 at 20:55
0

I would say that you need to align metrics from the business units to the programming. Programmers don't want that because they feel the business unit will screw up and they won't look good.

My answer is: Dude--or Dudette--work with your business users and make awesome tools for them that will make them ultra productive and their success will be your success.

Christopher Mahan
  • 3,404
  • 19
  • 22