12

Here is a quote from a training manual at work regarding SLIM and software estimation:

Notice also, there is a correlation between Effort and Defects. This means, the more people there are assigned to a project of a given size, the more Defects there will be.

Effort is person-time (person-years, person-months) for the project. Defects is the count of defects detected at any point in the lifecycle. Size is defined as the use cases, function points, or SLOC that compose the project.

This seems counterintuitive, assuming a good process and capable engineers. For example, having more people means more eyes on all of the artifacts (requirements specs, designs, code, tests). Aside from having more eyes, my intuition suggests that there is little relationship between effort and defects on a project that utilizes appropriate quality techniques.

I haven't been able to find any documents, aside from those about the Putnam Model (which is used by SLIM), that suggest any kind of known relationship between defects and effort or defects and number of people on a project. Is this a known relationship, and is the assertion that "more people = more defects" valid?

Thomas Owens
  • 79,623
  • 18
  • 192
  • 283
  • 10
    "adding manpower to a late software project makes it later" - Fred Brooks – Oded Sep 30 '11 at 13:07
  • 2
    @Oded Adding people late is not mentioned at all. Also, Brooks' law says nothing about defects, but an increase in communication channels to make decisions and keep people informed. I would suspect that, like Karl Bielefeldt suggests, communication difficulties might lead to defects. – Thomas Owens Sep 30 '11 at 13:26
  • @ThomasOwens - Yes, the quote is indeed about the increase in communication channels (and hence difficulties), with the assumption that this would lead to defects. – Oded Sep 30 '11 at 13:28
  • I would still say that when loads of developers are thrown on a project, it is often indicative of a death march. – Morgan Herlocker Sep 30 '11 at 13:43
  • @ironcode The number of developers on a project should be dictated by the size and scope of the project, and how it's organized. Having too many developers or adding too many developers later are signs of poor management, perhaps a death march. – Thomas Owens Sep 30 '11 at 14:07
  • Any way of comparing projects of the same size but not same number of developers? – JeffO Sep 30 '11 at 16:31
  • @Jeff Haven't found anything in regards to research studies on that. Most of what I've found compares complexity and size to defects, not development staff size to defects. There's got to be something, though. It seems like a fairly common question. I'll keep poking after work. – Thomas Owens Sep 30 '11 at 17:36
  • @ThomasOwens - Could you define Effort according to your source? Does it just mean number of people, or is it man-hours? I assume it means number of people and size of project means man-hours, but it's not fully clear and it would affect the answer. – psr Sep 30 '11 at 18:08
  • @psr Effort is people-time (person-years, person-months, etc.) applied to the project. I'll edit my post to also reflect this. – Thomas Owens Sep 30 '11 at 18:16
  • @ThomasOwens - Sorry, that's what I thought project size was. So what's project size defined as then? Presumably they have a way to know if 2 projects are the same size to be able to make such a claim. – psr Sep 30 '11 at 18:34
  • @psr Size is any of a number of measures, according to this text. It's use cases, function points, SLOC, and a few others that I've never heard of before. I'll also add that to the post. However, there are no hard numbers as to "this project with n engineers and m size had x defects, while this project with c engineers and d size had y defects". – Thomas Owens Sep 30 '11 at 18:37

7 Answers7

14

My intuition goes like this:

The more people assigned to a project of given size, the bigger the communication overhead
=> the higher the chances of misunderstandings and all sorts of communication problems
=> the higher the number of resulting defects.

And

Good developers are rarer, thus harder to find and hire, than mediocre / bad ones
=> the more people assigned to a project of given size, the lower their average level of competence
=> the higher the number of resulting defects.

But these may be just my reasoning, I have no supporting evidence.

As a side note, your assumptions emphasized below are IMHO (sadly) quite strong, taking into account the state of our profession:

This seems counterintuitive, assuming a good process and capable engineers. [...] my intuition suggests that there is little relationship between effort and defects on a project that utilizes appropriate quality techniques.

Péter Török
  • 46,427
  • 16
  • 160
  • 185
  • I attributed McConnell's Thrashing/Process/Productivity graph to solve that. If you don't introduce new people, you get used to the communication overhead involved in the project early and maintaining the appropriate communication becomes easier and more natural. This breaks down under Brooks' Law, when you add people to a project late, which would be the same as introducing process late to a project - an increase in communication channels means an increase in thrashing and a breakdown of communication that leads to defects. I might be off base on that, though. Your reasoning might be valid. – Thomas Owens Sep 30 '11 at 13:22
  • But with fewer people you're less likely to allow them to stick to their strengths. Is it easier to hire a few good developers who may actually be better if they can focus on an area instead of just one guru who can do everything? – JeffO Sep 30 '11 at 14:50
  • @Jeff O, you have a point. OTOH if each developer focuses only on his/her own strong area, communication between them may get even more troublesome. – Péter Török Sep 30 '11 at 14:54
  • 1
    Is the communication more troublesome or just becomes required? – JeffO Sep 30 '11 at 16:23
  • @Jeff O, IMO the less they understand about each other's area, the more communication is required, and the higher the chances of misunderstanding and false assumptions. – Péter Török Sep 30 '11 at 19:49
5

It could just be a correlation. Management tends to assign more people to help on projects they deem will be more complex, or projects that are falling behind due to lots of intransigent bugs, or projects that require a lot of coupling between various components. If you could take management decisions out of the mix, I suspect that correlation would at least diminish, if not reverse.

Karl Bielefeldt
  • 146,727
  • 38
  • 279
  • 479
  • The only problem with this is that change (specifically an increase) in staffing over time is not mentioned. It just says that if you have a project with n people, you will have m defects. Adding people does introduce communication overhead and problems, but that (from what I can tell) is beyond the scope of the simple relationship between people-defects. I do agree that a side effect of adding people late is not only an increase in communication channels and the need to train people and bring them up to speed (Brooks' Law), but also a potential increase in defects. But that's out of scope. – Thomas Owens Sep 30 '11 at 13:38
  • Adding people late is only one factor I mentioned. Management still has a tendency to assign more people _up front_ to projects they anticipate being more risky or complex. – Karl Bielefeldt Sep 30 '11 at 14:46
  • The point of SLIM (and other estimation tools, when used properly) is to help with estimation of the needed number of people, cost of a project, time required, and so on. However, that does require the tool to be understood and used properly. – Thomas Owens Sep 30 '11 at 16:56
3

Given the recently updated definitions of size and effort, I would suggest that perhaps the results are due to the fact that Effort (total man-hours) is actually a better estimator of true project size than the measures the source is using (Effort would be a perfect measure if all teams and team's resources were equivalent).

Therefore what is really happening is that larger projects have more defects, which is not surprising at all.

psr
  • 12,846
  • 5
  • 39
  • 67
2

This seems counterintuitive, assuming a good process and capable engineers.

I do not think you can assume either of these in the real world. The more people on a project, the more likely you are to have bad apples who cannot keep up and will slow down the best developers. Even if you go with the assumptions there are a few other things that slow down projects and cause more defects as you increase the number of people:

  • communication overhead
  • code reading overhead (by this I mean that even the best devs take more time to read and consume other people's code than their own)
  • testing must be more thorough (We all shoot for 100% test coverage, but that rarely happens in the real world. The more people you put on a project, the scarier refactoring gets without extremely thorough automated testing, since you are just hoping your changes will not have side effects. Not all tests can even be automated in any reasonable manner, which takes even more time.)

In my experience the negative effects of projects loaded up with developers goes way down when the project is very modular and you have just 1 or 2 people per tier. I don't care what version control system you happen to be using, having 4 developers all auto-merging checkins to the same file at once will probably be a bad idea.

Morgan Herlocker
  • 12,722
  • 8
  • 47
  • 78
  • If the only way to prevent 4 devs working on the same file is to limit the team size to 3, you got bigger problems. – JeffO Sep 30 '11 at 14:59
2

There's a difference between correlation and causation; the quote seems to be saying that the total number of defects tends to be higher for projects where larger numbers of engineers are allocated. This makes perfect sense to me, I am sure MS Windows has more defects than the applications that I create, but that doesn't mean that my applications are superior quality.

To give another, more abstract example - if we took the total number of deaths per country and correlated that with the total number of doctors in the country, I am sure we could say "countries with more doctors had more deaths". This would not be because the doctors caused the deaths, but rather that large numbers of doctors are indicative of a large population.

Daniel B
  • 6,184
  • 1
  • 22
  • 30
  • For your example with Windows, I'm sure that Windows also has more opportunities for defects simply because it's larger. If one developer wrote a system that was 10 SLOC and a system that was 10000 SLOC, the system with 10000 SLOC would have a greater chance of having a defect (which includes typos that prevent compilation, missing semicolons, logic errors, and so on). Typically, larger projects would have more engineers, but the relationship isn't between the number of people and defects, but size and defects. – Thomas Owens Sep 30 '11 at 14:16
  • @ThomasOwens - yepp, that was my point exactly. – Daniel B Sep 30 '11 at 14:17
  • Why wouldn't errors be compared relative to the size and complexity of the project? – JeffO Sep 30 '11 at 14:52
  • @JeffO - Measuring it in a relative way is completely non-trivial (how exactly do you do it?). Perhaps the original statement is being taken out of context, but authors rarely refer to complex, calculated results simply as "defects". In such a case, I would expect another word like "quality" (which implies that some calculation was done), or a longer phrase like "defects per assigned engineer". Perhaps I am being a bit cynical towards the authors here :) – Daniel B Sep 30 '11 at 15:13
  • @Jeff They would be. But you would compare defects to the size and complexity, not to the number of people. As size and complexity increase, defects increase **and** number of people increase. So yes, although the number of people do increase, that increase in people doesn't increase the number of defects. – Thomas Owens Sep 30 '11 at 15:18
  • @Daniel My quote is taken in it's full context. It's a bullet point on a slide and the presenter notes do not address it. People who were at the presentation also can not explain it, and tell me that it wasn't addressed there, either, but see my point about complexity driving up number of people and number of defects, rather than number of people driving up number of defects. – Thomas Owens Sep 30 '11 at 15:20
  • @Thomas I don't think people on a project increases proportionately with size and complexity. It may eventually, but then it's too late. Usually the understaffed team continues to push too hard before the problem gets noticed. – JeffO Sep 30 '11 at 16:29
  • @Jeff It's not mentioned as a proportional increase, and I don't think I every suggested it was. Part of the point of using SLIM is to estimate duration, effort, and cost of a project. – Thomas Owens Sep 30 '11 at 16:45
1

Let's put aside the assertion about the number of people for a moment.

Looking at the number of defects injected as a function of effort might make sense as long as you assume that increased effort necessarily requires increased size, as there is a strong correlation between the number of defects and software size.

So if you assume that the more effort that is put into a project, the more lines of code are written, then you could probably use effort as a proxy for size to predict the number of defects. Correlation between size (e.g. LOC) and the number of defects has been shown time and again in work by Watts Humphrey, Capers Jones, and others.

I don't see how the number of people fits, other than more people implies more effort.

As a side note, don't confuse correlation with causality. While there is a correlation between size and the number of injected defects, size is not the cause. The cause usually comes from, as you've pointed out, people and process issues. That said, defects as a function of size is a great metric for understanding if there is a problem and for gauging the effectiveness of change.

Michael
  • 2,102
  • 16
  • 22
0

I'm assuming this is limited to members of the core programming team and not a situation where there are specialists like: UI, UX, DBA, etc.

I think it needs to be managed well, but that's not easy. Small teams made up of quality developers can manage themselves. It's more likely to avoid large sections of code created someone with less talent.

Having more team members can make it easier to divide duties. Put the better or more experienced devlopers on the difficult areas. Take away some of the mundane or non-programming tasks away from your better developers and let the junior devs handle the interuptions. This is going to take more planning and thought, but provides an opportunity to leverage your talent.

There's the notion that it's better to have a great developer who needs to pick up on a new skill than a below-average dev who already knows it. This is great if you have the time. Probably the reason more develpers are being assigned to a project is because of the amount of work required and the time limts. You may have someone who can focus on a specific area and become more skilled. It's great to have that well-rounded knowledge, but sometimes with a little direction, a lesser dev can take some instruction and run with it.

The reality is, there aren't a lot of people who have ever managed a large team on a successful project. Even if they are all talented, it's difficult for large teams to self-manage. Do egos get in the way?

JeffO
  • 36,816
  • 2
  • 57
  • 124