9

A friend's father, who is a software engineering manager, said, emphatically, "The number one cause of scheduling overruns is scheduling pressure."

Where does the research stand? Is a moderate amount of scheduling pressure invigorating, or is the manager I mentioned right or wrong, or is it a matter of "the more scheduling pressure you have, the longer the delivery time and the more TCO?" Is it one of those things where ideally software engineering would work without scheduling pressure but practically we have to work with constraints of real-world situations?

Any links to software engineering literature would be appreciated.

Christos Hayward
  • 311
  • 2
  • 6
  • What does TCO mean in this context? – Andres F. Sep 22 '12 at 17:04
  • I'm assuming that TCO stands for [total cost of ownership](http://en.wikipedia.org/wiki/Total_cost_of_ownership). Is this correct? – Thomas Owens Sep 22 '12 at 19:44
  • @ThomasOwens So I guessed, but does it make sense in the context of project schedules and budgets? I thought TCO referred to _ownership_ of a product, not development. – Andres F. Sep 22 '12 at 22:32
  • @AndresF. My understanding is that it's comprehensive - design, development, shipment, maintenance and upgrades. But I'm not an expert (or even have any measurable amount of experience) in the financial side of things. – Thomas Owens Sep 22 '12 at 22:55
  • @ThomasOwens Judging by that link from Wikipedia, that's not the impression I get. Development is definitely _not mentioned_ (do a search!), even though technology and software products are (and related concerns, such as deployment and maintenance). TCO is related to _ownership_, it even says so in the name! My understanding is that TCO is a consideration when choosing which product to _buy_, not which product to _build_. – Andres F. Sep 23 '12 at 16:25
  • My point is that when one is discussing software development, the keywords are "schedule", "deadlines", "budget", "risks", etc. Not TCO :) – Andres F. Sep 23 '12 at 16:29
  • TCO makes total sense in terms of the cost of a development unless you are silly enough to think that software stops costing someone money once the development is finished and it is shipped. It's more like a atomic energy - there's a legacy of toxic waste left over at the end of the development that is part of the TCO, but not usually factored into the development cost. Depending how well the implementation is done, depends how much the toxic waste is contained and how much the cleanup will cost, therefore the TCO can be reduced by spending more now, and increased by spending less (Schedule). – mattnz Sep 24 '12 at 04:01

6 Answers6

5

The number one cause of scheduling overruns is scheduling pressure.

I disagree. The number one cause of scheduling overruns is a schedule that doesn't reflect reality, and in an overly optimistic manner. Human nature dictates that some scheduling pressure is an absolute necessity. Just a couple of the problems that arise without some amount of scheduling pressure are interesting problems and "the best is the enemy of good enough." We technical people would far prefer to work on the problems that interest us rather than the problem that needs to be solved to get the product out the door. Take away deadlines (aka schedule pressure) and we will work on those interesting problems, to the detriment of the product.

Another problem is that the product needs to be "good enough". It does not need to be perfect. Engineers and scientists see the simplifying assumptions that aren't quite valid in some special case circumstances. Graphics designers see aliasing problems that are invisible to everyone else. Programmers see warts in their architectures that have zero impact on the product's behavior. "The best is the enemy of good enough," which means that sometimes we have to live with those problems that aren't really problems.

Lack of scheduling pressure will lead to a product with a very high cost of ownership. What causes overruns is bad schedules. This can come in a variety of forms. Underestimating effort needed, failing to account for dependencies, adding people to an already late project. Just to name a few.

David Hammen
  • 8,194
  • 28
  • 37
  • +1 Also, scheduling "pressure" sometimes -- though not always -- reflects real business concerns. No way to avoid that. "Whenever it's done" is not an acceptable due date for many projects. In fact, if all that can be promised as a target date is "whenever", then an acceptable possibility is to just cancel the project. – Andres F. Sep 22 '12 at 17:10
  • Steve McConnell enumarates "Overly optimistic schedules" as one of the classic software development practice mistakes, and a major source of project problems; this would be the cause of scheduling overruns in the first place. http://www.stevemcconnell.com/rdenum.htm – Only You Apr 24 '13 at 16:09
2

the more scheduling pressure you have, the longer the delivery time and the more TCO?

Well, any scheduling done by manager without discussing it with technical lead is prone to that. It is very obvious truth that Scheduling or estimations that are NOT based on facts are prone to failure.

In addition, managers who avoid Evidence-Based Scheduling are also moving toward next failure of their project. There are number of studies on this topic and metrics based scheduling is the right way to follow.

Yusubov
  • 21,328
  • 6
  • 45
  • 71
2

Time, quality, resources, and the number of features are all connected. You can fix any three of them, and get the last one as the output of your scheduling process.

The way your question is formulated implies that the time is your variable, and the other three (the quality, the resources, and the number of features) are all fixed. The question may benefit from changing the perspective a little by fixing the time*, and letting the quality float.

Now your question becomes this: "Do timing constraints negatively impact quality"? The answer to this question is a resounding "Yes": research confirms that people do worse under pressure on math-related problems** that they have not extensively practiced before (i.e. attempted fifty times or more), so your friend's father is right.

* A top manager at one of my previous companies used to say that the time is an input to scheduling process, not its output. He was letting the number of features float, though, insisting on the uniformly high quality of the deliverables.

** There is an implicit assumption here that programming is similar to doing math; I think this assumption is fair.

Sergey Kalinichenko
  • 17,393
  • 4
  • 57
  • 73
2

Right to left scheduling.

Someone in management always thinks they are Steve Jobs with his famous reality distortion zone. Until someone in product development educates them, non-technical managers may often have a view about scheduling that is as sophisticated as the early French film "Le voyage dans la lune" ("A Trip to the Moon") was for rocket science.

The problem has been around for a while. Fred Brooks talks about gut-less estimation in Mythical Man-Month. Barry Boehm talks about it in his proposals for a Theory-W approach to management. More recently, Steve McConnell (author of Code Complete) brings the concept of principled negotiation issue into focus in "How to Defend an Unpopular Schedule".

Agile pushes the scope of projects to a place where it is highly visible. The Agile Manifesto calls for "Customer collaboration over contract negotiation". It also, I hope, empowers the people who are held accountable. The planning game avoids non-technical stakeholders coercing developers to promises they made long ago that have been overrun by changes in requirements or new discoveries.

If your organization rejects agile, there are great methods related to calibration of estimates to rebaseline a schedule based on earned value. I don't think earned value does a great job with some of the real problems with prediction, but it can help disspell delusions about the velocity of projects and the harping on the estimates we have as somehow being facts.

There is a saying that the sooner you start to code, the longer it takes. Schedule pressure can have the effect of forcing methodology change. Sometimes it is from waterfall to "code like hell". This can have negative impact on quality, not to mention morale when workers can't do their best and their peers and future maintainers see them at their worst, not their best. In an environment like this, some degree of the chaos can be controlled with source control, daily build and test (or continuous integration and unit test), code reviews, using an experienced and highly skilled team, resisting the temptation to add staff late in the project, and the old standby, overtime.

Other times the methodology change might be from waterfall to iterative incremental. My experience has been that management was slow to embrace Agile. But then after a while there was new management that was more supportive toward Agile. Time boxing can be like budgeting - it can force us to think about the best use of a limited resource. Scrum has two time boxes - one is daily for feedback between team members, the other is monthly for the sprint through the burn down list.

Scrum diagram - Creative Commons Licence see

Creative Commons License - see http://en.wikipedia.org/wiki/File:Scrum_process.svg

DeveloperDon
  • 4,958
  • 1
  • 26
  • 53
1

You don't need software engineering literature. Conceptual probability and statistics, from undergrad, is all you need.

An estimate is just that, an estimate. It is not precise, it is not guaranteed. For any estimate, there is some probability that you will underrun it, or hit it on the nose, and some probability that you will overrun it.

Probability 101: p(underrun or hit exactly) + p(overrun) = 100%.

A schedule based on an estimate has the exact same characteristics.

You cannot eliminate the uncertainty completely. There will ALWAYS be some probability of overrun. It might be small, the probability of Iran nuking your office building, but it is still there. The best you can do is look at EVERYTHING, and reduce the uncertainty as much as you can. Once you have done that, you will, if you are lucky, have a schedule with small uncertainty, and 50% probability on each side.

Now, think about it: If you pull the schedule in, the probability that you will underrun or exactly hit the schedule decreases. The total still has to sum to 100%. Where does that probability go? Answer, it goes into the overrun probability.

General Dynamics / Fort Worth Division learned this one the hard way. They did their initial estimate for F-16C/D development, and sent it up the food chain. Somebody higher up arbitrarily cut a year out of it, and sent that to the Air Force. As a direct result, GD/FW was a year late to flight test, and the Air Force was NOT happy. (Note that "one year late" was according to the revised schedule, meaning that the original schedule was RIGHT ON TARGET.)

John R. Strohm
  • 18,043
  • 5
  • 46
  • 56
  • best answer so far - Scheduling and Estimating are two completely different issues. Too many people fail to grasp it. – mattnz Sep 24 '12 at 03:54
1

I think that a certain amount of pressure in a project is OK because it helps to maintain focus.

However, if the pressure is not realistic, or if the communication between management and technical people does not work properly, yes, there is a the risk that scheduling pressure results in bad quality and / or late delivery.

An experienced developer will know that he / she is not expected to produce the perfect solution but rather a solution that is good enough. So the estimate given by such a developer will already reflect their understanding of what is good enough for a particular project.

There many factors that influence the definition of good enough.

For example, how many months does the project last? If the project lasts one year, you can write a prototype of that particularly difficult module rather quickly at the beginning of the project and then you have several months to test and debug it as a side activity to the development of other, more routine modules. (You can let that module ripe over several months until it is good enough so you do not need to try and get it right from the very beginning.) I find this strategy very effective but you need a manager that trusts you and will let you keep a project open for months. Another (distrustful) manager might push you to finish that module as soon as possible (no matter if you'll have to fix it later and if this approach will eventually cost much more time in total).

Another example: the project is for a product that will have only one release. Then you can try to get it done quickly and rely on extensive tests to ensure that the product works as expected (quick and dirty is good enough). On the other hand, if the product is going to have two or three releases, better spend some time on designing it, to avoid extensive rewriting of the code for later releases. (In this case, quick and dirty is not good enough because the total development time over the three releases is bigger.)

Bottom line, I think that bad communication between management and technical people and a lack of a common understanding of what is good enough for the project at hand can lead to excessive scheduling pressure, resulting in bad quality / late delivery.

There is never enough time to do it properly the first time, but there is always enough time to fix it later.

Giorgio
  • 19,486
  • 16
  • 84
  • 135
  • +1: "There's never enough time to do it properly the first time, but there is always enough to fix it later." That question has been on my mind about whether taking twice as long initially to do it right, plus moderate time addressing defects, has a substantially lower TCO than a rush job that takes less than half the time initially, and then facing the music in the consequences of a quick rush job at first. – Christos Hayward Sep 23 '12 at 18:52
  • As I pointed out, if you only have one release and you have a good testing department, your product can be OK even if you save time on coding: the code might be messy but thorough testing ensures that it works as expected. But if you have subsequent releases on the same code base, you may need to rewrite a lot of the code for the second and third release. In the latter scenario you may save time over several releases by designing your code more carefully the first time. – Giorgio Sep 23 '12 at 19:18