34

How does a Scrum team account for dev/infrastructure tasks in the planning meeting?

At first glance, they do not seem like user stories since they do not deliver end user value.

However, attaching them as tasks to a particular user story sometimes does not make sense either. For example, say the task is: "Setup Bamboo." That task is not required to complete any user story since the team could manually build and deploy. So attaching it to a user story does not make sense since this task is not required to complete the user story.

So, then, this suggests that these tasks become user stories. But then, if the team story points them, then this changes the velocity which is odd since the Product Owner wants to know the velocity against his backlog, not against his backlog with a bunch of technical user stories attached to it.

user11347
  • 773
  • 1
  • 7
  • 10

10 Answers10

27

They're not actually user stories. They're stakeholder stories. Unless the software is actually paid for direct by users, it's rare that a story is created entirely for their benefit.

I give you a couple of examples:

  • keyworded articles, which allow advertisers to have more effective adverts
  • CAPTCHAs, which are there to stop moderators having to deal with spam manually.

Most technical stories actually provide a business benefit, but it's rarely for the users. Phrasing them in a different way can help. I normally use Chris Matts' Feature Injection template:

In order to <achieve my goal>
As <the stakeholder who wants the goal>
I want (<some users to do>) <some stuff>.

This explicitly recognises all kinds of stakeholders, including the development team. Now you can phrase your technical stories too, calling out the business benefit:

In order to minimize the risk of deploying something broken
As the team deploying the code
We want to spend a few days on an automated deployment system.

I've written a couple of blog posts on this: They're not User Stories, and Feature Injection and handling technical stories. Hope they help.

Lunivore
  • 4,232
  • 1
  • 18
  • 23
  • 3
    Semantics...IMHO this goes against the Agile philosophy; adding needed separation where it provides no real value other then warm fuzzy feelings. – Aaron McIver Mar 24 '11 at 16:13
  • Hi Aaron, I don't really understand your comment. You say the separation is needed, but provides no real value? Which separation, and what aspect of Agile philosophy? – Lunivore Mar 24 '11 at 16:25
  • @Lunivore I'm referring to changing the verbiage with regard to the user story template; it's semantics which provides no real value over a typical user story template. – Aaron McIver Mar 24 '11 at 16:29
  • 6
    Are you speaking from experience, or theorising? I ask because I've used this template with a number of teams now, and found that putting the goal first *really* helps establish what's necessary to achieve the project vision. Mike Cohn uses "so that" optionally. I don't believe it is. – Lunivore Mar 24 '11 at 16:55
  • @Lunivore Experience. Who is it helping; versus the standard user story template? Product owner? Developers? – Aaron McIver Mar 24 '11 at 17:01
  • 1
    I see that this template is useful to help communicate the value of the technical task to be performed to the non-technical PO. There is a difference between "as a QA analyst I want a continuous integration server so that the application is tested every day automatically" and "In order to reduce the amount of manual testing needed at the end of the project, and the probability of an error slipping to production, as a QA Team we want to test up a continuous integration server". Showing the hidden business gives the OP enough information to decide whether to include it or not. – Soronthar Mar 24 '11 at 17:17
  • 1
    @Soronthar Where does it end then? "In order to reduce the frustration levels, as a development team we want make our own rules" It's circular in nature. That is one reason that you remained focused on the user and that is it. Tasks should be hidden from the PO; as the PO should not have to concern themselves with those details. – Aaron McIver Mar 24 '11 at 18:17
  • 1
    @Aaron It the ends when the technical task can be described with the same qualities as a story (think S.M.A.R.T). This is all about staffing and expectation: using one or more developers in the team for a technical task will decrease the Velocity in the eyes of the PO, and that needs to be explained to him. Think about it this way: if you were paying some external consultants by the hour by task, you would want to know why they want to play and spend time (with your money) on a CI server, don't you? – Soronthar Mar 24 '11 at 18:44
  • @Soronthar This is not about staffing. You are morphing it into a staffing component. This is about correlating work effort within the team to those who care. The PO does not need or want tasks explained to them. The PO wants to know when "it" will be done; "it" being the user story. The PO at some point has to assume the development team is self sufficient (Scrum proponent) and capable of their assigned roles within the team. This will allow the PO to focus on the product and not the CI server. This sounds more like a case of CYA then anything else. – Aaron McIver Mar 24 '11 at 19:46
  • @Aaron Think about it this way: The Plan is scheduled to finish by the end of next month, with 10 developers. They "discover" that CI servers exist and think it would be good for them to have one. Who decides that is it appropriated to divert developers effort into it, perhaps risking the delivery? The PM/SM. And with whom should he check if the schedule/scope can be changed? With the PO. Its about feedback, not CYA. Should the PO decide to "risk" not having the CI, so be it. Or the team can decide to implement it anyway on their "spare time". – Soronthar Mar 24 '11 at 20:45
  • This discussion can be tricky when thinking about infrastructure tasks. At least in my mind, developers should not be doing infrastructure in an software factory. But this is quite clear when talking about technical debt or technically enhancing the application. If the team discover that there is a change that could improve performance by 100%, they must present it to the PO stating the value to have the performance increased, the risks of doing it and the estimate, and the PO must decide based on that to include it or not in the current plan. – Soronthar Mar 24 '11 at 20:48
  • @Soronthar Fair arguments. There is significant caveat though. The majority of PO's I have encountered often have little technical aptitude; thus formalizing a CBA around why a CI server should be implemented goes completely against the grain of Agile. – Aaron McIver Mar 24 '11 at 21:29
  • 1
    Can I please recommend that the people now involved in this discussion go read something about Lean, feedback, project visioning, Feature Injection, MMF or BVI and maybe a bit of Kanban? It might not solve the question, but it'll help us have different arguments on different questions which will provide more information than trying to do this in a comment thread. Don Reinertsen's work is a good start, and if that's too heavy then just read Eliyahu Goldratt's "The Goal", which is a fantastic fiction book, let alone for teaching Theory of Constraints. – Lunivore Mar 25 '11 at 01:44
  • 10
    Oh, and just in case it wasn't clear - I care more about getting useful work done than I do about Scrum. Or Lean. Or BDD. I also believe that most useful work in software has to do with learning and managing risk. When methodology gets in the way of doing useful work, I tend towards the useful work. – Lunivore Mar 25 '11 at 02:06
12

Velocity is a measure of the team's capacity to do useful work (as opposed to Drag). Infrastructure tasks still deliver end-user value, albeit indirectly, by making the team more efficient in the long run. I have no problem tracking these things as user stories (the user is the dev team in this case) and prioritizing them appropriately. A Product Owner in good communication with the customer(s) should be able to figure out where such tasks can fit without disrupting deliverables.

Michael Kristofik
  • 1,503
  • 9
  • 15
  • 4
    I think it's dangerous for teams to blur the distinction between things that are directly valuable to the user and things that hopefully provide indirect value. In particular, a "everything we like is valuable" approach encourages developer gold-plating and infrastructure for infrastructure's sake. I strongly encourage people only count stories with direct business value toward velocity, because that's the only thing that customers will pay cash money for. – William Pietri Mar 24 '11 at 17:19
  • 3
    Waltzing with Bears. Everything you do that's truly valuable is mostly valuable because nobody has done it before (otherwise there are other, cheaper, ways of getting it done). Most of what we do that's valuable is about learning how to do the new things. The infrastructure tasks help us get feedback on the new things, and learn more quickly. I'm with @Kristo if it helps us learn more quickly. – Lunivore Mar 25 '11 at 02:10
  • 1
    @Lunivore - The difference is that nobody pays you for learning. They pay you for what you do with the learning. Teams should always take some time to improve their tools and their knowledge. But counting it as velocity confuses it with the kind of work that the team is there to do. – William Pietri Mar 26 '11 at 00:29
  • It's not just about tools and knowledge. Thought experiment from Ashley Johnson: Think about the last project you did. Think about how long it would take to do it again with the same people, same requirements, same technology, but having learnt everything you learnt. Quotes from PMs run at about 25% to 33% - the rest is how much learning we do in software projects. Read Dan North's post on Deliberate Discovery: http://dannorth.net/2010/08/30/introducing-deliberate-discovery/ – Lunivore Mar 26 '11 at 11:59
12

Do them gradually.

If no stakeholder wants it, don't make it a story. Just take care of it, a bit at a time. E.g., the first time you deploy manually. The second time, you automate a bit of it. The third time, you automate a bit more. Eventually, your build is not the biggest problem, so you focus on something else.

You will have more of these developer-focused tasks at the beginning, and that's fine; your velocity (as measured by stories) will just be lower. That's a correct representation of the situation. But you will always have some, so it's important for the team to get in the habit of doing what's necessary to improve the process.

William Pietri
  • 3,060
  • 1
  • 17
  • 20
  • +1: Spike solution the first time, then refactor it to be better and more reliable the second time. – S.Lott Mar 24 '11 at 18:32
  • So how do you make sure that developer-focused tasks aren't taking over the sprint, especially when you haven't established a good velocity metric yet? I would not want to miss an early deliverable because we spent too much time on stuff that will help in the long run. – Michael Kristofik Mar 25 '11 at 01:42
  • And you should be making time for regular reflection and making process improvements like this anyway. – Michael Mar 25 '11 at 01:43
  • @Kristo, I don't think your customer/product manager would let you get away with this. Even without an established velocity, you'll make a good guess and negotiate the value to be delivered for those first sprints. Plus if you spike like @S.Lott suggests you won't be delivering anyway. – Michael Mar 25 '11 at 01:57
  • @Kristo: You make sure by doing it gradually and reflecting regularly. When you start out, all you know is that you'll definitely be doing the wrong amount. Each week, talk about whether you should be doing more or less infrastructure, and whether you're focusing on the highest-value stuff. It's always a balancing act. – William Pietri Mar 25 '11 at 07:18
  • @William, then what does it matter if Infrastructure tasks are stories? As long as you can tag the ones that are directly customer-driven vs. not, then I don't see the difference. Your planning tools will help measure how much time you're spending on them. In my mind, it's easier to find that balance if *all* stories are in the same place. – Michael Kristofik Mar 25 '11 at 10:45
  • @Kristo - The point of the team to produce something useful to others. The primary metric, velocity, should only count the stories that achieve that. Muddling that number, or creating a variety of competing numbers, just confuses something where there should be clarity. – William Pietri Mar 26 '11 at 00:26
8

IMHO the ideal approach is placing infrastructure effort as tasks under the user story where it first holds value; as you have mentioned.

Taking your example; manually building and deploying implies it is an ongoing effort and has no form of completion. It exists indefinitely.

The same could be said for code which automates any portion of effort in a typical application which was previously done manually. Defining this effort as a task under a user story defines complete; which by its very nature has value to the end user.

You could certainly build and deploy the application each and every sprint but that then becomes part of the day to day tasks which are not formally tracked via the backlog and then this all becomes moot.

Aaron McIver
  • 3,262
  • 16
  • 19
  • Thank you for this answer. It finally clears up how this should be done: "the ideal approach is placing infrastructure effort as tasks under the user story where it first holds value". – Igor Popov Oct 04 '14 at 20:31
  • Actually this infrastructure work should be part of the Definition of Done. – Igor Popov Oct 04 '14 at 20:32
4

User stories define a buisness value from the user perspective. Because of that infrastructure tasks are generally considered as a "waste". It doesn't mean they are not need. It means that doing more infrastructure tasks will deliver less business value. Because of that infrastructure tasks should not be considered as a user storie and should not be attached to any user stories.

On a planning meeting team must consider what infrasturcture tasks will be absolutely necessary during the next sprint. The commitment will be done with these infrastructure tasks in mind. It will affect team's velocity which is correct result because velocity measures how much business value the team can deliver.

Ladislav Mrnka
  • 7,326
  • 1
  • 23
  • 32
3

From what I have seen much of the infrastructure is considered a given. This includes things like:

  • Revision control system;
  • Automated build system;
  • IDE and other developer tools;
  • Development servers;
  • Deployment process; and
  • Project process and standards.

Most methodologies I have worked with don't give much attention to them. These form what I call Release 0. These things should be in place before you begin development. Once you start working on the stories any changes in these things could be tracked as process improvements.

While the development team may have input, most of these items should be handled by a project support team. Standardizing these items across multiple projects should have a significant payback for the organization.

BillThor
  • 6,232
  • 17
  • 17
  • 1
    +1: If this isn't in place, Agile is really hard. Stable, proven infrastructure and platform a kind of pre-requisite for agility. – S.Lott Mar 24 '11 at 18:32
2

I never equated user stories to having to deliver end user value. It may be common, but its not how we handle user stories. Sometimes, these types of tasks are considered spikes, but we've also had regular user stories, point estimated like any other user story.

Andy Wiesendanger
  • 690
  • 5
  • 13
  • Some teams work that way, but that makes it harder to measure delivered value. Personally, I suggest teams only create stories that have business value. (Spikes have business value because the product people are buying information about future options and their costs.) – William Pietri Mar 24 '11 at 17:15
  • But what is business value? Its a broad term, and anything that allows a business to release software sooner/better/etc has value to that business. – Andy Wiesendanger Mar 24 '11 at 17:37
  • The distinction I'm drawing is between things of direct value mainly to the team, and things of direct value mainly to the people you are actually there to serve. I think you should only count the latter toward velocity because that's the only value that ultimately matters. Things done in aid of improving that value creation are accounted for in velocity through improved long-term velocity. Counting it right away distorts incentives, and double-counts the gain. – William Pietri Mar 24 '11 at 20:49
1

Consider the following:

  • Scrum team adding major features to an existing product suite.

  • There is a need to upgrade development technology/tools/utilities to stay current based upon engineering best practices.

  • It makes sense to front load a release with this work so that over the course of the release Sprints issues can be resolved.

  • Since the business obtains indirect value from these items I suggest that in the interest of transparency these are Product Backlog Items (PBIs).

  • The team sizes these items and treats them as they would any PBI.

This issue for me boils down to the fact that I don't want to waste time trying to figure out how to hide this work as tasks beneath other more business centered PBIs.

I don't want PBI sizing skewed by this sort of infrastructure work. I want to see what is being done and understand what I'm paying for.

I also think there is value in having the team understand the commitment the business is making by investing in the infrastructure required to deliver quality solutions.

Matthieu
  • 4,559
  • 2
  • 35
  • 37
Chris
  • 11
  • 1
0

XP recommends suggests having an "iteration zero" where all the tools and infrastructure are set up. Writing stories for these is optional, but is probably a good idea. Being able to test your infrastructure (incremental build, automated test and deploy, notifications, etc.) is beneficial

Steven A. Lowe
  • 33,808
  • 2
  • 84
  • 151
  • 2
    XP does not recommend that. Some people do, but it is definitely not part of Extreme Programming as defined by Beck, et al. Personally, I think Iteration Zero is a bad idea. – William Pietri Mar 24 '11 at 17:09
  • Another problem, you don't always start at 0, you may realize you need something now, or in the next sprint. – Andy Wiesendanger Mar 24 '11 at 17:39
  • @William: see "Planning Extreme Programming" by Kent Beck, Chapter 15, page 66. – Steven A. Lowe Mar 27 '11 at 16:38
  • That is not a recommendation. They say it's an idea, and say, "If you haven't worked with your technology before, consider spending two weeks getting the technology just right before you begin programming." And they don't suggest "all the infrastructure", just basic automated test, build, and deploy scripts. – William Pietri Mar 28 '11 at 04:34
  • @William: aha, I see what you're getting at. I didn't mean all the _software_ infrastructure, just the stuff you mentioned – Steven A. Lowe Mar 28 '11 at 05:31
  • Hi! Actually, what I'm getting at is "recommended" and "iteration zero". Ten years ago in a peripheral book, they mentioned an option of filling the first iteration with some technical setup tasks if you were new to your tools. That's different than the "Iteration Zero" concept, which is much more elaborate, and it regardless wasn't recommended. – William Pietri Mar 28 '11 at 18:16
  • @William: I've always done a basic iteration zero (for new projects/teams), but I see via google that others have taken the notion to absurd, waterfall-esque levels. I'm not talking about that. As for 'recommended', technically I guess not: "If you haven't worked with your technology before, consider spending two weeks getting the mechanism working just right before you begin programming." consider != recommend ;-) – Steven A. Lowe Mar 28 '11 at 18:22
  • Glad to hear you haven't been infected with that meme. But I think iteration zero is essentially waterfall-esque. It's pretending that there's a period before the project really starts when time doesn't count. I strongly encourage teams to just start. One many not deliver much in the first iteration, but always delivering something helps people avoid waterfall thinking. – William Pietri Mar 28 '11 at 18:32
  • @William: I absolutely agree that overblown 'zero' iterations should be avoided - they defeat the purpose of agile! – Steven A. Lowe Mar 28 '11 at 20:56
0

In our team we do the following:

  1. Assume a lower focus factor.
  2. Try to include such tasks into user stories that actually need implementing them.
  3. If some tasks are totally necessary, but provide no direct business value (such as migrating unit tests from one framework to another), at the beginning of the sprint we create a list of "continuous tasks". These are development-related tasks that are not stories, but the development team wants them done. We list these tasks on our blackboard, still keeping it separate from stories. During the sprint, at each daily meeting we review what's been done to accomplish them.

The step 2 is the most important. As in an agile practice, in Scrum you try to do as few as possible to accomplish your tasks. Take it as a way to not waste your life on doing unnecessary work: my experience shows that as much as 50% of "would-be-cool" things end-up abandoned and unmaintained in the long run.

P Shved
  • 7,427
  • 35
  • 50