User stories are too high level and conceptual, management expects developers to fill in the blanks

Question

I am employed in a very brilliant company with a true intention of doing XP. Communication is good and management is open to constructive discussion but due to pressing time constraints, some certain things are considered too RUP to be discussed.

At the moment I'm a little troubled with the volume of change that becomes necessary while implementing the stories. I believe many of these discoveries (which take time and effort of course) are the responsibilities of the story writers (customers, end users and product owners) and not the developers. To put is short, user-stories are too conceptual and just convey the underlying intention but lack enough details (specially pre-conditions and post-conditions, relevance to other stories, dependencies and alike). The developer is expected to fill in the blanks at his own discretion by the virtue of XP developers being designers and analysts at the same time. The problem is many of these blanks are discovered after some wrong assumptions have made made their way into evaluation time and code since noticing added complexities emerge than initially anticipated. Even then finding the right thing to fill in takes time which is - to various degrees - considered as deviation from the initial estimations.

I'm looking for a constructive way of conveying these implications to the management in a way that would not pose me as someone who is trying to unnecessarily complicate things. I'm new and as yet I have not established much credibility.

You insights are most welcome.

Closely related and somehow gives an answer : How much detail about a user story can a developer expect?

Answer 1

The trick is not to avoid there being blanks. The trick is to fill in those blanks as early as possible in the process of development.

You are correct that, if developers make assumptions, they will invariably be wrong and that will cost time redeveloping the software later. But, equally, if business people are expected to do a full up-front design when they don't really know what they want, the same thing will happen.

It's a large part of a developer's job to figure out what the customer wants, when they often don't really know.

First, ask questions. Where the answers you get seem unsatisfactory, create a prototype. Show the customer what they're asking for, and let them tell you how it's not what they really want. Start with a paper prototype, then move onto an HTML-based one, with no code behind it. Then do the smallest amount of development you need to produce a nearly-working product and show them that. Leave the tricky bits as late in the process as you possibly can.

This might sound time-consuming in itself but, when compared to redeveloping a supposedly-finished product, it isn't.

Also, keep the stories as small as possible. Invariably, what the business wants is an epic, something that can be broken down into many deliverables. This is better because you won't have developed too much when they look at the final release candidate and scream "Oh no, that's really not what we were looking for."

Answer 2

Even then finding the right thing to fill in takes time which is - to various degrees - considered as deviation from the initial estimations.

That does not sound very "XP"ish to me.

I am in no way an XP expert, but AFAIK the XP idea is to adapt your specs and your estimation continuously whenever you got feedback from the process. And the process is "analyse a little - design a little - code a little - test a little - and then get user feedback to correct your wrong assumptions as early as possible. You can also try to get user feedback even more early, for example, after designing some parts of your software (like the UI), on a sheet of paper or a whiteboard and discuss that with a user or customer. I don't think the "XP way" forbids such an approach just because of having "user stories".

Here is a nice article about how to get feedback more early by using specs. I think this kind of thinking is "methodology"-independent, and the arguments presented there will help you with your debate with management.

Answer 3

To summarize you are in the following situation:

1. You are new.
2. The project (I assume your are talking about a running project) has pressing time constraints.
3. The developer is expected to fill in the blanks at his own discretion.
4. The company you are working in is intending to practice XP. However user stories seem not to be applied in a way which fits into the XP methodology. On the other hand "The developer is expected to fill in the blanks at his own discretion".

Think about point 4: My opinion is that agile practices should be adapted to the needs and the culture of a company/team (not the other way around). Identify where the company sticks to the XP methodology and where it deviates. This is the groundwork for a constructive approach to your concerns.

Due to 1 and 2 you are currently not in a good position to question if the company applies XP in a reasonable way. Starting a discussion with the management will very likely pose you as someone who "complicates things". However you can start to discuss your concerns with your fellow developers. Maybe you will find some developers who think the way you do. Maybe there is a senior developer who will then convey your concerns to the management. But do not expect that things will change fast, especially not in the current project. However the project will give you a good opportunity to gather more data which adds more substance to a constructive approach.

Now to point 3: I think that good user stories are collaboratively written by customers/end users/product owners and developers. Show some initiative: Look for some opportunity to directly ask the authors of the user stories. If this is not possible ask some senior developer or the management how to deal with open questions which must be answered by the authors of the user stories. Maybe you can at least have some written correspondence. Since you need to fill out the blanks on your own discretion, then your choice should be to actively involve the customers/end users/product owners.

At some point you have made enough thoughts and observeration about how your company applies XP (or agile practices in general). Maybe some time has already passed and you are not perceived as a greenhorn anymore. Maybe your active involvement with the customer has shown some positive effects. Maybe the next project is already starting. Maybe your also have already some backup from other develeopers. Maybe you find out that the way it works is not bad at all. The point is that then you will then have enough ideas to convey your concerns to the management, based on real experience and data within your company.

Answer 4

Frankly, user stories should not have a whole lot of detail. "I want the software to do X, to meet Y business need" should be sufficient. After all, you not want business people dictating how to do that - you're the expert at the software and best practices therein.

That said, the developers also need to ask: "how do you expect this to work?", "what happens when that interacts with feature Z?". Developers don't make requirements, they make implementation.

It also sounds as though there is too much of a gap between implementation and evaluation. Stakeholders should be looking at prototypes, at half-done code every few days. That lets you get feedback before getting too far into the weeds.

Answer 5

If you are asked to estimate a story that seems incomplete to you, make it known that you have questions about the story and that you can't give a proper estimation before those questions are answered.

Then, ask your questions and make sure the answers become part of the story.

If you are forced to give an estimation even when your questions are not (all) answered, you can choose to either refuse to give an estimation or to clearly indicate that you are assuming the worst possible outcomes for the remaining blanks in your estimation (which probably will make your estimation a high outlier).

Answer 6

What you do is not an agile way of development. Instead, you are working with low quality requirements. It is false that an agile way of development is not to specify requirements.

Instead, they need to initially specify as much as possible, and if needed change later. Then you split your work into parts and implement in iterations. After each iteration, you have something finished.

Difference to waterfall development, is in waterfall development, everything is implemented with initial requirements, and changed at the end.

Answer 7

You are correct. User stories are too high level for actual work that goes into a sprint/increment. They are essentially what waterfall calls a user requirement and are good for describing the big picture, but not the work tasks a developer has to do in a sprint.

Evangelits of the "user story" concept defend it as the holy grail of a user requirement definition but will never acknowledge that they are not detailed enough for developer work tasks. The "as a [persona], I [want to], [so that]" is really just a user requirement from which you could create developer tasks and sub tasks after a lot of detailed analysis and technical design work, hence XP, or even scrum or Kan-ban.

Can you really say "As a developer a want a button that when pressed will fire an asynchrnous function to call an API method that will perform a GET RESTful method to return data in Json format that I will render after reformating that data into several html elements on the page. I also need to design the normalised data in the database and code the Get method using webAPI...".

No you can't. Because a user story is just a high level requirement. It is not the actual work that needs to be done.

The best way to do this is to treat user stories for what they are, abstract user requirements, nothing more. You can then child tasks and sub tasks that describe the work that you have to do. Only put the tasks in the sprint because user stories are too high level. Child many tasks/subtasks to a user story and spread them across several sprints and when all the tasks are done mark the user story as complete.

Treat user stories as high level features that must be broken down into tasks and sub tasks. A user story can have 1 to many tasks, and a task can have 1 to many sub tasks. Only tasks and sub tasks go into the sprint because they describe the unit of work to be done at the granular level.

As a guide a task and/or sub task should be about 2 days in duration so that it can be completed in a 10 day sprint along with some other 2 day tasks perhaps from a different user story or the same user story. A task can be as small as 1 hour or even upto 4 days but usually needs to be broken down enough to fit into a sprint (sprint duration's vary from 1 to 4 weeks in general practice).

For example, a task is not: "Do detailed analysis to work out how to build this feature". It is more like "Workshop a user story requirement with the stakeholders to get more details" and that is a really 2 hour meeting you attend after which you might record some of new details you got from that meeting for further analysis. From that task are born other tasks. For example, after the workshop meeting you discover that you have to import data from an external system as part of the overall project, so you would child a new task to the user story (or a related user story) that says "meeting investigate how to export data from external system", and so on. Keep it real bro.

Simply keep childing tasks to a user story that represent the real work that you need to do to complete the user story. The real work should be a sentence that accurately says what you are doing, for example, "code next and back buttons to get pagination working". Think small.

Answer 8

It sounds like the developers are completely removed from the creation of the user stories. Do you expect just to be able to read them like a detailed spec and build it right the first time? If you could do that, you wouldn't need XP or any other agile methodology.

Someone should be asking questions if the story is too vague. Where does Acceptance Testing occur?

User stories are meant to change. You need to deal with it.

Answer 9

As a developer,

I need to fully understand the specifics of a user story,

so that I can confidently estimate and implement it.

On other words, you must ask questions until you understand the specifics of the story. This is done in iteration planning and acts as decision point: if you cannot estimate it, you cannot build it.

Answer 10

Though a developer could write a story/ detailed requirements i have not seen many who are good at it. we are good at pointing out issues, suggesting better flows but as an input to already well written case.

Have worked on new and existing products and with both had cases where the requirements where just 5- lines and we were expected to fill the blanks and make an 'understanding' or more elaborate document.

Projects moved much better then we had our own professional services person who helped with this (or in one case a VP who jumped in as there was no one else available). Either way its a waste of time to develop (unless no feedback is coming back and the deadline has not changed). so my suggestion : ask for more details, provide more, ask for time bound feedback to your assumptions and documentation and state that its a risk that there will be rework and delays if you do not get this information by x date.

Answer 11

Regardless of the development methodology, if whatever you are using to define requirements makes the developer make assumptions, it needs to be kicked back to the business side. I often have a good idea of what answer I would prefer so I kick things back like this:

XYZ is unclear to me. Does it mean ABC? Or am I missing something? (Assume XYZ is the requirement, assume ABC is the assumption I would like to make as a developer)

It takes far less time to get things cleared up before you make bad assumptions than to redo. Developers who make guesses about requirements without getting confirmation that their guess is right tend to be bad developers and they cost their company a lot of money. If a bad manager won't let you kick things back, then explain to him how much more expensive in terms of time and money it is to do it wrong. If he still insists, then do what he says and when it fails UAT, the next time you want to kick something back, remind him of how costly it was the time he wouldn't let you. If he still won't listen, find a better boss.

The other value of kicking things back is that, gradually, the business will learn what you need and give you better requirements.