How do you prioritize initial sprints to maximise the chance of catching architecture errors early?

Question

I've worked on several projects that have used agile successfully as a means of providing continuous improvement on an already mature piece of software. But I've found it's much harder to be "agile" when pre-planning something. An element of waterfall is necessary to produce initial functional specifications and a sensible architecture, as per this question: How do you explain to an "agile" team that they still need to plan the software they write?

However, you still want to be as fast, as light and most of all as flexible as possible. One of the big dangers in traditional software design is misunderstandings, resulting in major architectural changes that are only caught late in the process.

I have recently joined a project which had woefully insufficient planning and is in the most horrendous mess. While salvaging what I can, I'm left wondering what previous developers could have done (besides some planning) to catch the huge architectural problems in the system at an early stage.

If we presume a moderate amount of pre-project planning, how can I best allocate work in initial sprints to catch the (probably inevitable) architectural issues in the solution as early as possible?

EDIT: After reading the answers, and some suggested articles, I feel I made need a bit of clarity in the question. I understand that Agile requires some upfront design, but that it ought to be kept to a minimum, and it needs to be done with an eye to flexibility.

What I'm trying to discover are whether there are any tricks, tips or techniques to ensure that we correctly identify the parts of the project that need the most rigidity, prototype and iterate them first, and catch problems there as early as possible?

Answer 1

The first mistake that often gets made is that people don´t look beyond the stories in the current sprint.

While it is correct that those stories are not in scope now and might change or be removed at a moments notice, completely ignoring them when making high-level design decisions is a nice route to chaos.
When making high-level design decisions, you must also look to the stories that are still on the product backlog to see how your decision affects those stories and to revise your decision if it turns out that it makes those future stories hard to implement.
Stories higher on the backlog should have a higher priority in this evaluation, as they are more likely to be implemented.

Answer 2

how can I best allocate work in initial sprints to catch the (probably inevitable) architectural issues in the solution as early as possible?

By not mistaking quality of planning with quantity of planning.

The entire point of agile is that planning is inadequate. Business people don't really know what they need until they can have customers try it out. Technical people don't really know what is going to work until they can run it and measure it.

So the best allocation of work is to get something working ASAP. Sure, you should spend some time in a room discussing what business thinks they need. And you should spend some time with a whiteboard discussing the relative merits of potential architectures. That will take you a day. Maybe.

Then take your best guess and prototype it out. See what works. See what sucks. Show the business and see what you mis-communicated. Then iterate. It sounds as though your predecessors didn't iterate. They didn't stop and evaluate how things were going and course correct as necessary. They just tacked onto the existing mess making it worse.

Answer 3

When I hear this argument against Agile versus waterfall, it sounds like they envision the history as,

"We create these great specifications that the clients love because all of our projects are always on time and budget, but if only we could be lazy and accomplish the same thing without having to write requirements."

That's not the way I understand what was happening. I also don't fall under the trap of thinking of things as all or nothing. There is nothing wrong with writing requirements up front, but that doesn't mean that ALL requirements have to be written in advance of any programming. That's the biggest difference.

Even if you truly think you can completely write out all of the specifications for a complex project, don't do it if it is going to take a long period of time. If you talk to a client for weeks on end, go through a lengthy contract negotiation process, build the entire application and return months later with a finished project, don't be surprised when the client tells you that's not what they expected and more likely it doesn't really work the way you thought it would.

I worked as head of technical support and after months of not seeing our programmers during the building of yet another version of our application completely from scratch (much better architecture this time - yeah right) I couldn't get it to install on my PC. It broke every step of the way. I was so pissed-off, I made them stay all night and keep fixing the thing until I could actually see the app.

They spent so much time writing specs, they didn't have time for the much bally-hooed pseudo-code phase of the project. OK, maybe taking to much time to write specs can be a good thing.

Answer 4

I believe the concept of "Sprint 0" which might typically be a longer initial sprint covers this, doesn't it? Here's an article (What is Sprint Zero?) that I think describes what should be done in Sprint 0, including sufficient initial design to facilitate emergent design in subsequent sprints.

The article describes some of the common misperceptions and 'malpractices' of an 'initial sprint' to help new development projects get off the ground. Both the article and the comments below are useful to read. I can quite understand the challenge of deciding what the initial user stories should be when starting development of a large/ complex app or product. Given that many (most?) teams come from some waterfall-ish background, doing away with BDUF and a lot of planning up-front can be scary!

I found the article's 3 points of a working definition of a Sprint 0 useful - one, that it should be used to create the 'basic skeleton and plumbing' for the project to facilitate future sprint planning; two, doing a minimum design up front to enable emergent design possible as the project proceeds; and three, ensuring that even this sprint delivers value in terms of some completed stories, some of which include the product skeleton/ framework.

Hope this helps!

Answer 5

I have been working on a project from its early stages and what I have learnt is adopting mindset of agile and 'failing fast' can really help project moves forward.

Firstly you have to make sure that you can make things working by having proper POC and from their starting building your software but at same time you shouldn't spend lots of time and resources to design best architecture. code also can be kept simple and basically just workable and then you always can have refactoring PBIs in later iteration where you make it more complicated.

Beside that it is good to find out those critical things in your project and try to fail your architecture as fast as possible! so you can validate it fast and early instead of late phases. for instead if you have different components in your arch and when they are workable (not necessary complex enough) it is a good idea to start integration those components together and see if they can work together or something is missing

Answer 6

Grooming and spikes is the way we addressed this sort of thing, not always successfully I might add, all depends on how (of even if) the stories are written. We also have a dedicated architect, well some time from them.

So in grooming people familiar with the backlog and the proposed architecture should be looking at how a 'new' story impacts what has or will be done. Think of it as a sort of meta dependency.

The other thing we did when there was uncertainty about whether there was an impact or the best way to ameliorate it was a Spike Story. Basically this was a timeboxed proof of concept story as in all the non-trivial and therefore most problematic cases, a lot of the problems don't surface until a developer actually has a go at it.

Answer 7

tricks, tips or techniques to ensure that we correctly identify the parts of the project that need the most rigidity,

Guess and hope you get lucky. Experience will help.

prototype and iterate them first, and catch problems there as early as possible?

Deliver something as soon as possible. The sooner it fails the less painful it will be to toss it out. Don't plan. Do and learn.