We're not a software company. Is a complete re-write still a bad idea?

Question

I understand the reasoning behind Joel Spolsky's article "Things You Should Never Do, Part I", but I always see it referenced in regards in situations where the end goal is the production of software. What if I'm a developer who maintains an ecommerce site? My job isn't writing a retail platform, but instead putting it to use. In fact, this wouldn't even be a re-write, as such, but a big database and web design transition.

The software our site is based on is written in classic ASP, and is fundamentally missing many features that customers expect from a current shopping site. Rather than continue to add these features in piecemeal, my gut feeling is that I should start to transition to a more modern platform. We would lose the customizations that we've made over the years, but frankly, many of this features already exist (and have have almost certainly been implemented better!) in the package that I'd like to switch to.

Am I falling victim to the spirit of Netscape, or am I right in thinking my time is better spent in places besides making our tools do what we need?

To clarify, this is the equivalent of switching blogging platforms for us. Any "development" that I do is essentially rewriting the front end of our website, while the back end is out of my control.

Suppose WordPress development had stopped years ago, and was missing "modern" features like commenting, static pages, clean permalinks, etc. Sure, I could write a plug-in to add those things, but after a while, wouldn't it be better to switch platforms to something that had all those (needed) features built in from the beginning?

Answer 1

Before You Upgrade...

It could be a good idea if, and only if:

• it adds features that your customers requested,

• it don't lose any existing features over the current solution,

  (some users will ask form them, and if you're a small company, leaving customers are very costly - and might start a wave)

• it doesn't add invisible/hidden requirements and side-effects,

• it doesn't come with environmental constraints (or consider them in calculations),
• your new target middleware is actively developed and supported, and will be so for a (long) while,
• you carefully consider the costs (and see a relatively short-term benefit afterwards) of:
  • development,
  • deployment,
  • training,
  • maintenance & support,
  • maintenance & support,
  • maintenance & support.

Make sure that it won't be harder to maintain than it is now.

Plus, you do mention your current technology, but what you'd want to move to. It's even more dangerous if you shift technologies.

If You Decide to Upgrade...

Be sure to:

• backup your data,
• rehearse a business recovery plan...:
  • to revert to your current working status as quickly as possible if things go awry,
  • to communicate with customers about the (failed) transition and issues, and the status of their data.

If you upgrade (and if possible) do it iteratively!

1. Update your database or create a new one,
2. copy over your data,
   • update your backups regularly to work on recent copies,
   • and backup the old copies as well (safely encrypted, etc...)
3. Develop on a separate environment,
4. test internally a lot,
5. test with a focus group of some trusted customers.

If a complete switch of middleware is necessary, then see if there are any success stories of migrations between these 2 software solutions, and read carefully what other people have done and what pitfalls and roadbumps they encountered.

Refactor and Monitor Quality

If it's only (or almost entirely) about messy code, don't do it. Instead:

• Read my answer on how to organize uncommented, dirty code (it applies to specific code snippets, but also to larger codebases, and the recommened material could help!)
• Just refactor over time,
• Use continuous integration and continuous inspection systems to monitor quality and the impacts of the refactoring,
• And try to reach out to customers to clearly ask what features they are missing and build a very sound business case around each of them to know if they're worth it for small improvement projects.

Development is costly, maintenance even more, and building stuff for no particular reason might be nice to please users but remember that you will need to maintain and support it.

Also, if you're not a software company, do you have enough trained staff to support this task during development and after deployment? What if some of your staff leaves in the middle of the task?

Answer 2

The failed projects that Joel Spolsky wrote about were major undertakings. For the few examples that he provided there are some examples of rewrite successes:

• Mac OS X
• Windows NT

Now Windows has been rewritten more than once, and each time it has been hit or miss whether the changes would be well received or not. For example, NT was a hit as it made Windows usable in the Enterprise (where Unix lived before). On the other hand Vista was not nearly as well received. However, in both of those cases, Microsoft did hedge their bets.

The point that Spoelski makes is that a software rewrite is an incredibly risky proposition. As the complexity of the project goes up, the risk goes up exponentially.

The counterpoint with Brooks and the make one to throw away comment is that your first version of anything is a learning process. By trying to treat it as more than a learning process, you are going to lock yourself into bad code for a long time. In Pragmatic Programmer and many other sources, small refactoring to clean up bad decisions is much less risky than whole scale rewrites.

Now, in your situation you have to look at the choices in front of you. You have the following challenges:

• Your current software is based on an obsolete platform
• Newer platforms solve problems you had with your current platform better, and more completely
• Changing platforms is risky
• Rewrites need planning, or they will fail

Before you consider doing a rewrite, you need to answer a few questions for yourself:

• Is this software already near the end of its life? (i.e. if it has a limited shelf life, the cost and risk associated with changing the platform may not be worth it)
• Is the software simple/small? The larger the software or the more complex it is, the more things that can and will go wrong.
• Is your target platform able to do everything you need? To assess this, it might be worth simply doing just the questionable thing(s) to see if it is feasible. It might be possible, but just as ugly as what you have now. Better to find out with a controlled test than after you've made the decision.
• How long can your customer wait between releases? If a complete rewrite can be done in the same release cycle (or more quickly), then it's probably a no brainer. However, most customers don't care about your problems and they don't see your problems as their problems. They won't want to wait while you do a full rewrite.

I've gone through this exercise more than once, and sometimes it came out in favor of a rewrite and sometimes it came out in favor of keeping the status quo. Several times, we discovered that if we rewrote just one part of the app, we could significantly reduce our pain while still providing value to the customer.

Answer 3

Migrate incrementally

The point about not re-writing from the ground up is that you risk messing up your whole product through unstable specifications, runaway bug lists, etc, and the time/costs you'd spend with nothing available to show for your work is hard to justify.

As you're already using ASP, I recommend implementing new pieces of the site in ASP.NET and gradually moving older pages into the newer framework as you update them. We did this with a legacy product and over 18 months we completely migrated it, but we could ship an update every couple of weeks to keep up with customer demand.

Answer 4

The whole idea of a complete rewrite is that there is nothing, no single thing, of value anywhere in the existing code. Indeed the whole idea was flawed and should be purged from the earth.

In my experience, this is seldom the case. Generally it's just obsolete, and needs to be given a fresh coat of paint via a transition to some less obsolecent platform. Previous applications nearly all represent a fossilized strata of learning behaviour and optimization. They are layer upon layer of incremental improvements in an attempt to reach a functional peak. Ignore this process at your peril.

Too too too often I see people commit to a "complete rewrite" and then rewrite the application to day 1 of it's improvement cycle, without incorporating any of the lessons or improvements gleaned in years of operation. Yes, that stuff is horrible cruft glomming onto an otherwise clean codebase, but it's there because the original codebase was insufficient. Yes, rewrite, but incorporate the things that turned out to be missing from the original idea. The very best software comes about this way: by incorporating the lessons learned in inferior releases into a more perfect release.

Answer 5

It is conventional wisdom that doing a rewrite is a bad idea. And for good reason. Full blown re-writes are generally a bad idea, and often fail spectacularly.

But a better way to state the advice is... "A complete re-write is always a bad idea, except when its a good idea".

Dont substitute conventional wisdom for your (or your team's) personal judgement about what is best for your application. A simple bit of conventional wisdom cannot and does not apply to all situations. Its up to you to evaluate for yourself - keeping in mind the warnings others provide - whether or not a rewrite is necessary. No one else here knows as much about your application and project goals as you do.

Answer 6

It doesn't sound like you're doing a complete re-write, more that you're moving your business to a new platform that already implements existing functionality, which is much more sane than re-writing a new system entirely from the ground up. Go for it! ...and make sure you have backup plans, recovery plans, rehearse and test the transition multiple times (in a proper isolated test environment), get some usability testing done on the new platform, etc...

Many business transition from one platform to another. I know there are financial companies that are migrating off of their COBOL mainframe systems and it's much bigger than just a database and UI design transition. But it's also essential.

Answer 7

The question is about what strategy is "best", right? As you can see from the responses here, there are pros and cons for both. So how do you decide? My answer is that you have to make a proper risk analysis. You also have to weigh in the consequences at two levels: A short time perspective and a long one, and that has to do with the expected lifetime of the system or product. It all boils down to taking the most economic route, and to do that you have to know how much (time) everytning costs and the risks (in terms of costs) that are involved. This not an easy task, but it has to be done if you like to make a valid business decision rather than an educated guess.

Beware of categoric answers.

Answer 8

I am faced with a similar problem with a messy Java Struts web application. I plan to rewrite the entire site in time, but cannot stop enhancing the existing code. Instead I plan to develop new pages under a better architecture, and migrate the old pages when I have to touch them, or when there are no urgent tasks.

Answer 9

I am a proponent of rewrites, contrary to Joel's advice. This is mainly because I have encountered far too many systems that were just flat out wrong and had been done "wrong" for so long that it would be impossible to refactor everything that needed to be fixed. And I mean "everything": Database was bad, code was poor and not reusable, right down to using outdated HTML/CSS. When something is that bad, refactoring is like duct taping a leaky pipe - you will have to replace it at some point, and doing it sooner rather than later will save you time and money. I have seen firsthand the effects of following the "rewrites are evil" mantra, and the end result was when the system finally crashed, it crashed spectacularly and dragged the company into bankruptcy, all because nobody could see that things were awful and a rewrite would have been the only saving grace.

To answer the OP's question, if the site can't be sustainable for the future, if it's using very obsolete technologies that are no longer supported (making it harder to get people to maintain it), then it's time to consider a rewrite. Whether or not it's a software company or not isn't as relevant as the question if what you save (LONGTERM, do not fall into the trap of thinking short-term; it's that pitfall which causes systems to stagnate and fester in the first place!) by doing a rewrite will outweigh the added baggage of maintaining a terminally-ill system.