Engineering a better solution, coming from existing codebase

Question

The Code

I have high-business-value daily-used-by-customer software that is written in PHP and spans approximately 600K lines of code.

Customer for a long time needs, wants, and demands new features and functionality. The time to have it done is yesterday. So, just write the new features and implement new functionality and deliver it to the customer, yes? Well, no, here are some problems that have been causing considerable pain to current developers:

Existing code-base is ... a big ball of mud.

Notable problems:

• code is hairy - a single feature permeates everything, tracing code is a pain, and adding a feature may potentially impact everything else
• there are no tests
• mix of procedural and object oriented code ridden in bad programming practices
• files reaching 6000-lines of HTML, CSS, PHP, SQL, jQuery, JavaScript, comments
• complete disregard/non-existence of MVC/separation-of-concerns pattern. Code is intermixed together
• some business logic depends on volatile things that have no relation to the code (like database metadata)
• hardcoded values, paths, and lack of configurability contribute to lack of security of current architecture
• large repeated blocks of essentially the same code contribute to similar features working slightly differently. Updating one does not update the other
• code is slow, lack of documentation., etc, etc. quite a few other things can be done better

It works...

The good thing is that it works... The functionality that's there, is reasonably worked-out for actual business cases, but ... going forwards is painful.

The Problem

It is easier now (and faster) to add a new feature using existing code style, mostly using cut-n-paste-n-modify approach, thus perpetuating the badness, than it is to do rewrites of this thing using currently existing modern best practices.

Solution?

There is a talk about rewriting the whole thing using one of the currently-leading frameworks (i.e. ZF2), but in doing so it means ignoring customer demands, taking a long time to build the software, and essentially creating new software (version 0.0.1) for the customer, with all the new-software bugs and lack of mature-software feel and functionality.

Another thought is to do some incremental development. i.e. When a new feature comes about, write it using a new approach. This is currently failing for the reason stated under "The Problem" title above.

Yet another idea is to do some slow refactoring of existing codebase... It might work in cleaning up things like MVC and a host of other things, and it will take a long time, and it will essentially feel like unraveling a messed up tightly-wound knotted ball of yarn. But doing this will not address things like unit testing, dependency injection, modern framework principles, and so on.

So, in the end, new features are coming, code is being added, and the existing codebase is not getting any better. What might you suggest to do in this case?

Answer 1

But doing this will not address things like unit testing, dependency injection, modern framework principles, and so on.

Why not? You do not need to unit test everything - just around what you change/refactor. And if you test that, you'll progressively add mocks and DI.

I strongly recommend Michael Feather's book: Working effectively with legacy code, as it describes that problem and solutions quite well.

Answer 2

I'm in a similar situation, if well in a completely distinct technology stack, and in my case (and I guess that also in yours) it is quite obvious that a whole rewriting of production code is not an option, due to practical limitations. That makes the most sense is taking it in small steps. When you are required to introduce a new feature, you do it in a more manageable way; when you have to modify an existing module/class/method/whatever, you take the opportunity and refactor it. Always try to end up with a better code, if perhaps at the expense of reducing your velocity a little in order to get some slack to pay your debt. Also, if you can silently introduce a framework that can be helpful in the long run, while coexisting with the current mud, consider it.

Answer 3

Duplication itself is not a problem - it's a way to get quick result. You just need to remember that you have technological debt and allocate time to generalize your code in the future.

What do you mean by going forwards is painful - are you planning something big or it's just additional features? There is only one reason for rewriting everything - the case when architecture of your current applicaiton doesn't fit your future development.

Once I had a project that I was developing for several months and it wasn't in particular 'dirty'. I also did unit testing. And then requirement changed and it took me two more months to understand that I have to change architecture. After I realised it I spend 3 weeks to completely rewrite that service - it was active development but I knew what to do.

I think your case is different - you just need to spend more time on cleaning. Start from small things. During next bugfix or implementation of a new feature don't jump right into development. Spend some time on cleaning. Take a rule to slightly improve any code that you touch. Of course without unit tests it's risky - and you will understand it very quickly so next time you will have more insentive to write unit tests.

At the same time you need to start thinking about high level design so you can start generalizing and decomposing system. Strictly speaking this is not refactoring. People tend to misuse this term.