Dangers of huge monolithic application

Question

The big project I'm working on for a couple years now is a control (and everything) application of an advanced device, heart of its firmware.

The device is quite advanced, with more different functionalities than I could say from memory, and 98% of them are handled by this one huge executable. In one hand, the program is quite maintainable, well modularized inside, properly documented, there's a reasonable separation of functionalities by directories and files and so on.

But in the end it gets all clustered into one application that does everything from remote database communication, touchscreen handling, handling a dozen various communication protocols, measurements, several control algorithms, video capture, sunrise time and date of easter (seriously, and they are needed for very serious purposes!)... In general, stuff that is very thinly related, often related only through some data that trickles between some far modules.

It could be done as several separate executables communicating with each other, say, over sockets, with more specific purpose, maybe loaded/unloaded as needed, and so on. No specific reason why it is made this way.

In one hand, it works, and it works okay. The project is more simple, without maintaining build of multiple binaries. The internal structure is easier too, when you can just call a method or read a variable instead of talking over sockets or shared memory.

But in the other hand, the size, the scale of this thing just creeps me out, it feels like piloting Titanic. I was always taught to modularize, and clumping everything into one gargantuan file feels wrong. One problem I know is a heavy crash of one (even insignificant) module crashes all - but code quality assures this doesn't really happen in release versions. Otherwise, internal separation and defensive programming assures this will still run mostly correctly even if half of the internal modules fail normally for some reason.

What other dangers did I overlook? Why does this creep me out? Is this just irrational fear of unknown? Is making serious big projects this way an accepted practice? Either calm my fears or give me a good reason to refactor version 2.0 into multiple smaller binaries.

If the sun rises too early, *England will drift out to sea.* — Maxpm, Jul 13 '11 at 15:21
@Maxpm: Luckily, our app doesn't drive sunrises and sunsets. We trust Princess Celestia won't fail at her job. — SF., Jul 13 '11 at 17:42
(1) If software update is needed in the future, will the entire application be uploaded at once? Can the device survive an monolithic software update where all software functions are disabled for a fraction of a minute? (2) Is the software run in a single processor, a group of processors executing in lockstep, or a group of heterogenous processors each exercising different part of the application? — rwong, Jul 13 '11 at 19:48
@rwong: 1. The device is stopped for upgrades. While it would survive updates on the fly, we don't want any unexpected results in case update goes wrong for any reason. 2. Single-core ARM9 with embedded Linux. There's a second, supervising CPU running without OS, verifying the main CPU produces sane output. — SF., Jul 13 '11 at 23:24

score 23 · Answer 1 · edited Jul 13 '11 at 20:51

The device is quite advanced, with more different functionalities than I could say from memory, and 98% of them are handled by this one huge executable. In one hand, the program is quite maintainable, well modularized inside, properly documented, there's a reasonable separation of functionalities by directories and files and so on.

You said it yourself - it is quite maintainable, well modularized ...

In my opinion, and most of management will agree with me on this one, changes should never be made for the sake of changes. There is nothing wrong with this program (your description) apart from that it doesn't follow the latest programming trends (which are mentioned in other answers).

Yes, it is a larger program ... many before have been also. It is well documented also, so all you have to do is to study its internal organization and you will see the logic in it. I doubt that all those before you who wrote it were incompetent. So, explore it a little, learn it ... after that, maintain it as it is until a solid reason for rewriting it pops up. Your manager will be grateful to you for having a good cost / effect ratio.

What other dangers did I overlook? Why does this creep me out? Is this just irrational fear of unknown? Is making serious big projects this way an accepted practice? Either calm my fears or give me a good reason to refactor version 2.0 into multiple smaller binaries.

It creeps you out because it is large - but then again, no one is expecting you to learn it all by heart in a week. So work, piece by piece, bit by bit, until you get the hang of it. In the end, you will learn that it is probably well organized as it is, and how it is organized.

Were you to rewrite it, you would accomplish the same, but at the same time ruin it for all those who were used to the previous organization of the code. This way, only you have to "adapt".

score 16 · Answer 2 · answered Jul 13 '11 at 14:46

16

I'd feel better if you said you had everything wrapped in unit tests. If you don't, you should build a test suite before even thinking about rearchitecting the application.

Having a test suite will improve your understanding of the application, and will tell you when a change in the application breaks something.

answered Jul 13 '11 at 14:46

Robert Harvey

198,589
55
464
673

2

But that applies to both possible architectures... – SF. Jul 13 '11 at 14:56
3

Yes, it does... – Robert Harvey Jul 13 '11 at 14:57
10

...and therefore this answer doesn't add anything to the discussion except to promote unit testing. – benzado Jul 13 '11 at 19:18
2

@benzado: Which, in the OP's scenario, is pretty much of paramount importance. – Robert Harvey Jul 13 '11 at 19:19
Some answers below also address why SF's alternative architecture also has some disadvantages. I think the (unfair) comparison of the real code to an imagined ideal has as much to do with his unease as testing does. – benzado Jul 13 '11 at 19:26
-1 - Testing is important and so is brushing your teeth; I fail to see how this relates. The OP did not even mention testing once. If this answer applies to this question then it applies to practically every question on Programmers SE. The OP should also have version control, but that does not apply to this question. – Morgan Herlocker Jul 13 '11 at 19:57
2

@ironcode: Take a look at a book like "Working Effectively with Legacy Code." The first thing he says in that book is "If you don't have a suite of unit tests with high code coverage already, *write them first, before you do anything else.*" I'd say that advice is more than relevant here, given that the OP specifically asked *"What other dangers have I overlooked."* – Robert Harvey Jul 13 '11 at 20:31

score 8 · Answer 3 · answered Jul 13 '11 at 16:48

If the code is well modularized with low coupling and high cohesion, it is effectively partitioned. Communication overhead should be lower within the one process than would be the case with multiple processes.

For embedded systems, your one process may eliminate the need to implement an O/S to run all the processes you would be creating. You would also need to implement a system to check that all the processes where running, and restart them if possible. If it wasn't possible, you would need to react accordingly.

Splitting the code into separate executables would also increase the overall size of the source code as you would need to implement all the client/server code between the modules. You would also need more test cases to handle this code, and cases when the server process wasn't there. Big projects are big, eliminating unnecessary complexities as appears to have been done here helps keep them as small as they can be.

Testing is sort of a red herring here, as the problems will still be there with or without testing. Good testing with either design choice should certainly encouraged. I expect, testing is simpler with the monolithic code.

Forcing a crash when necessary is also easier with a monolithic executable.

+1, engineering is about tradeoffs, smaller binaries have a cost, too — benzado, Jul 13 '11 at 19:21
+1 I agree wholeheartedly. There is no specific reason given to warrant multiple executables. Communication and coordination between executables has its price. — Erick Robertson, Jul 14 '11 at 17:54

score 5 · Accepted Answer · answered Jul 14 '11 at 12:37

Except for the tiny comment at the end (second, supervising CPU) you could be describing my company. Yup, we need Easter too.

Well, we're a bit further along. We did split the big executable and tried to use standard components for standard bits. Not exactly the big improvement you'd hope for. In fact, performance is becoming a major pain even on beefier hardware. And maintenance costs haven't really gone down now that there's tons of code to serialize and synchronize data over narrow interfaces.

The lesson I've learned? Having a single executable is a well-proven solution for small systems, and we have decades of experience in managing it. All our tools support that natively. Modularity can be done within a single executable, too, and when you need to compromise on modularity for other reasons the hack remains small.

Thanks - no "best practices to be followed/sacrificed" and "weighting potential benefits vs potential disadvantages" entries are ever as helpful as someone with first-hand experience from the other side of the fence. — SF., Jul 14 '11 at 19:35

score 1 · Answer 5 · answered Jul 13 '11 at 15:10

If I were to work on such a big monolithic application, the things that would freak me out are lack of encapsulation, low cohesion, high coupling and how to test all of this. Big monoliths are often an invitation to abandon proper architecture and start the descent into a big ball of mud.

Once that is happening, testing becomes a nightmare and you are well on the road into obsolescence.

However, if your code is well structured, and well maintained, and the principles of high cohesion, low coupling and proper encapsulation are adhered to, then I see little reason to refactor this into smaller units.

You do want to have good tests in place, though.

score 1 · Answer 6 · edited Jul 13 '11 at 20:51

If you are using a single process, there is some chance than a wild pointer from one of your sub module would corrupt a critical piece of memory (and crash the whole thing).

If you are using different processes, you are at least not using the same heap or call stack, and it might also be easier to restart a single subprocess.

The internal structure is easier too, when you can just call a method or read a variable instead of talking over sockets or shared memory.

Redividing everything in multiple processes will also force you to clean up the design and avoid that every module start using internal methods of other modules.

That being said it is probably a daunting task.

What you could start doing, is decide that every new module should be an isolated process.

You're assuming an awful lot about the runtime hardware and OS environment, especially when firware and embedded devices are involved. — Patrick Hughes, Jul 13 '11 at 19:34

score 1 · Answer 7 · answered Jul 13 '11 at 15:11

Ideally, the answer is yes. Having multiple binaries has the potential to make it more stable....

...however, you also mentioned that the code within the monolithic code base is well written and modularized, which means your not dealing with a huge tangled mess, just a huge code base...

Overall, I would say that the applicable saying is, "Don't let the perfect be the enemy of the good." While I feel that you are correct in saying that a monolithic code base is bad, I also feel that it is not worth the effort to worry about it.

It would be reasonable to move forward by putting new pieces of code in their own binaries, but going back and refactoring is probably not worth your while.

score 0 · Answer 8 · answered Jul 13 '11 at 19:39

When I get into the realm where the project is too big to grok all at one go, I build a chart to hang on the wall that lays out all the big sections and how they fit together. Then when I'm working I can reference the module I'm focusing on and have a visual reminder of how it fits into the whole.

It's quite possible that the complexities and dangers of maintaining a build and versioning system for multiple, disconnected binaries will far outweigh your unease with such a large and monolithic project.

Dangers of huge monolithic application

8 Answers8

Linked