I would like to ask people experienced in working with systems the
scale of Visual Studio: what is it that makes them slow? Is it the
layers upon layers of abstractions required to keep the codebase
within the human comprehension capabilities? Is it the sheer amount of
code that needs to be run through? Is it the modern tendency towards
programmer-time-saving approaches at the (mindbogglingly huge) expense
in the clock cycles / memory usage department?
I think you guessed a number of them but I would like to offer what I consider to be the biggest factor, having worked on a reasonably large codebase (not sure if it's as big as Visual Studio -- was in the millions of lines of code category and around a thousand plugins) for about 10 years and observing phenomena occur.
It's also a bit less controversial since it doesn't go into APIs or language features or anything like that. Those relate to "costs" which can spawn a debate rather than "spending", and I want to focus on "spending".
Loose Coordination and Legacy
What I observed is that loose coordination and a long legacy tends to lead to a lot of accumulated waste.
For example, I found around one hundred acceleration structures in this codebase, many of them redundant.
We'd have like a K-D tree for accelerating one physics engine, another for a new physics engine that was often running in parallel with the old one, we'd have dozens of implementations of octrees for various mesh algorithms, another K-D tree for rendering, picking, etc. etc. etc. These are all big, bulky tree structures used to accelerate searches. Each individual one can take hundreds of megabytes to gigabytes of memory for a very average-sized input. They weren't always instantiated and used all the time, but at any given time, 4 or 5 of them might be in memory simultaneously.
Now all of these were storing the exact same data to accelerate searches for them. You can imagine it as like the analogical old database which stores all of its fields into 20 different redundant maps/dictionaries/B+ trees at once, organized identically by the same keys, and searches all of them all the time. Now we're taking 20 times the memory and processing.
In addition, because of the redundancy, there's little time to optimize any one of them with the maintenance price tag that comes with that, and even if we did, it would only have 5% of the effect it ideally would.
What causes this phenomena? Loose coordination was the number one cause I saw. A lot of team members often work in their isolated ecosystems, developing or using third party data structures, but not using the same structures other team members were using even if they were outright blatant duplicates of the exact same concerns.
What causes this phenomena to persist? Legacy and compatibility was the number one cause I saw. Since we already paid the cost to implement these data structures and large amounts of code were dependent on these solutions, it was often too risky to try to consolidate them to fewer data structures. Even though many of these data structures were highly redundant conceptually, they weren't always anywhere close to identical in their interface designs. So replacing them would have been a big, risky change as opposed to just letting them consume memory and processing time.
Memory Efficiency
Typically memory use and speed tend to be related at the bulk level at least. You can often spot slow software by how it's hogging up memory. It's not always true that more memory leads to a slowdown, since what matters is "hot" memory (what memory is being accessed all the time -- if a program uses a boatload of memory but only 1 megabyte of it is being used all the time, then it's not such a big deal speed-wise).
So you can spot the potential hogs based on memory usage a lot of the time. If an application takes tens to hundreds of megabytes of memory on startup, it's probably not going to be very efficient. Tens of megabytes might seem small when we have gigabytes of DRAM these days, but the largest and slowest CPU caches are still in the measly megabytes range, and the fastest are still in the kilobytes range. As a result, a program that uses 20 megabytes just to start up and do nothing is actually still using quite "a lot" of memory from the hardware CPU cache point of view, especially if all 20 megabytes of that memory will be accessed repeatedly and frequently as the program is running.
Solution
To me the solution is to seek more coordinated, smaller teams to build products, ones who can kind of keep track of their "spending" and avoid "purchasing" the same items over and over and over.
Cost
I'll dip into the more controversial "cost" side just a teeny bit with a "spending" phenomena I've observed. If a language ends up coming with an inevitable price tag for an object (like one that provides runtime reflection and cannot force contiguous allocation for a series of objects), that price tag is only expensive in the context of a very granular element, like a single Pixel or Boolean.
Yet I see a lot of source code for programs which do handle a heavy load (ex: dealing with hundreds of thousands to millions of Pixel or Boolean instances) paying that cost at such a granular level.
Object-oriented programming can kind of exacerbate that. Yet it's not the cost of "objects" per se or even OOP at fault, it's simply that such costs are being paid at such a granular level of a teeny element that's going to be instantiated by the millions.
So that's the other "cost" and "spending" phenomena I'm observing. The cost is pennies, but pennies add up if we're purchasing a million cans of soda individually instead of negotiating with a manufacturer for a bulk purchase.
The solution here to me is "bulk" purchase. Objects are perfectly fine even in languages that have some price tag of pennies to each one provided that this cost is not being paid individually a million times over for the analogical equivalent of a soda can.
Premature Optimization
I never quite liked the wording Knuth used here, because "premature optimization" rarely makes real-world production programs go faster. Some interpret that as "optimizing early" when Knuth meant more like "optimizing without the proper knowledge/experience to know its true impact on the software." If anything, the practical effect of true premature optimization is often going to make software slower, since the degradation in maintainability means there's little time to optimize the critical paths that really matter.
This is the final phenomena I observed, where developers reaching to save pennies on the purchase of a single can of soda, never again to be bought, or worse, a house, were wasting all their time pinching pennies (or worse, imaginary pennies from failing to understand their compiler or the architecture of the hardware) when there were billions of dollars being wastefully spent elsewhere.
Time is very finite so trying to optimize absolutes without having the proper contextual information is often depriving us the opportunity to optimize the places that genuinely matter, and thus, in terms of practical effect, I would say that "premature optimization makes software much slower."
The problem is that there are developer types who will take what I wrote above about objects and try to establish a coding standard that bans object-oriented programming or something crazy of that sort. Effective optimization is effective prioritization, and it's absolutely worthless if we're drowning in a sea of maintenance problems.
