Is memory management in programming becoming an irrelevant concern?

Question

Background
I revisited an old (but great) site I had not been to for ages - the Alioth Language Shootout (http://benchmarksgame.alioth.debian.org/).

I started out programming in C/C++ several years ago, but have since then been working almost exclusively in Java due to language constraints in the projects I have been involved in. Not remembering the figures, I wanted to see, approximately, how well Java fared against C/C++ in terms of resource usage.

The execution times were still relatively good, with Java at worst performing 4x slower than C/C++, but on average around (or below) 2x. Due to the nature of the implementation of Java itself, this was no surprise, and it's performance time was actually lower than what I expected.

The real brick was the memory allocation - at worst, Java allocated:

• a whopping 52x more memory than C
• and 25x more than C++.

52x the memory ... Absolutely nasty, right? ... or is it? Memory is comparatively cheap now.

Question:
If we do not speak in terms of target platforms with strict limits on working memory (i.e. embedded systems and the like), should memory usage be a concern when picking a general purpose language today?

I am asking in part because I am considering migrating to Scala as my primary language. I very much like the functional aspects of it, but from what I can see it is even more expensive in terms of memory than Java. However, since memory seems to be getting faster, cheaper and more plentiful by the year (it seems to be increasingly hard to find a consumer laptop without at least 4GB of DDR3 RAM), could it not be argued that resource management is becoming increasingly more irrelevant as compared to (possibly implementation-wise expensive) high-level language features which allow for faster construction of more readable solutions?

Answer 1

Memory management is utterly relevant since it governs how fast something appears even if that something has a great deal of memory. The best and most canonical example are AAA-title games like Call of Duty or Bioshock. These are effectively real-time applications that require massive amounts of control in terms of optimization and usage. It's not the usage per se that's the issue but rather the management.

It comes down to two words: Garbage Collection. Garbage Collection algorithms can cause slight hiccups in performance or even cause the application to hang for a second or two. Mostly harmless in an accounting app but potentially ruinous in terms of user experience in a game of Call of Duty. Thus in applications where time matters, garbage collected languages can be hugely problematic. It's one of the design aims of Squirrel for instance, which seeks to remedy the issue that Lua has with its GC by using reference counting instead.

Is it more of a headache? Sure but if you need precise control, you put up with it.

Answer 2

The real brick was the memory allocation - at worst, Java allocated a whopping 52x more memory than C, and 25x more than C++.

Do you understand the numbers you base your question upon?

• How much memory was allocated?
• What were the programs doing?

When there's a big disparity between those Java and C programs, it's mostly the default JVM memory allocation versus whatever libc needs:

• n-body
  Java program 13,996KB :: C program 320KB :: Free Pascal 8KB

Look at the tasks that do require memory to be allocated (or use additional buffers to accumulate results from multicore programs):

• mandelbrot
  Java program 67,880KB :: C program 30,444KB

• k-nucleotide
  Java program 494,040KB :: C program 153,452KB

• reverse-complement
  Java program 511,484KB :: C program 248,632KB

• regex-dna
  Java program 557,080KB :: C program 289,088KB

• binary-trees
  Java program 506,592KB :: C program 99,448KB

...should memory usage be a concern when picking a general purpose language today?

It depends whether the specific usage, for your specific approach to solving the specific problems you need to solve, will be constrained by the specific limits of available memory on the specific platform that will be used.

Answer 3

As with all things, it's a trade-off.

If you are building an application that is going to run on a single user desktop and can reasonably be expected to control a large fraction of the RAM on that machine, it may be worth it to sacrifice memory usage for implementation speed. If you're targeting that same machine but you're building a small utility that is going to be competing with a bunch of other memory-hungry applications that are running simultaneously, you may want to be more cautious about that trade-off. A user may be fine with a game that wants all of their memory when it's running (though, as World Engineer points out, they'll be concerned if the garbage collector decides to pause the action periodically to do a sweep)-- they are likely to be much less enthused if the music player they run in the background while doing other things decides to gobble up a ton of memory and interferes with their ability to work. If you're building a web-based application, any memory you use on the servers limits your ability to scale out forcing you to spend more money on more application servers to support the same set of users. That can have a major impact on the economics of the company so you may want to be very cautious about making that trade-off.

Answer 4

It depends on a number of factors, especially the scale at which you're working.

Just for the sake of argument, let's assume a 30x difference in memory and 2x in CPU usage.

If you're dealing with an interactive program that would take 10 megabytes of memory and 1 millisecond of CPU if written in C, it's pretty much inconsequential -- 300 megabytes of memory and 2 milliseconds to execute is normally entirely irrelevant on a typical desktop, and unlikely to mean much even on a phone or tablet.

The difference between needing around half the resources of 1 server and needing 15 servers is a much bigger step though -- especially since scaling out to 15 servers is likely to require a lot of extra work to develop instead of less. As far as future expansion goes, the same factors you mention tend to suggest that unless your customer base undergoes massive growth, that if it'll run on one server now, chances are pretty good that when you outgrow that server, you'll be able to replace that with one newer server without any problem.

The other factor you really need to consider is exactly how much difference in development cost you're going to see for your particular task. Right now, you're basically looking at one side of an equation. To get a good idea of costs vs. benefits, you (obviously enough) need to look at both costs and benefits, not just one in isolation. The real question is basically: "is x greater than y?" -- but you can't determine that by looking only at x. You clearly need to look at y as well.

Answer 5

Memory management is absolutely relevant in today's world. However, not in the way you might expect. Even in garbage collected languages you have to ensure you don't have a reference leak

You're doing something wrong if this is your code:

static List<string> Cache;

...
Cache.Add(foo); //and then never remove anything from Cache

Garbage collection can't magically know you will never use some reference ever again unless you make it so you can't use it again, ie by doing Cache=null, you effectively alert the garbage collector that "hey I'm not going to be able to access it anymore. Do what you will with it"

It's more complicated than that, but reference leaks are just as, if not more, harmful than traditional memory leaks.

There are also some places where you can't fit a garbage collector into. For instance, the ATTiny84 is a microcontroller with 512 bytes of code ROM, and 32 bytes of RAM. Good luck! That's an extreme, and probably wouldn't be programmed in anything but assembly, but still. Other cases you might have 1M of memory. Sure, you could fit a garbage collector, but if the processor is very slow (either by limitations, or to preserve battery), then you're not going to want to use a garbage collector cause it's too expensive tracking what a programmer could know.

It also gets significantly more difficult to use garbage collection when you need guaranteed response times. Like, if you have a heart monitor or something and when it receives a 1 on some port, you need to guarantee you can respond to it with a proper signal or something within 10ms. If in the middle of your response routine the garbage collector needs to make a pass and it ends up taking 100ms to respond, that might be someone dead. Garbage collection is very difficult, if not impossible, to use when timing requirements need to be guaranteed.

And of course, even on modern hardware, there is some cases where you need that extra 2% of performance by not worrying about the overhead of a garbage collector.

Answer 6

As Donald Knuth said, premature optimization is the root of all evil. Unless you have a reason to believe that memory is going to be the bottleneck, don't worry about it. And given that Moore's law is still delivering increased memory capacity (even though we're not getting faster single-threaded code out of it), there is every reason to believe that in the future we'll be even less constrained on memory than we are today.

That said, if optimization is not premature, by all means do it. I'm personally working on a project right now where I understand my memory usage in great detail, I actually need precise control, and a garbage sweep would kill me. I am therefore doing this project in C++. But that choice seems to be a once every several years event for me. (Hopefully in a few weeks I won't touch C++ again for a few more years.)

Answer 7

For people dealing with "big data" memory management is still a huge issue. Programs in astronomy, physics, bioinformatics, machine learning, etc., all have to deal with multi-gigabyte datasets, and the programs run a lot faster if the relevant portions can be kept in memory. Even running on a machine with 128GB of RAM doesn't solve the problem.

There is also the matter of taking advantage of GPU, though perhaps you'd classify that as an embedded system. Most of the hard thinking in using CUDA or OpenCL boils down to memory management issues in transferring data from main memory to GPU memory.

Answer 8

To be fair, a lot of Java out there indulges in some truly and pointlessly class-explosive patterns that just murder performance and hog memory but I do wonder how much of that memory is just the JVM which in theory (heh) let's you run the same app in multiple environments without having to completely rewrite new ones. Hence the design tradeoff question it all boils down to: "How much of your users memory is such a development advantage worth to you?"

This is, IMO a perfectly worthwhile and reasonable tradeoff to consider. What pisses me off though is the notion that because modern PCs are so powerful and memory is so cheap, we can completely ignore such concerns and bloat features and bloat code and be lazy about choices to the point where it seems like a lot of the stuff I do on a windows PC now, takes just as long as it did in Window '95. Seriously though, Word? How much new crap that 80% of their user-base actually needs could they have possibly added in 18 years? Pretty sure we had spellcheck pre-windows right? But we were talking memory which is not necessarily speed if you have plenty of it so I digress.

But of course if you can get the app done in 2 weeks at the cost of maybe a few extra megabytes rather than 2 years to get the needs-only-a-few-K version, it's worth considering how a few megs compares to (I'm guessing) 4-12 gigs on the average users machine before scoffing at the idea of being so sloppy.

But what does this have to do with Scala beyond the tradeoff question? Just because it's garbage collection, doesn't mean you shouldn't always be trying to think about the flow of data in terms of what's in scopes and closures and whether it should be left sitting around or used in such a way that it will be deallocated by GC when it's no longer needed. That's something even us JavaScript UI web devs have had to think about and hopefully will continue to as we spread into other problem domains like the perf-savvy cancer (that you all should have killed with Flash or Applets or something when you had the chance) that we are.

Answer 9

Is memory management in programming becoming an irrelevant concern?

Memory management (or control) is actually the primary reason I'm using C and C++.

Memory is comparatively cheap now.

Not fast memory. We're still looking at a small number of registers, something like 32KB data cache for L1 on i7, 256KB for L2, and 2MB for L3/core. That said:

If we do not speak in terms of target platforms with strict limits on working memory (i.e. embedded systems and the like), should memory usage be a concern when picking a general purpose language today?

Memory usage on a general level, maybe not. I'm a little bit impractical in that I don't like the idea of a notepad that takes, say, 50 megabytes of DRAM and hundreds of megabytes of hard disk space, even though I have that to spare and abundant more. I've been around for a long time and it just feels weird and kind of icky to me to see such a simple application take relatively so much memory for what should be doable with kilobytes. That said, I might be able to live with myself if I encountered such a thing if it was still nice and responsive.

The reason memory management is important to me in my field is not to reduce memory usage so much in general. Hundreds of megabytes of memory use won't necessarily slow an application down in any non-trivial way if none of that memory is frequently accessed (ex: only upon a button click or some other form of user input, which is extremely infrequent unless you are talking about Korean Starcraft players who might click a button a million times a second).

The reason it's important in my field is to get memory tight and close together that is very frequently accessed (ex: being looped over every single frame) in those critical paths. We don't want to have a cache miss every time we access just one out of a million elements that need to all be accessed in a loop every single frame. When we move memory down the hierarchy from slow memory to fast memory in large chunks, say 64 byte cache lines, it's really helpful if those 64 bytes all contain relevant data, if we can fit multiple elements worth of data into those 64 bytes, and if our access patterns are such that we use it all before the data is evicted.

That frequently-accessed data for the million elements might only span 20 megabytes even though we have gigabytes. It still makes a world of difference in frame rates looping over that data every single frame drawn if the memory is tight and close together to minimize cache misses, and that's where the memory management/control is so useful. Simple visual example on a sphere with a few million vertices:

The above is actually slower than my mutable version since it's testing a persistent data structure representation of a mesh, but with that aside, I used to struggle to achieve such frame rates even on half that data (admittedly the hardware has gotten faster since my struggles) because I didn't get the hang of minimizing cache misses and memory use for mesh data. Meshes are some of the trickiest data structures I've dealt with in this regard because they store so much interdependent data that has to stay in sync like polygons, edges, vertices, as many texture maps as the user wants to attach, bone weights, color maps, selection sets, morph targets, edge weights, polygon materials, etc. etc. etc..

I've designed and implemented a number of mesh systems in the past couple of decades and their speed was often very proportional to their memory use. Even though I'm working with so, so much more memory than when I started, my new mesh systems are over 10 times faster than my first design (almost 20 years ago) and to a large degree because they use around 1/10th of the memory. The newest version even uses indexed compression to cram as much data as possible, and in spite of the processing overhead of the decompression, the compression actually improved performance because, again, we have so little precious fast memory. I can now fit a million polygon mesh with texture coordinates, edge creasing, material assignments, etc. along with an spatial index for it in about 30 megabytes. My oldest version used several hundred megabytes and even testing my oldest one today on my i7, it's many, many times slower, and that several hundreds MB of mem use didn't even include a spatial index.

Here's the mutable prototype with over 8 million quadrangles and a multires subdivision scheme on an i3 with a GF 8400 (this was from some years ago). It's faster than my immutable version but not used in production since I've found the immutable version so much easier to maintain and the performance hit isn't too bad. Note that the wireframe does not indicate facets, but patches (the wires are actually curves, otherwise the entire mesh would be solid black), although all the points in a facet are modified by the brush.

So anyway, I just wanted to show some of this above to show some concrete examples and areas where memory management is so helpful and also hopefully so people don't think I'm just talking out of my butt. I tend to get a little bit irritated when people say memory is so abundant and cheap, because that's talking about slow memory like DRAM and hard drives. It's still so small and so precious when we're talking about fast memory, and performance for genuinely critical (i.e., common case, not for everything) paths relates to playing to that small amount of fast memory and utilizing it as effectively as we can.

For this kind of thing it is really helpful to work with a language that allows you to design high-level objects like C++, for example, while still being able to store these objects in one or more contiguous arrays with the guarantee that the memory of all such objects will be contiguously represented and without any unneeded memory overhead per object (ex: not all objects need reflection or virtual dispatch). When you actually move into those performance-critical areas, it actually becomes a productivity boost to have such memory control over, say, fiddling with object pools and using primitive data types to avoid object overhead, GC costs, and to keep memory frequently-accessed together contiguous.

So memory management/control (or lack thereof) is actually a dominating reason in my case for choosing what language most productively allows me to tackle problems. I do definitely write my share of code which isn't performance-critical, and for that I tend to use Lua which is pretty easy to embed from C.