Why isn't DSM for unstructured memory done today?

Question

Edit: Comments suggested, that DSM just faded out by being not used recently. What were the reasons for this, what are DSMs drawbacks? Literature lists many positive aspects like easy to port programms to, no marshalling and therefore faster and so on but also negative aspects like only usable in homogenous environments due to endianess and word size issues. So why is all data synchronizing done by databases and not DSM anymore? Has there been a historic comparison or study at the time when both ways existed concurrently?

Old text:

Ages ago, Djikstra invented IPC through mutexes which then somehow led to shared memory (SHM) in multics (which afaik had the necessary mmap first). Then computer networks came up and DSM (distributed SHM) was invented for IPC between computers. So DSM is basically a not prestructured memory region (like a SHM) that magically get's synchronized between computers without the applications programmer taking action. Implementations include Treadmarks (inofficially dead now) and CRL.

But then someone thought this is not the right way to do it and invented Linda & tuplespaces. Current implementations include JavaSpaces and GigaSpaces. Here, you have to structure your data into tuples. Other ways to achieve similar effects may be the use of a relational database or a key-value-store like RIAK. Although someone might argue, I don't consider them as DSM since there is no coherent memory region where you can put data structures in as you like but have to structure your data which can be hard if it is continuous and administration like locking can not be done for hard coded parts (=tuples, ...).

Why is there no DSM implementation today or am I just unable to find one?

Answer 1

DSM (Distributed Shared Memory) is a topic from Computer Architecture rather than software development; however, I have the answer:

Summary

DSM has never stopped being used. There is an evidence with enough additional information to prove the use of DSM today and an active research in that area.

Numbers

Only for year 2012, there are 173 new unique papers at IEEE Explore describing various applications of DSM.

Additional Resources

Many other recent papers can be found in the ACM Digital Library.

Sample Papers and DSM Implementations

Software distributed shared memory and its various implementations can be found here, i.e. this paper describes a particular SW DSM implementation.

Another example of a DSM paper.

Answer 2

If I understand your question, it's why don't developers use DSM to build applications, preferring other abstractions? I think it's because of complexity.

I really think shared memory, in general, is a deceptively simple abstraction. It seems easy to understand but the particulars make it complex. Other abstractions, like message-passing, are actually simpler to write and deploy. When I was getting my Master's, Linda was still in common use, but most people developing distributed systems were using a variation of message passing or RPC.

Even within a server, where shared memory is available, developers often use sockets (Unix sockets or loopback) to communicate between processes. Using message passing, developers don't have to worry about concurrent access, obtaining locks, what to do if a process holding a lock fails, etc. Taking message passing from a single machine to multiple machines is fairly straightforward. However, as you can see from the number of papers, it is not as clear cut with DSM.

At another level, clustering that involves shared resources has kind of fallen out of favor. For example, single-system-image clustering is no longer a hot topic. Most modern operating systems have taken a "shared-nothing and we're fine with that" approach. There are some clustering toolkits available as add-ons to Linux but it's not 'mainstream.' (A lot of clustering add ons are really failover clustering and not clustering in the OpenVMS sense).

DSM is not abandoned, dead, or buried. I'm not saying the problems around DSM are insurmountable, or it is too complex to use in every circumstance or that it will never be as convenient as message passing. Some products use DSM under the hood to manage their clustering. I think Oracle's WebLogic app server can use some region of shared memory to facilitate their clustering, as an example. At the very least, it looks like it's still fertile ground for research if you decide to go for a PhD.

Answer 3

One merely needs to revisit the "Dining Philosophers" problem to understand the difficulties encountered with distributed shared resource access. How does one synchronize symmetrical access to a shared resource without a common clock? The reason why the client/server architecture is so popular in distributed systems is because the architecture introduces an asymmetrical node that provides synchronized access to one or more shared resources; thereby, avoiding race conditions.