46

I am a very solid Relational Database guy and understand all the way to 3rd normal form, appreciate the algebraic set theory roots of SQL, and can probably relationalize a broken heart (or not).

I haven't figured out a relational database structure FOR date nights with my wife, but I HAVE thought about relational database projects ON date nights with my wife..

Now I'm hearing about NoSQL, and researching it. Cutting to the chase, is there anything about NoSQL that is ground breaking, mathematically novel, or a "hey you don't even really need to organize your data relationally, this is so much easier" type of approach?

Is NoSQL like a super shell to the data structure? In my mind, data must ultimately have structure to be retrieved and the retrieval must be defined in a language of some sort.

Samuel Fullman
  • 363
  • 3
  • 8
  • 2
    What types of NoSQL databases don't have a structure? Document databases can be hierarchical, but minimally store their data in documents that contain the data in some format. Graph databases and key-value stores are rather self-explanatory. What NoSQL databases don't have a language to query? Some document databases are simply textual search or XQuery for XML, as two examples. SPARQL is used for RDF stores. – Thomas Owens Aug 13 '14 at 02:13
  • 2
    Update for "I HAVE thought about relational database projects ON date nights with my wife.." :) LOL. Good question too. – Rocklan Aug 13 '14 at 05:20
  • 2
    NoSQL databases do have structure, but the structure isn't always easy to map to relational algebra. Different NoSQL uses different structures, some are key-value hashmap, hierarchical, object database, graph database, or document store are common types of NoSQL. All that NoSQL really is is the realization that SQL isn't a panaceae, that some problem domains maps poorly to SQL/relational algebra. – Lie Ryan Aug 13 '14 at 10:41
  • 7
    NoSQL is a misleading name. It implies a group with characteristics while the only common characteristic is being not a classic relational SQL database. It's an open set. It's like describing every food that isn't bread as the "family of non-breads". – Pieter B Aug 13 '14 at 13:48
  • NoSQL... non traditional database systems... you mean like [LDAP](http://en.wikipedia.org/wiki/Lightweight_Directory_Access_Protocol) from the '80s? ;-) –  Aug 13 '14 at 17:44
  • LDAP is hierarchical, it's not like NoSQL – Charles Salvia Aug 13 '14 at 17:53
  • @Pieter B, yeah it's a misleading name - lots of nomenclature sucks; everyone knows what NoSQL means. When people here NoSQL they basically think of like, MongoDB and things like that – Charles Salvia Aug 13 '14 at 17:54
  • 1
    @CharlesSalvia Then you are very welcome to go ahead and explain below what NoSQL really means, MongoDB and things like that ;) – Daniel Dinnyes Aug 13 '14 at 19:08
  • 2
    Ok what's the big fuss about NoSQL? It's easy: we had a generation of people that grew up with relational databases and that was the only tool they had. Because if you only have a hammer, everything becomes a nail. Then you get ugly stuff like trying to put objects in a relational database or building a search-engine on top of that. The big realization was that: an SQL database is good for a lot of things, but not everything. the "not everything" is the big thing. – Pieter B Aug 13 '14 at 19:32

4 Answers4

24

NoSQL is more evolutionary than revolutionary. It essentially combines the existing ideas of "external database storage" with "using familiar data structures, not relational tables."

There are more types of databases than relational, for example hierarchical databases. While archaic by today's standards, it meshed really well with the data structures of its data (e.g. COBOL records). The point is, the data in the database was modeled closely to how records were laid out in the programming languages that used them.

Fast forward to the invention of relational databases, where finally the database separated concerns and, when properly normalized, is a great way to visualize most types of data and relationships between data. It is really easy to understand compared to other types of databases. What it utterly fails at, however, is storing data in a way that mirrors objects and classes in a program. Hence, the invention of object-relational mapping. In other words, the design of the database is actually a hindrance to the design of the program that uses it, which is why we need ORM libraries such as Hibernate. While clean and consistent, there is always that nagging doubt in the back of my mind that something is not quite right there.

This gave rise to two more types of databases, object databases and NoSQL.

Both attempt to solve the issues introduced by relational databases while not exposing us to the mind-bending horrors of hierarchical databases. Data is still laid out in repositories that vaguely resemble tables, but in actuality are more like programming data structures than relational tables. While object databases follow mostly well-defined rules, my understanding is that NoSQL is rather arbitrary. For example, a table might be visualized as a hash table or an array. There is not an easy, well-defined way to query them using an arbitrary tool analogous to Oracle SQL Developer or SQL Server Management Studio.

The idea is that one may define data structures that are easily searched in code, rather than piecing together SQL queries that are better-suited to a SQL database engine rather than expressing the query one desires. For example, fuzzy or partial matches are more difficult and perform worse in a relational database, while a NoSQL database may have a structure that is optimized for such a search and completes in a fraction of the time.

There are languages for querying NoSQL. However, there is no universal language such as what SQL is for relational databases.

Late Edit:

While I am familiar enough with NoSQL databases, this question was the impetus for me to buy a quality book on the topic and to start reading it with the eventual goal of being a real expert on the topic. The remaining comments are based on NoSQL Distilled: a Brief Guide to the Emerging World of Polyglot Persistence by Pramod Sadalage and Martin Fowler.

The authors state that relational databases do not scale well to clusters capable of serving the data needed for sites such as Amazon and Google: NoSQL was developed to fit this niche, relaxing the concurrency and durability in ACID in order to server large number of queries that largely use static data (hence, ACID transactions are not as important).

Furthermore, they posit that NoSQL databases operate without a schema (page 10) which allows NoSQL databases to modify the structure of data more easily. I am not sure that the presence or absence of a formal schema matters in this regard, since SQL databases allow modifying schemas as well. Regardless, the two renowned authors make the claim so it is worth examining.

I believe that both of these main points serve only to enforce my primary point that NoSQL is evolutionary, not revolutionary. They still store data, and make incremental improvements to the scale and modifiability. They also make the point that NoSQL does not seek to usurp relational databases as the king of data storage, only to provide an alternative means of data storage for the types of data that need to scale and morph in a way that (they believe) relational databases do not support well enough.

  • 2
    Some NoSQL databases do have a query language. I've used XQuery and SPARQL before, if the data is stored in XML or RDF. Those languages tend to be structured and for querying. But then again, that only covers NoSQL databases that contain well-defined data formats and don't do much for text or key-value pairs. – Thomas Owens Aug 13 '14 at 02:18
  • @ThomasOwens I edited my answer to be more specific. –  Aug 13 '14 at 02:33
  • so my thoughts on this are, 1) YES I realize that programming structures like JSON objects are different and I have for years seen that that structure, to be expressed in SQL, is sometimes quite complex (though not undoable). 2) At the same time, the basis for SQL was algebraic set theory, but it doesn't appear that there is (necessarily) some super-elegant (that is what I meant by super-shell) entity, but rather, I see elements of set theory in comparing arrays in some sample MongoDB queries http://docs.mongodb.org/manual/tutorial/query-documents/ But good summary, thank you – Samuel Fullman Aug 13 '14 at 02:57
  • @SamuelFullman I wrote all that and did not even think to talk about relational calculus or set theory. Then again this is Programmers SE, not Computer Science SE. Anyway, the feeling I get from NoSQL is it is based more on programmatic needs and less on math. –  Aug 13 '14 at 03:01
  • 1
    This is an interesting overview of NoSQL but doesn't really answer the actual question. As far as I can tell the only thing "groundbreaking" about NoSQL is that your data doesn't need to conform to a specific structure before being stored. – Rocklan Aug 13 '14 at 05:24
  • 'NoSQL' is to grove theory http://xml.coverpages.org/groves.html and tree automata http://books.xmlschemata.org/relaxng/relax-PREFACE-3.html as SQL is to relational theory - just like SQL having tables instead of true relations, NoSQL grew out of the theory, but the practicalities of the day mean it's not a pure implementation. – Pete Kirkham Aug 13 '14 at 10:34
  • "my understanding is that NoSQL is rather arbitrary" - My understanding is that for all the hype it's basically a persistent `HashMap`. Which is useful in many cases, but not generally sufficient in and of itself. Think of it as sideband storage for things that don't benefit from being stored in a relational way. – aroth Aug 13 '14 at 12:44
  • @aroth The reason for the hype is likely that the performance is much better when you're dealing with very large record sets. – Casey Aug 13 '14 at 13:37
  • @LachlanB I did address the questions in the question body, but not the title of the question. I will add a note about that to my answer. –  Aug 13 '14 at 14:20
  • 2
    `In other words, the design of the database is actually a hindrance to the design of the program that uses it, ...` I get the feeling this is putting the cart before the horse in a lot of cases. For big businesses with big data sets, the data is incredibly valuable and will be around for a very, very long time - longer than the current hot programming languages, tools, and paradigms. It might be more accurate to say that OOP is a hindrance to database design, and trying to change the database design to fit the programming paradigm may not be the best idea. – Doval Aug 13 '14 at 14:42
  • 2
    I'll just point out we are all using one heirarchical database implementation pretty heavily still -- it is called a filesystem. – Wyatt Barnett Aug 13 '14 at 16:44
  • @WyattBarnett that is very true, and I respect that. My experience with non-FS hierarchical databases is with an old military COBOL system that is older than I am. Very difficult to grok for a 70s child. I actually bought a book the day I wrote this, and thanks to Amazon Prime, have had some time to review it. I updated my answer with additional notes. –  Aug 26 '14 at 02:30
13

I think you'd definitely like to look at this paper by Erik Meijer & Gavin Bierman, titled "Contrary to popular belief, SQL and NoSQL are really just two sides of the same coin". In short, it claims that mathematically speaking both approaches base on the same theory, but with some differences.

Couple of interesting differences are, from my opinion, the following: the direction of cross-type dependencies (FK in SQL) are the opposite in SQL and NoSQL and collections type is not limited to set in NoSQL (and therefore some set-theoretical operations might not apply in NoSQL world anymore, but some others are still valid). Yet another interesting point from the article is the single query language proposed for querying both SQL and NoSQL databases. It's called LINQ, and if you think you might've heard this name before, you're correct: that's the Microsoft's querying language from C#.

Thomas Owens
  • 79,623
  • 18
  • 192
  • 283
Haspemulator
  • 456
  • 2
  • 7
  • 1
    "Yet another interesting point from the article is the single query language proposed for querying both SQL and NoSQL databases. It's called LINQ", not entirely true, 'Linq' isn't mappable to SQL in a 1:1 fashion, so a translation has to occur to make it work on SQL DBs. WHich means anything can be introduced to query both, as long as there's a translation layer making sure it runs on the target DB – Frans Bouma Aug 13 '14 at 09:25
  • 6
    Well, one can argue that SQL is also not directly executed on a database. What is executed is an execution plan, and one can also argue that Linq could be translated directly to the execution plan. So no big difference after all. – Haspemulator Aug 13 '14 at 09:49
10

Snowman's answer correctly describes how SQL and NoSQL differ in their data structures and how these are accessed. However, a probably even more important difference is their respective problem domain.

NoSQL is not a successor of SQL. Rather, the various branches of NoSQL sacrifice some qualities of SQL in order to be better at others. The CAP theorem states that it is impossible for any distributed database system to satisfy all of the following properties:

  • Consistency
  • Availability
  • Partition tolerance

Thus, some NoSQL variants follow the BASE principle instead, which relaxes the always-full-consistency constraint of ACID, which is the basis for classical SQL databases. By losing some consistency guarantees, they gain the possibility of combining high availability and partition tolerance in widely distributed systems, such as for websites with high amounts of data and user queries, but little demand for perfect consistency. Thus, such NoSQL databases are at the heart of Google, Facebook, and Amazon. So, to answer your question: Yes, NoSQL is ground-breaking in that it pretty much enables such massive web services.

This is only one example, as NoSQL is a diverse field, and its variants cover pretty much all possible combinations of parameters within the CAP triangle.

Florian von Stosch
  • 200
  • 1
  • 6
  • I am not familiar with ElasticSearch. A [quick google search](http://aphyr.com/posts/317-call-me-maybe-elasticsearch) seems to show that it sacrificies consistency (C), and is therefore AP, not CAP, which is theoretically impossible. Edit: This was in response to a now gone comment suggesting that ElasticSearch fulfills all CAP properties. – Florian von Stosch Aug 13 '14 at 15:39
  • 3
    Oh how cute, they went with BASE to counter ACID. Is there a middle-ground approach called REDOX or SALT? – Patrick M Aug 13 '14 at 18:43
3

Common use cases of NoSQL are groundbreaking in their productivity gains as compared to common SQL-based databases. There are several factors in this.

One is housekeeping. Most NoSQLs are open source and can be installed on a workstation or a VM with a few commands, and work with reasonable defaults out of the box. In my experience, even Postgres and MySQL are not that way; configuration is usually necessary to get started even on a workstation for development purposes.

Another is convenient development, as other answers have described in detail. The JSON indexing capabilities of Mongo, or the key/value semantics of redis and Riak, may be all certain webapps need to get the job done, and the APIs are easy. Some NoSQLs provide their own RESTful APIs, whereas with SQL you typically have to write them yourself.

These factors make NoSQL databases attractive for small-scale projects. The pick-up time tends to be low. Sure, when you go to production you have to configure for security and scale, but the ability to start coding and collaborating quickly is powerful and, I argue, groundbreaking.

Also, related to the above, for small-scale applications (like company-internal or app-to-app services), a team may be able to stand up a production NoSQL database without involving their DBA teams, and without suffering performance or integrity issues as a result. Professional DBAs may not like this, but developers who see DBAs as a source of obstruction (right or wrong) sometimes see NoSQL as a way to bypass having to deal with them. I confess to this - I once changed a small-scale application from Postgres to SQLite to cut out the adversarial DBA, and I have chosen to implement on Mongo rather than Oracle to avoid DBA approval processes and access restrictions. With no adverse consequences in either case.

wberry
  • 481
  • 3
  • 8