14

I have an idea in mind, but it still confuses me the database area.

Imagine that I want to show real time data, and using one of the latest browser technologies (web sockets - even using older browsers) it is very easy to show to all observables (user browser) what everyone is doing.

Remy Sharp has an example about the simplicity about this.

But I still don't get the database part, how would I feed, let's imagine (using Remy game Tron) that I want to save the path for each connected user in a database and if a client wants to see what is going on with a 5 sec delay, he will see that, not only the 5 sec until that moment but the continuation in time ...

how can I query a DB like that?

SELECT x, y FROM run WHERE time >= DATEADD(second, -5, rundate);

is not the recommended path right?

and pulling this x in x time ... this is not real data feed correct?

If can someone help me understand the Database point of view, I would greatly appreciate.

gnat
  • 21,442
  • 29
  • 112
  • 288
balexandre
  • 243
  • 3
  • 8

4 Answers4

7

Relational databases are optimized for relationships. Time Series Databases are optimized for storing and working with a series of values that change in time. In the industrial world, they call these "Historians". Probably the most famous is OSI's PI.

You can query across a set of signals for a given point in time, even if it didn't store a value for that particular point (it can interpolate between known points).

However, each query only gives you a single point in time. You still have to poll.

Continuing on the time series database theme, check out openPDC (a Phasor Data Concentrator is specific to the electrical grid, but it is a time series historian). Specifically, this quote:

streaming playback utility that can be used to replay events from the local historian archive and extract large datasets

That's really what you want... some kind of playback mechanism.

I don't think you really want a database. You want a bunch of FIFOs.

Scott Whitlock
  • 21,874
  • 5
  • 60
  • 88
7

Real Time and Database have nothing to do with each other.

When I made radars (and heat pumps and ship control systems) I learned that the real time data lives entirely in buffers and shared memory.

The data may also get sent to a database for subsequent analysis and retrieval. The volume has to be small, so it's summaries or digests or samples of data go to the persistent storage.

The actual real-time data, though, flows through the network via message queues for reliability or UDP packets for instant (but possibly fragile) notification.

S.Lott
  • 45,264
  • 6
  • 90
  • 154
4

Stumbling upon this question I think you're thinking about this wrong, and the other answers don't really highlight the problem.

Hammering your database isn't really a good idea. While I'm pretty sure you've realized this, others might not have. I remember a friend of mine tried to use a php script and a Javascript AJAX function in a loop for a semi-real time game. He very quickly realized that performance degraded as more people joined, simply because he was executing a ton of queries per second which hammered the database.

The way I see it, there needs to be a long running application that sits between the user and the database. Users connect directly to this application and pass all its information to the app. This app them immediately copies the information to everyone connected to it. This loads to a very lightweight app, little traffic, and very real time.

If you must use a database, the app must then handle this in a background queue of updates. Obviously you need to pick and choose what you save as you don't want to hammer the database with a wall of small bits of information (like current position of a user in a game), which brings you back to the same problem you started with. Instead of position, store distance traveled over 5 seconds or other more meaningful bits of information

TheLQ
  • 13,478
  • 7
  • 55
  • 87
1

This site has some "real-time" features. You'll find that many high-traffic sites have many toys between the web server and the database. Redis is something you can use to hold data in memory. I'm sure there are others for various stacks.

I think Firebase.com is a great real-time data service (Not sure what they do for the actual database backend, but since they serve it up, I'm not sure I really care beyond curiosity.).

JeffO
  • 36,816
  • 2
  • 57
  • 124