CQRS: How to restore read model

Question

It's very interesting how to restore read model in system based on CQRS.

In regular mode system processes commands, creates domain events and posts them to message bus. Then another part of system (call it RM subsystem) processes these messages and saves them to read model. This mode is good enough for regular purpose.

But how should I repair my read model? For example storage with read model was corrupted or I changed location of storage. I want my system to restore read model during initialization, before queries begin to try read data. And I want to know the end of repairing process.

I can imagine two ways:

1. Create REST controller, throug which my RM subsystem will be able to query all domain events (in messages forms) and restore it synchronously.
2. Create special mechanism, calling which my RM subsystem will be able to start replaying all messages. As for me, this way isn't very good, because I cannot control time of finish of repairing process. And the second, if there are other consumers of messages, they probably can corrupt their data.

Which way is more preferable?

Answer 1

Both methods are totally fine, and as usual, the answer is: "it depends".

The 2nd method you suggested is easy to implement and is used quite often - in particular, I'd consider it to be the standard method to bring new event handlers/projections online into an existing system, since new consumers need to be replayed the full event history at least once.

Regarding other consumers, please not that

• you can (I'm tempted to say should) make them idempotent consumers, which at the same time helps with the "at least once"-guarantee paradigm of common message bus systems, and
• you can always opt to only replay events into a single consumer, such that other consumers are not affected by the selective replay.

Furthermore, have you actually measured the time that it takes to replay all events into a single projection (the corrupted one)? Usually you can easily handle tens of thousands of events and read model updates per second (use one big transaction), so replaying all events to repair a single read model should be a matter of minutes. We actually replay all events on every system startup, since our read model is only stored in memory, and it's blazing fast.

If your event store/messaging infrastructure does not support query by event type, the 1st method you suggested is a bit harder to implement, since you need to implement the query interface. This might be extremely hard or it might be trivial, depending on how your event store is designed. So if you don't want to use the second method you suggested, implement the 1st method, use selective queries to repair a read model and call it a day.

Answer 2

If the query model ("read model") is stored in a full-fledged transactional database, normally you do not have to care about such things, because "reparing a corrupted read model" should be possible by the backup/restore mechanism of your database. So lets assume this is not the case (maybe you backup your read model only once per night), whilst your command model is fully backuped (maybe in some kind of sequential log). This means a repair of the read model will be include a restore from the latest backup and afterwards executing the commands from the command model. (Note that transactional database internally work with a transaction log which works exactly like that, allowing you to restore your "read model" up to the point immediately before a system crash).

I am not an expert on CQRS, but when I understood that concept right, there is a time lag to expect between the state of the data in the "query model" and the state of the data implied by the command model. So your system should be able to handle eventual consistency anyway, which is what you have to deal with when choosing an asynchronous repair process. Therefore, option 2 is a viable solution, as well as option 1.

The difference is that option 2 might bring your system faster on-line back again after a corruption of the read model, for the price of a bigger time lag until the query model and the command model are in-sync again. Option 1 will block your system during the repair process, but when the system goes on-line again, the consistency time lag between the two models is (hopefully) much smaller.

So what you pick depends on the acceptable trade-off between availability and consistency for the particular system, and how much time the actual "repair process" for the system will take. The latter will also depend on speed of the backup/restore process of the databases involved, the amount of data to process, the number of users, the traffic involved, and so on.