How to handle side effects in CRQS when replaying events?

Question

It is said that in CQRS it is easy to fix a bug, you just redeploy and then replay the events.

But, what if one of the events should cause an external system not in your control to "ship an item" to the customer if you just replay the events the item would be shipped twice.

How do you resolve that?

Answer 1

You need to make a clear separation between events modifying the state of your read model, and events (potentially) modifying the state of external systems. Make sure you do not have any "mixed events" modifying both states together. That way, you can replay your events in a specific "replay mode" where those events for the external system are not fired again. In this mode, you also "simulate" any events which had been originally been initiated by the external system (now you take them from the replay queue instead).

Don't forget, the redeploy step means actually to reset the state of the read model to an earlier point in time. This is probably nothing you can do (or need to do) for the state of the external systems.

Answer 2

From Martin Fowler's event sourcing article:

The fundamental idea of Event Sourcing is that of ensuring every change to the state of an application is captured in an event object, and that these event objects are themselves stored in the sequence they were applied for the same lifetime as the application state itself.

So when you need to restore the state of your system to a certain moment in time you replay the stored state, not the event handlers, up until that moment.

That being said, if you're only working with state data, there shouldn't be any effects on the external system. Unless you have triggers or watchers on your event store in which case you should disable them for the duration of the restore. Since you say that you have no control over the external system there shouldn't be any attempts to restore its state by using the exposed API since you don't know what side effects it may have in their system. If the restore puts the system in an intermediate state (eg. due to failed operations in the external system) this shouldn't fall within the responsibilities of an event replay.

Answer 3

But, what if one of the events should cause an external system not in your control to "ship an item" to the customer if you just replay the events the item would be shipped twice.

To choose a specific example, let's consider how an "at least once" approach to side effects might work.

State currentState = State.InitialState
for(Event e : events) {
    currentState = currentState.apply(e)
}
for(SideEffect s : currentState.querySideEffects()) {
    performSideEffect(s)

So the domain model tracks what needs to be done; but leaves the actual doing to the application

In the context of running a command, the basic idea looks the same. The actual side effects happen outside of the transaction that updates the model.

So the unit tests for your model could looks something like

{
    // Given
    State currentState = State.InitialState

    // When
    Events events = List.of(OrderPlaced)

    // Then
    List.of(SendEmail) === currentState.applyAll(events).querySideEffects()
}

{
    // Given
    State currentState = State.InitialState

    // When
    Events events = List.of(OrderPlaced, EmailSent)

    // Then
    List.EMPTY === currentState.applyAll(events).querySideEffects()
}

The main points here being

• Updating the model is side effect free; the actual side effects happen outside of the transaction that updates the model.
• An event describing the outcome of the side effect needs to come back.