3

I have a couple of web applications that write to their own databases. They also share a few entities, for example, the customer entity. My business case is such that the same field for the same record can be updated around the same time in both the applications. It brings in dual sources of truth. How can one ensure data consistency and integrity in such a scenario? What's the best integration architecture to work with here? Is optimistic concurrency achievable in such a distributed architecture with multiple databases? What are some architectural ideas/themes that one could explore here to ensure that we have some form of data consistency going on even if the consistency model needs to be eventual?

What complicates matters here a bit is that both of the web applications are SaaS applications who's read and write APIs to their databases are tightly coupled to their respective front end UIs. So, it is difficult to decouple the UI from the database and segregate the read and write pipelines to maybe read and write from a distributed commit log in a event sourcing based architecture.

Any help here would be deeply appreciated. Thanks in advance...

mantadt
  • 39
  • 1

2 Answers2

3

There are multiple ways to go about solving the issue of having distributed entities.

Avoiding it

You could make sure the customer entity only exist in one place and have the other parts of the system modify it. It could be done by having a microservice for the customer entity.

Events

Instead of modifying the entity directly, you could trigger an event that is consumed in a central place. It will require more infrastructure but it will give you the benefit of having an ordering of the changes that can be observed.

A more in-depth walkthrough https://medium.com/better-programming/event-driven-architecture-as-a-strategy-dfb8370724c9

Saga pattern

It is probably overkill for you case, but it is a way to ensure consistency in a distributed system. https://microservices.io/patterns/data/saga.html

Frederik Banke
  • 311
  • 1
  • 5
0

Just Pick One

One option is to let the two systems operate independently, then if there's a conflict detected (one that you can't automatically merge) when the two synchronize with each other, have both systems agree on a winner and go with that.

The critical part is that the choice has to be deterministic, so that both servers will always come to the same conclusion. This could be as simple as "server 1 always wins if there's a conflict", or it could be something a little more involved like taking the hash of the two conflicting records and always choosing the one with the lower or higher value. How you decide is a business choice, the important part is just that both servers come to the same answer (and thus the two systems are consistent).

If the records are simple enough that one side losing their changes isn't a big deal, or infrequent enough that it's not a major issue, you can probably stop there.

If you need to avoid changes being lost, you can still choose one to "win" (as above), but then flag that record as having a conflict and store both the winning and losing versions so someone can come in later to resolve the conflict. This could potentially be a lot of extra work to implement, so I would only do it if the value of the otherwise-lost data outweighs the cost of the extra work.

Mike B.
  • 111
  • 1