7

Say that we have Kubernetes running with n pods, each being a separate microservice. Now, some of those services need a caching layer. When we take into consideration:

  • the fact that Redis was tested to hundred of millions of keys and that it was performant
  • the fact that you can tag keys by namespaces (which would be microservice names in this instance)
  • the fact that the data would have a relatively short expiration time (we'd hardly hit the number of 10M+ keys at a given moment)
  • the fact that we have a single (although replicated) database from which data is fetched. PS: I know this goes against reactive principles. (Thanks to Ewan for reminding me to add this)

Would we spin up separate Redis pods, each corresponding to a single service or would a single Redis master pod with a separate pod for slave containers be a better architectural choice? Would having separate Redis pods be an antipattern of some sort?

A diagram to help you visualize the dilemma:

enter image description here

Milan Velebit
  • 507
  • 3
  • 13
  • what is the slow operation that the microservices are caching? ie database/api? – Ewan Mar 19 '19 at 16:33
  • @Ewan Yeah, roundtrips to the database. – Milan Velebit Mar 19 '19 at 17:11
  • a shared db or one per microservice pod – Ewan Mar 19 '19 at 17:32
  • @Ewan I've edited the post when you first mentioned the db, completely forgot to add that previously, so thanks for that. A shared db, as stated above now. – Milan Velebit Mar 19 '19 at 17:38
  • Are you going to deploy this to the cloud? If yes, have you calculated the cost of such architecture? Are you aware of the minimus required by Redis to work? Do you have any certainity about each MS needing so performant (and dedicated) cache? – Laiv Mar 19 '19 at 18:14
  • @Laiv Yes, yes, yes and for the last one: No. That's really the question, I was merely looking for the better practice between the two in the given conditions. – Milan Velebit Mar 19 '19 at 18:57
  • 1
    Then KISS. Be conservative, sacle it up/out as the needs comes. Otherwise you run the risk of oversizing the architecture and It could be the death of the project when you go to the cloud. – Laiv Mar 19 '19 at 19:08
  • @Laiv Thanks for the tip, could you add an answer summing up the stuff that you've said in this and the thread below? – Milan Velebit Mar 19 '19 at 19:10
  • 1
    @MilanVelebit done. It took me a while because I did from the smartphone. – Laiv Mar 19 '19 at 20:13
  • Would you need that cache when you would split up the database into a database per microservice? The question might arise from an effect rather than the source of the performance issue. – Luc Franken Jun 06 '19 at 17:39

3 Answers3

11

If we consider the cache to be orthogonal to the architecture (and it's), the first pic is ok.

For the same reason that we don't deploy one security service, one API gateway, one message broker or one service locator per POD we don't have to deploy one Cache (in replica-set) per POD.1,2

Be aware of premature optimizations

Caches are meant to solve specific performance issues. Mainly those derived from costly IPCs or heavy calculations. Without evidence of any of these issues, deploying caches (in replica-set) per POD for the sake of the "MS' God", is premature optimization (among other things).

The Cloud can kill you.

If our goal is to deploy the architecture in the cloud, the smart move would be to start small. Be conservative. Scaling up|out as the needs come because the contrary is oversizing the architecture. Oversized architectures are potentially dangerous in the cloud because they can, literally, kill the project devouring the ROI in no time. 3

Size the solution according to the problem

Perform load tests first, get metrics and shreds of evidence of performance issues, find out whether caches are the solution to these issues. Then, size the solution according to the problem. If and only if services are proven to need a dedicated cache, deploy them. By the time you do it, you do it uppon objective metrics and keeping the bills under control.

I was told once

If the solution is more complex or more expensive than the problem it solves, then it's not a profitable solution. It's not even a solution!

Emphasis mine

Keep complexity at bay

MS architectures are complex per se, they don't need us adding more complexity for the sake of dogmas or beliefs we barely understand. Keep the overall complexity of the system as lower as possible but not lower. In other words, keep it simple but not at the cost of defeating the purpose of the architecture (total freedom of deployment and SDLC).

1: I'm assuming that every POD is a replica of the same Service, not different services.

2: MS architectures are not about many but small systems, it's about a single system composed by business "capabilities", working all together for a greater good.

3: Cloud is anything but cheap. Particularly, when it comes to buy RAM

Laiv
  • 14,283
  • 1
  • 31
  • 69
7

Separate services should use separate REDIS instances.

Reason: Bad usage of REDIS by another service, causing REDIS outage, should not impact your application.

Only queues which are used for inter-service communication should be shared.

Sahil Gupta
  • 189
  • 5
  • But is resiliency really a factor here as Redis is configured with a master-slave replication, plus those slaves are replicated through multiple containers in a pod as shown above? – Milan Velebit Mar 19 '19 at 13:49
  • 1
    Lets zoom it out - consider 2 different microservices as 2 different companies altogether. Now would you allow a aws hosted redis service to be shared with other companies service ? – Sahil Gupta Mar 20 '19 at 02:17
  • 4
    IMHO As long those microservices are delovepped by the same team it is fine. Furthermore we don't have the whole picture of how those microservices are used all together, they might have no value if one is down aniway. If it was handled by different cmopany I would however consider definitively to split up the architecture. – Walfrat Mar 20 '19 at 08:26
4

Can you prevent that one service writes into the namespace of another service? If not, sharing one Redis instance breaks service separation. And soon Redis will be used as a way to sync data. Not to mention the risk that one service accidentally manipulates another service's data.

If you can enforce the separate namespaces via Redis, i.e. different access credentials with access only to their own namespace like it's possible in most SQL database servers, then sharing would be fine IF you accept having one redundancy bottleneck for all services.

Frank Hopkins
  • 340
  • 1
  • 5
  • From a microservice architecture standpoint, I get your point, but as you've seen in the post, we have a single main database so we've already pretty much diverged from a perfect architectural pattern. I can't really see a scenario where a service could overwrite another service's data if the services are configured to write to a single namespace, that issue would be *huge*. Thanks for the input! – Milan Velebit Mar 19 '19 at 21:04
  • 1
    @MilanVelebit I get your "pragmatism", and if you decide it's okay for you, well, your decision. But it's one more step in the "wrong" direction (reminds me a bit of the broken window theory). One big advantage of proper microservice design is exactly that it prevents any such "huge" issues and "misuse" of such central components to creep in until they are hard to untangle. I've never bought that it *needs* to be like this, you can also enforce modularisation in a monolith after all, but if you go with services, it's a sane way and what many expect to see. In any case, your decision. – Frank Hopkins Mar 19 '19 at 21:25
  • Fully agree, and yeah this reeks of a microlithic design.. :/ – Milan Velebit Mar 19 '19 at 21:54
  • 1
    While I agree with the thing about central things, I don't think cache is one of these. In this specific case, moving from one cache to dedícated one has no impact. It would be moving the calls from one DNS to another. There's nothing coupling because cache is not a backbone. It's not a ESB or anything similiar. If it's used as cache, there's nothing wrong with having one single provider who at the same time scale out/up independently from the rest. – Laiv Mar 20 '19 at 06:27