5

I'm developing a microservice system. I have a discovery service (Eureka) for all my services and an API Gateway (Zuul) to call each service via an URL.

My problem: How can I assure that a service call via my API Gateway is high available?

If the gateway dies, my services aren't callable anymore. I could start multiple gateways but then the client needs to know which gateway to call. Wll than I need a load balancer between the client and my gateway. But what happens if the load balancer dies?

I could set up a second system and route a DNS entry to both systems but still, a broken load balancer could break one complete system?

So, is there a good solution to make load balancers or a api gateway high available? Or will there ever be a single point of failure per system?

Milgo
  • 161
  • 1
  • 5
  • On the one hand. making things reliable is not the same than making them available. If you need high availability you have to deploy at least 3 replicas of the Eureka (the more the better), as many Zuuls and services as your SLA requires. On the other hand, making thing reliable is a matter of implementing and configuring things well. So, what's your question? Making things reliable or available? – Laiv May 09 '19 at 14:46
  • Sorry, meant availability. But my clients do only know one Zuul URL, so Zuul is always a single point of failure, isn't it? Or the load balancer in front of Zuul is. – Milgo May 10 '19 at 05:40
  • You can have as many layers as you need between the Zuul and the API clients. But yes, there should be a good balancer in front of all of that. In front of the load balancer, there could be something else, for example, an API Manager which usually also acts as API Gateway (among other things). – Laiv May 10 '19 at 06:22
  • And why is an API manager not a single point of failure? – Milgo May 10 '19 at 07:58
  • Well I'm searching for a pattern to avoid any single point of failure. I want to give an URL to a client and be sure that if one single service instance fails my complete system isn't broken. I don't wish a single point of failure at all. – Milgo May 10 '19 at 08:19
  • A way large companies solves this is deploying the architecture in several regions as different clusters. Each region deploys several nodes of each component of the architecture (for obvious reasons). But in front of all of that, there's always a single point of failure the clients request in order to reach the services. Unless you make your API clients smart enough to switch from one region (cluster) to another, but then you have to provide the clients with such information, making the scalability of the services less flexible – Laiv May 10 '19 at 08:25
  • 1
    VTC; This is really more a system and network engineering problem than it is about software and is a well-solved problem. – Blrfl May 10 '19 at 20:13

2 Answers2

5

I would have thought this would have an answer by now as its a standard problem.

I'm going to have to give a fairly generic answer im afriad as its not my specialty.

Basically you have two load balancers, a master and a slave and a way to switch between them if one fails. Usually a shared ip address which is either active or not.

Now you have both boxes talk to each other so they can detect when one fails.

If the master fails, say its unplugged, the slave detects the lost connection and activates, becoming master. Usually by grabbing the network traffic for the shared IP address.

You can have whole sets of machines in such a cluster, as long as they all talk amongst themselves and decide who is the master you are good.

Ewan
  • 70,664
  • 5
  • 76
  • 161
2

Rather than reinvent the wheel I would explore software as well as potentially PaaS offerings in various cloud providers that solve all the sticky problems of HA and dealing with single points of failure.

There are hardware load balancers (eg. F5 Big IP), software load balancers (eg. ngnix, HAProxy, Kubernetes, etc...) and cloud load balancers (AWS Elastic Load Balancer) and all of these have redundancy and various failover schemes to prevent single point of failure.

In terms of DNS records, there are various strategies that employ this as well to ensure that redundancy and failover can be handled smoothly. Scripting the update of DNS records, adjusting TTL on DNS entries, utilizing a CDN like Akamai for automatic DNS resolution and geolocated endpoints. Many cloud providers have platform features that you can leverage here as well.

The possibilities are legion and one can write many books on the subject as a whole, so it is important to do your research and find out what makes sense for your application architecture.

maple_shaft
  • 26,401
  • 11
  • 57
  • 131