1

So, in terms of systems design:

  • given a highly scalable system being served through a single load-balanced anycast IP address,
  • when that system receives a load in the order of (hundreds of) millions of requests per second,
  • then how should the load balancing be designed to reliably distribute such load?

Elaboration:

I understand the concept of DNS round-robin as a technique to distribute load from a single domain name to multiple IP addresses, but I don't see it being applicable to an anycast IP.

From my current (poor) understanding of networking, a anycast IP address forces us to use a single machine/node at least somewhere in the request’s path, and I assume that vertical scaling on that machine would not be enough to satisfy the RPS rate in the millions.

Can someone help me wrap my head around this?

CBlew
  • 119
  • 3
  • 2
    Does not sound like a real question, more like "how to load 100 tons of gold into my Toyota Corolla at once"... You probably should find such system and ask its architect... Probably area of fiberoptic devices would get some closer number of requests but I don't think there is any DNS/IP at that level... Note that the scale you are talking about is roughly every device connected to internet sends request to that IP every couple of minutes... – Alexei Levenkov Dec 29 '20 at 02:17
  • Apparently, I was stupid enough to type 'unicast' instead of the correct 'anycast'. – CBlew Dec 29 '20 at 06:22
  • I know absolutely nothing about that but based on https://en.wikipedia.org/wiki/Anycast it is roughly the same as DNS round-robin just at IP level, and whatever case described in the post does not apply to such routing... – Alexei Levenkov Dec 29 '20 at 09:09

1 Answers1

1

A DNS round robin is just a way to distribute the hits to different backend servers in a (at least to the client) seemingly random order. This means you have one data center and distribute requests to your computers at that one center.

The anycast idea is to check the IP of the client and based on that information, send the client of a precise server based on one's location. The search of the location for an IP address should be fast enough for that front end computer to support many requests, but certainly not 100 million per second.

So in most likelihood, assuming you really get that many requests, you would probably need a bit of both. Your DNS servers would use a round robin mechanism to send the user on one of your front end computer which will search for the location based on the client's IP address.

Second you get that search and the IP of a server close to the client (i.e. a client in Sacramento would get an IP to your server in Silicon Valley, a client in Austin would get an IP in Houston).

Then in these local data centers, you again have a front end which distribute the load between various backend servers.

That's how very large Internet companies (Google, Facebook, Amazon, etc.) build their infrastructure. It's very unlikely that you would need more than the first example mentioned above, though.

Now it sounds that what you're looking for is to determine which backend to send the request to. That means knowing whether the backend is already on its knees serving some existing clients and instead of using round robin, use a smart distribution. This is doable with a system such as HAProxy which is probably what you're looking for. Each backend can be fit with a small client which gives HAProxy info about its current load. Based on the load of each backend computer, HAProxy can then make a smart decision and send requests to the backend with the smallest load.

Is that even necessary? That very much depends on your type of service(s). If your backends all reply within 500ms or less, you can use round robin all the way. If you have backends that have a huge discrepancy in the amount of time it takes to send a reply (i.e. 0.5 to 30 seconds), then a front end service like HAProxy is very useful.

Alexis Wilke
  • 241
  • 1
  • 7