2

The problem

I have a number of clients connected via websocket to nodes of my web application through a load balancer. What I need is to deliver notifications on a per-user basis.

My idea

My idea is to have each web application node connected to a messaging system (RabbitMQ, Apache Kafka), and have each node creates a queue (eg. "node1-queue") and then to have a message publishing system that is aware of the relation user -> queue.

Now, this seem a broad problem I wonder if I'm, reinventing the wheel and if some framework already provides this functionality out of the box.

Piero Nadello
  • 31
  • 4
  • A similar idea had been implemented before. The [Plezi messaging system](https://github.com/boazsegev/plezi/blob/14854c858b8bb2add21b4d261748fe12ff603cd0/lib/plezi/websockets/message_dispatch.rb) works this way, with each application instance listening to both a global pub/sub channel and a private pub/sub channel. Global messages are routed through the global messages and private ones are routed using the the application instance's private channel (the application instance will forward the message to the specific connection)... maybe you can use this design. – Myst Sep 24 '16 at 05:34

1 Answers1

2

Why not just broadcasting the message to all the nodes, and then let the nodes chose the messages they want to forward through WebSockets to clients?

If you have a lot of messages (that much that the impact on the network becomes noticeable), using the user ID in the routing key may be the easy way to reduce the traffic between the MQS and the nodes.

Imagine you have a chat application with rooms and users. Using a topic exchange, you may use routing keys such as chat.room123.user456 when transmitting a message of user 456 posted in room 123. You can consume those messages by having a queue with a binding key chat.room123.*, which will listen to all messages in a specific room. You may also want to consume the messages posted by a specific user, independently of the room, using the binding key chat.*.user456. In order to scale it, you may have some servers handling some rooms, and other servers handling some users. For instance, a given server may have WebSockets connections to users 17, 41 and 48, and have the binding keys chat.*.user17, chat.*.user41 and chat.*.user48.

The tricky part here might be the handling of the case where the WebSockets connection is dropped, and the client reconnects to a different node. Sticky sessions may solve this issue. Aside that, handling errors and loss of WebSockets connection is not different from any other scenario where messages from a message queue service should be preprocessed and sent to a third party.

Community
  • 1
Arseni Mourzenko
  • 134,780
  • 31
  • 343
  • 513
  • I have to assume *lot* of messages, for the moment. Do you have any source to elaborate on "using the user ID in the routing key"? – Piero Nadello Aug 02 '16 at 10:49
  • Don't assume. Measure. If you assume, with no statistical data to support it, you're doing premature optimization and searching for answers which may not matter at all in your case. As for your question about the routing key, I would imagine a topic exchange, and routing keys such as `chat.room123.user456` (for instance if you need to be able to listen to `chat.*.user456` for all chat rooms, as well as to `chat.room123.*` for all users of a chat room). – Arseni Mourzenko Aug 02 '16 at 11:11
  • If you could add an external resource (a link to some documentation basically) to the topic exchange argument to the answer I'd be happy to mark it as solved. :) – Piero Nadello Aug 02 '16 at 12:05
  • Also, will this solution scale well, given that a node will endup listening for --as-many-as -possible- users? – Piero Nadello Aug 02 '16 at 12:11
  • @PieroNadello: external resource? All I was talking about is in the [official RabbitMQ tutorial](https://www.rabbitmq.com/tutorials/tutorial-five-python.html). This is just the beginner's stuff you already know; when writing my answer, I had nothing advanced in mind. As for your scalability question, the node won't listen for *all* the users; that's the point of routing. If the node subscribes to `chat.*.user123`, it won't receive any messages of `user789`. – Arseni Mourzenko Aug 02 '16 at 17:27