Relationship between RESTful URIs and PubSub topics

Question

I am designing a system which fits quite nicely into a RESTful architecture. Users can navigate a resource hierarchy from a root node, each resource links to other resources, resources have URIs etc. In many ways this is very similar to 99% of the CRUD web applications out there.

Where things get a bit different is that I want to integrate a Pub/Sub model into this architecture. So when a resource is modified I want the architecture to support updates being pushed out the relevant subscribers.

Many of my questions relate to how Pub/Sub 'topics' relate to RESTful 'URIs'. Really I want to just use the URI as the topic. I feel like in many scenarios that makes perfect sense - but I have a niggle of doubt about this and can't find any architectures in the wild doing this.

Where it gets strange is is that some URIs are queries do not fit easily into this model. It's way too complex to support pub/sub dynamic queries and I have no intention or need, but the fact this concept is widely implemented in RESTful systems makes it feel like there is a mismatch or limitation on using URIs in as topics in pub/sub system.

My resolution at the moment is that 'canonical' URIs (i.e. those that refer to a single resource at a canonical location) are fine, or even a good thing, to use as a topic. But my confidence is low due to not seeing other people doing this.

Any thoughts appreciated.

Answer 1

Mapping URIs (from REST resources) to topics, depending on what your are modeling, could result in a perfect mapping.

Through a simple example, i will try to give you some hint about how to perform this mapping. I would take as an example, two APIS: a REST API (resources) and MQTT API (topics).

By the way, MQTT is a pub/sub protocol for the Internet of Things and M2M: there are some topics located in a MQTT broker where clients can publish (PUB topic1/topic2/...) or subscribe (SUB topic1/topic2/...) .

Hint 1: A resource representation is a message

The idea is to map your resource URI with a corresponding topic name. So the resource representation from a resource URI corresponds to a message in a topic.

Imagine you have a resource name /people/:id, you will have a corresponding topic name /people/:id.

So when a client make a PUT /people/1 { "loveBeer":true } , clients that have subscribe to this topic (through a SUB /people/1) will receive a notification. Notice that PATCH could be use also. Same apply in the opposite, if a client post a message in the topic PUB /people/1 "loveBeer":true, the corresponding resource will be update.

You would end up with this kind of architecture (picture is taken from this blog) :

Hint 2: Resources creations are also messages

From the previous example, you could think...

Hey, what about client creating a new resource through POST /people ?

You will create a new resource and a message will be published in the corresponding topic (/people)

For example, if a client performs POST /people { "name":"schneider", loveBeer:true}. Clients that have performed SUB /people will receive the notification

So you could think again and say:

Hey, what about client publishing a message in a topic through PUB /people ? Vice-versa, it will create a message in the topic /people and create a new sub-resource /people/:newid

Imagine a client make PUB /people {"name":"Ailurus", "loveBeer":true}, when message is receive, a new resource /people/42 is created and the clients have subscribed to /people are notified.

Bonus hint: Consider meta-data

That's not really a hint, since it's not really related to a mapping between resource URI/topic :)

From this blog, meta-data could be added in the payload of the messages published (but not in your resource representation).

Mainly because MQTT Client that subscribe to a topic don't know if a resource is updated or created, whereas a REST client know this information (when a resource is created, through the location header) So consider to add a metadata such as "event": "created" or `"event": "modified"

You could even think about when a resource is created, in addition to put {"event": "created"} in the payload, to give the resource URI/topic name, such as {"event": "created", "location": "people/42"}

Answer 2

A subscription to a topic can be done with a POST to a topic URI. This should generate a queue that is returned in the response for the invididual subscriber.

Using POST on this queue URI should create a new URI that contains a feed of items (URIs to the messages) that have to be delivered to the user. After GET on this current feed state, you can GET selected items that the feed refers to. If the items are shared (this is probably what most people want), you cannot just DELETE them. You need POST again to mark them delivered.

A new POST on the queue URI will create a feed with current queued items again (possibly the ones that also haven't been marked delivered). But this is your choice how you want to handle the messages.

Of course the subscription can be DELETEd when you want to unsubscribe and it is advisable that some URIs (feeds and eventually items) have expiration restrictions to keep the system clean.