3

I'm doing an project where I'm using a microservice architecture. All the services within this architecture communicate with each other using an AMQP broker (RabbitMQ).

When something happens in a service, it publishes an message to RabbitMQ and the services interested in this message can process it whenever they are ready.(Eventual consistency). Now I need to build functionality that can estimate the impact of a change in a message by checking which consumers are effected by that change.

My question is:

When I change the format of a message as a producer, how can I know which consumers are going to be affected by that change?

The whole idea of using a AMQP broker is to be sure that the services can communicate with each other without them knowing each other. So the producer doesn't know the consumers. That's what is making this pretty confusing for me.

gnat
  • 21,442
  • 29
  • 112
  • 288
CGeense
  • 191
  • 1
  • 5

1 Answers1

2

Regardless of what kind of channel or medium of communication exists between two interfaces, when the producer modifies the format of the message that consumers know how to accept, there is a risk of change on clients.

The whole idea of using a AMQP broker is to be sure that the services can communicate with each other without them knowing each other. So the producer doesn't know the consumers. That's what is making this pretty confusing for me.

This isn't the reason one would use asynchronous message queuing in an application. This is the reason one would employ SOA (Service Oriented Architecture) and Microservice architecture. Think of message queues as merely a communication channel with some special properties like zero-message loss, guaranteed message delivery, asynchronous messaging, and easier ability to develop reversible transactions on a message.

The format of the message is essentially the contract that your producer and client agree to when exchanging messages. The client need not know about the inner workings of the producer, however it must be able to reason about the producer in terms of the contract. Software components are reasonably decoupled. This means that if the producer were persisting data to a database, then you could feasibly refactor the database, and as long as you don't change the format of the message then your clients are reasonably abstracted from this change thus minimizing the risks.

That is what Software Architecture is really all about. It is measuring and reasoning about qualitative attributes of software (modularity, composability, etc...) to make decisions and trade offs to mitigate risks.

If you must change the format of the message, then you are modifying the contract. The best way to proceed forward here is to take a disciplined approach.

  1. Formalize the format of your message. Structured formats like XML are good here because I can define an XML schema that I can use to enforce and validate a message.
  2. Version your XSD so that it is easier to reason with, identify, and describe clients that use your service.
  3. Stub the service with a mocking tool. You can do this with unit testing tools or something more sophisticated like SoapUI to emulate a provider message format change that hasn't been implemented yet.
  4. Perform proper unit and component testing of your clients against your stubbed interface and verify your use cases.
  5. Implement your provider change and then unit and component test this against your use cases.
  6. Perform system integration testing to verify that clients and providers are talking correctly to one another.
maple_shaft
  • 26,401
  • 11
  • 57
  • 131
  • With the disciplined approach you are already thinking towards a solution for the problem. But my problem is that i need to recognize which consumers are breaking because of a change in the contract So lets say im in charge of an application with multiple services and one developer walks up to me and asks me to change field Foo into field Bar. Then i need to check which consumers break after that change and what the impact would be of that change. I hope this is a better explanation. I was thinking that consumers could reject the message in some way and the producer gets notified – CGeense Oct 05 '16 at 12:52
  • @CGeense I get what you are saying, but I think you misunderstand the whole point of microservices. You wouldn't expect an ATM to be able to tell if your wallet is empty. If you are the arbiter of your Wallet, then it is YOUR RESPONSIBILITY to ensure that you keep it well stocked and that you maintain your bank card. The bottom line is, you have no control nor knowledge over how the client developers decided to develop their component interface. If you feel you need to add a new field then your job is simply to do so, communicate the upcoming change and new contract to client devs .... – maple_shaft Oct 05 '16 at 13:23
  • ... and let them figure out if any work needs to be done to accommodate a new field. Rockstar Roger might be able to deal with a new field with no changes in his client because he did defensive coding, Spaghetti Code Jim though might have to do a large rewrite. Their client is NOT your responsibility. You may be nice and offer legacy endpoints, but eventually clients need to stay up to date on your service. – maple_shaft Oct 05 '16 at 13:27
  • 1
    Yeah, thats indeed how microservices should work. Yet the problem still remains that i need to determine those dependencies and estimate the impact. Simply because thats a requirements. Anyway, thank you for the response and I'll talk about it with my client. – CGeense Oct 05 '16 at 13:47
  • 1
    @CGeense I know how the intersection of the ideal and the business reality can be tricky sometimes. If it was easy they wouldn't pay us to do it. Good luck. – maple_shaft Oct 05 '16 at 13:58
  • I'm afraid that you'll need to actually look at the code of all consumers, and determine how they unpack the messages. You could just run each of them, in a respective test environment, against a stream of new-format messages, and see if anything breaks. Likely you even have unit tests, or integration tests, that do a very similar thing. (If you don't, it's high time to write such a suite.) – 9000 Jan 03 '17 at 16:28