2

My team is currently developing an application using a microservice architecture with publish/subscribe interservice communication over a message bus. Certain types of HTTP requests and messages received as inputs to a service might cause that service to subsequently publish certain types of messages as a result in addition to the HTTP response if there was an HTTP request, and these published messages might trigger publication events from other services as well. I'd like to be able to programmatically generate a chart of all of the possible message flows without having to run any of these services.

Our backend services are currently developed in Kotlin and Ruby. In Kotlin, our messages are statically typed, and I believe it should be possible to programatically determine message flow inputs and outputs from each Kotlin service at compile time or through static code analysis.

How should I start to tackle this problem? I'm focusing on the Kotlin services first, as that's where I have the most experience. I'm trying to determine if the Arrow functional programming library can encode potential message output side effects (of specific message types) at the type level, but I'm not sure if that's possible. I'm also considering using static code analysis provided by an external program or programs rather than using the Kotlin compiler and type system, which seems more likely to bear fruit but also more involved.

Ashok Bhaskar
  • 131
  • 1
  • I am not sure if this is feasible. In an event driven system a component that reacts to an event knows nothing about a component that sends the event. And this is a benefit of this design as well. Hand written documentation might be your best shot. Static code analysis could work, although I doubt it would be easy, but why really bother ? – Cap Barracudas Dec 09 '19 at 18:55
  • "In an event driven system a component that reacts to an event knows nothing about a component that sends the event" - my line of approach is to try to programmatically chart HTTP and message inputs to message outputs for each service and then combine them into a larger diagram using some other software module that pulls in each service's message IO information. – Ashok Bhaskar Dec 09 '19 at 20:12
  • "why really bother ?" - to see if it's possible, and if the solution works well it has the potential to significantly reduce the resources spent on documentation and dynamic analysis. – Ashok Bhaskar Dec 09 '19 at 20:15
  • It should be possible but complex and error prone I presume. A piece of advice though. Dont let anyone tell you you cant do something. And if you succeed, I would be glad to use your tool ! – Cap Barracudas Dec 09 '19 at 22:18

1 Answers1

1

Reverse engineering charts from code is hard. I had way better experience with "forward engineering": generating code from models. Especially for data flow models, this seems pretty feasible:

  • you design your components with input and output ports using a graphical tool like Star.UML, Dia or Draw.IO which allows you to access the graphical content programmatically

  • then you build a generator to translate this to some boilerplate code for doing the publish/subscribing or sending the messages.

To give you an idea how this could work, I suggest you have a look into this Flow Design Cheat Sheet. You can skip the german texts - starting at page 6 ff, you see how elements of a flow diagram can be translated into C# code.

Doc Brown
  • 199,015
  • 33
  • 367
  • 565
  • Building a generator seems like a significantly harder problem to me than reverse engineering charts from code - the REST endpoints and message consumers live in different modules than the message producers do for composability and separation of concerns reasons, and this solution gets very complex unless you're tightly coupling inputs and outputs. Also, this is a very manual solution, and one of my goals is to automate out the need for diagramming. – Ashok Bhaskar Dec 10 '19 at 15:23
  • @AshokBhaskar: well, these are my experiences with reverse and forward engineering - I have seen and implemented several successful forward engineering tools, but reverse engineering only worked to some degree for me for very restricted use cases, where one could make some assumptions about the code structure and the task was not very complex. But YMMV. – Doc Brown Dec 10 '19 at 16:02