3

I work in a team of people with experience in varied programming languages. Recently we were required to build a report when an order is placed in our system. Our ecommerce store runs on PHP and the program to send the report was written in Java. The major challenge we faced was setting up a way so that the Java process is triggered as soon as the order placed. In the same vein, we have plenty of programs written in Python, Scala etc that need to be coordinated and be aware of when to do a task X when task Y happens.

As of now, our go to solutions to this problem is to setup a cron to routinely check whether a file or value in DB exists and THEN run (simple polling).

I believe a task queue is a right way to solve this problem. However, I'm not sure how to setup one, and how will the Java process know that a task is waiting for it? Also, would a lot of change be required in my PHP program to put something in task queue?

Thanks a lot!

metacubed
  • 966
  • 5
  • 14
Prakhar
  • 133
  • 4
  • 2
    Read about [event loops](http://en.wikipedia.org/wiki/Event_loop), [message passing](http://en.wikipedia.org/wiki/Message_passing), [poll(2)](http://man7.org/linux/man-pages/man2/poll.2.html), [RPC](http://en.wikipedia.org/wiki/Remote_procedure_call), [JSON-RPC](http://json-rpc.org/) – Basile Starynkevitch Jul 16 '14 at 09:33

1 Answers1

4

We faced a similar problem, and used RabbitMQ for message transport, and formatted the messages using Google Protocol Buffers.

RabbitMQ is excellent, it just works.

Protocol buffers format the data compactly (far more so than XML or JSon) and is a known schema allowing interoperability between different languages and machine architectures.

On the Java side, the RabbitMQ Client Library will fire an event when a message is ready to be consumed. Read the RabbitMQ docs and follow through some examples, it'll become clear enough fairly quickly.

On the PHP side, there are RabbitMQ libs you can hook into, you authienticate against a Rabbit host and publish a message. I personally haven't used RabbitMQ with PHP, but given how well it's abstracted for other languages it shouldn't be a huge deal.

Binary Worrier
  • 3,112
  • 2
  • 25
  • 23
  • Thanks for protobuf. Regarding RabbitMQ, how does my Java program capture the event when the client library fires it? Does it mean that I need keep running my java program all the time? – Prakhar Jul 16 '14 at 10:36
  • @Prakhar: Essentially yes, the Java program needs to be running on the machine (we deploy many of our solutions as Windows Services). However this isn't as bad as it sounds, the app uses precious little resources while it's waiting for events to fire. – Binary Worrier Jul 16 '14 at 10:40
  • Ah ok. But that makes sense, when I'm running a service. However, my java program is just a one of program that I want to run. Any ideas? – Prakhar Jul 16 '14 at 10:44
  • I'm not sure I know what you mean by "my java program is just a one of (many?) program that I want to run". If - depending on message type - you want to run different programs, you can have a single service that listens for different messages, and then dispatches that message to an individual "java program". However - if possible - I'd refactor your many Java programs to sit within the service. – Binary Worrier Jul 16 '14 at 10:52
  • I was going to recommend json + SQS, but that's essentially the same pattern - a language-agnostic defined data format + asynchronous queues – Daenyth Jul 16 '14 at 18:01
  • JSon would be simpler to get up and running, and you can eyeball the messages in the rabbit queues too. Protocol Buffers might be overkill for a first step. – Binary Worrier Jul 16 '14 at 18:20