2

I have to design a system where I need to make some processing (which might take say 10 mins).

After some amount of time (say 30 mins), I need to come back and check the status of this processing and possibly stop any further processing.

There would be multiple such processes fired and running sequentially or in parallel. Each would have a different processing time, and each would have a different 'stop' time.

One way is that this would be a cronjob which comes back every 30 mins and checks the processing component.

My solution is currently designed on NodeJS and I intend it to be stateless. I am running a callback in a interval of 10 mins to check for the final state.

What I want to know is how are such systems designed? Is there a better alternative to having a cron job and / or continuously polling for the state of a processing. How would you do this?

If I am being too vague in the requirement, please let me know and I can add in more details.

Bart van Ingen Schenau
  • 71,712
  • 20
  • 110
  • 179
ksprashu
  • 381
  • 2
  • 5
  • 2
    Requests for resources are off-topic here, because they usually don't have the lasting value that we expect of questions and answers here. I have edited that portion out of the question. Also, the phrase "best practice" does not have good connotations around here. – Bart van Ingen Schenau Mar 28 '16 at 12:09

1 Answers1

2

Such designs call for a tool that handles job scheduling + monitoring as well as dependencies among jobs (such as : Trigger Job B only if Job A succeeds, if A fails then trigger job C, if A runs for more than 10 minutes, trigger job D etc.).

Autosys / Tivoli are probably most used products in enterprise, but would most probably be an overkill in this situation.

Autosys : https://en.wikipedia.org/wiki/CA_Workload_Automation_AE Tivoli : https://en.wikipedia.org/wiki/IBM_Tivoli_Workload_Scheduler

In this specific situation, options are:

  1. Use a tool like one mentioned above. This list has open source tools as well. https://en.wikipedia.org/wiki/List_of_job_scheduler_software
  2. Setup this mechanism with cron
  3. Write custom code in jods.js or shell scripts.
Shamit Verma
  • 759
  • 4
  • 9