Job Scheduling Software

Question

I am designing a Job scheduling software. Users of the software will define the job definition in XML like format https://github.com/etingof/apacheconfig/:

e.g.

<job>
    job_name myjob
    <command>
        command_name command1
        command_exec_string  'echo from command1
    </command>
    <command>
        command_name command2
        command_exec_string  'echo from command2'
    </command>
    <crontab> 5 4 */2 * * </crontab>
</job>

The above definition mean that run this job "at 04:05 on every 2nd day-of-month". After writing such job definition files users will submit this file to our install job service that will take care of scheduling the job at 4:05 AM everyday using unix cron.

Unix crontab entry will look something like below:

5 4 */2 * * /job_executor.py job-definition.xml

Unix cron daemon will kick launch the job_executor.py at 04:05 AM everyday.

I am using layered architecture for architecting this software. This means job_executor will go through multiple layers (script, business, persistence etc.).

Business layer again have multiple pluggable components. For example:

Check for dependencies before executing a command in a job - mark state of command as "failed_on_dependency" and state of job as "failed_on_dependency" if the dependency is not satisfied.
If command execution fails, then check if the configurations related to rescheduling a job are specified in job_definition, if yes then sleep till reschedule time interval - mark state of a job as "rescheduled" and state of command as "failed" etc.
And many more..

Each of these pluggable components can either update state of the job only, or can update state of command only, or can update state of command as well as job.

My question is should I keep logic to update these states somewhere centrally, or should this be taken care by individual pluggable business layer components?

I am even considering to have a immutable global Job object and have actions defined to update the Job object (something like redux architecture), though I dont need any publish subscribe type of logic here.

Is there a reason for writing this yourself instead of using a pre-existing solution? for example - https://stackoverflow.com/questions/1727138/an-enterprise-scheduler-for-python-like-quartz — Ben Cottrell, Feb 10 '19 at 15:33
These solutions are very limited on functionalities. I want rescheduling, dependency management, notifications and many more features. — , Feb 10 '19 at 15:48
Your sample XML appears to be a mix of XML and JSON. So which format are you trying to support? — Peter M, Feb 10 '19 at 19:08
@PeterM this is the format that I am using https://github.com/etingof/apacheconfig/ — , Feb 11 '19 at 05:07

Job Scheduling Software

0 Answers0