3

I'm working on a website that has a section that shows the company's stock details and investor-related data. This data is retrieved through scheduled SOAP API calls.

I created two scheduled tasks for achieving that:

  1. feth-data (every 10 minutes): retrieves data and stores it in cron_last_trade_price_graph which a graph which represents the 'last trade price' in relation to time throughout the day
  2. end-day (at the end of each day): cleans outdated (last two days) data records of cron_last_trade_price_graph and retrieves data to restore it in cron_historical_data that data will be used to populate Historical Data table and graph that show the 'Close Price' for the stock throughout time

Now, this is the first time I had to use schedules tasks, and since I'm working with important data I need to make sure that my implementation is solid and avoid any problems.

So generally my question is: What are the considerations (the dos and don'ts) when working with an application that depends on scheduled tasks?

For example:

  1. If the server fails to perform a scheduled task (say end-day). What is the best approach to re-try to perform that task?
    Possible Solution: Making a placeholder row for the next day, whose data needs to be retrieved and stored, that has a column is_historical_data_row_successfully_added that has a default value of 'NO' and then create another scheduled task (fix-historical-data) to check whether that placeholder row has been successfully populated and populate it if not.
    Concern: What if the fix-historical-data also failed? Then would it be best to schedule fix-historical-data for e.g. 3 times throughout the day? I feel like that's a pretty dirty solution.
  2. Should the application be bootstrapped for every scheduled task?: the application uses Wordpress as its CMS. Should I bootstrap Wordpress for fetch-data while it may barely need the functionality of Wordpress? Should I bootstrap Wordpress when the task depends heavily on it and not bootstrap Wordpress when the task doesn't? What's the best practice here?
  3. Code Structure: Should the scheduled tasks have an independent directory? Or should its file and dependencies be mingled with other files of the application?

Those are the concerns that I can think of at the moment. Are there any other considerations that I need to be aware of?

7asobate-
  • 39
  • 3

0 Answers0