Python code template for scripts to be run in batch

Question

I will be writing a number of Python scripts which will be run in batch daily on a Windows server. I am trying to develop a template for writing the scripts. My intention is to use a configuration file to hold the settings (for example: the location the log files will be saved in).

Below is the template I have developed so far and was wondering - can it be improved in some way? Thank you.

import datetime
from pathlib import Path
import configparser, argparse, logging

def main():
    logging.info('Start Execution')
    #
    # Code goes here
    #
    logging.info('Execution Complete')

if __name__ == '__main__':

    parser = argparse.ArgumentParser()
    parser.add_argument(
        "-cf", "--config-file", type=str, required=True,
        help="Path and filename to the config file.",
        default=None
    )
    parser.add_argument(
        "-ll", "--loglevel", type=str, required=False,
        help="Defines the level of detail to be recorded in the log files; default is INFO.",
        default='INFO'
    )

    argvs = parser.parse_args()

    #Check a valid log level has been given
    numeric_level = getattr(logging, argvs.loglevel.upper(), None)
    if not isinstance(numeric_level, int):
        raise ValueError(f"Invalid log level: {argvs.loglevel}")
    
    # Check the config file exists
    if not Path(argvs.config_file).is_file():
        raise FileNotFoundError(f"Config File: {argvs.config_file}")

    # Open and read-in the config file
    # The Config File is open and read here as it makes the values available globally.
    config = configparser.ConfigParser()
    config.read(argvs.config_file)

    #Configure logging
    logging.basicConfig(
        filename=config['SETTINGS']['log_path']+datetime.datetime.now().strftime("LOG_%G-%m-%d")+'.log'
        ,format='%(asctime)s - %(message)s'
        ,level=numeric_level
    )
   
    main()

When you say 'template', do you mean you plan to copy this into each project that will run here? — JimmyJames, Jul 15 '21 at 17:42

score 1 · Accepted Answer · answered Jul 15 '21 at 20:19

There's nothing about your script that strikes me as especially wrong. You might find an issue here or there when you start using it. One potential improvement, however is to not copy this into a lot of projects but instead use one module to drive all the scripts. You can do this using the add_subparsers method of argparse. If you've used commandline git or docker, the style of CLI is like that: there's a main argument which determines the subcommand which can then have it's own help and arguments specific to it.

If you were going down that route (and I'm not sure you need to), I would make each script a package which has a method that can take a parent parser and add it's own arguments as well as a method which accepts and args object.

This is absolutely a viable approach that I have used in actively maintained application. If you want me to provide some example code, please ask. I'm not sure, however, if your situation would indicate this solution. If this top level script will need to be copied to a a significant number of projects e.g. more than 3, I would consider it (or a similar approach) from a long-term maintenance perspective.

Thank you! That is very helpful and encouraging. I will stick with this approach for the moment but I will also look into the `add_subparsers`. — Garrie Powers, Jul 21 '21 at 19:57

Python code template for scripts to be run in batch

1 Answers1