Using Cron

Cron is a very common job scheduler for linux. It allows users to run the same command or shell script periodically. Typically it is used to automate tasks, for example, every Monday run my script to plot last week's data. There are many guides to using  cron and crontab (the command for loading the cron job table). 

Cron on JASMIN

Generally cron is disabled on the JASMIN general access machines such as jasmin-sci1.ceda.ac.uk. This is to avoid people killing the machine by setting up lots of processing jobs when better alternatives, e.g. Lotus, are available. However, there are times when it is appropriate to use cron and so a generic cron service machine is provided. cron.ceda.ac.uk is configured like jasmin-sci1.ceda.ac.uk, except cron is enabled. Anyone who can log into jasmin-sci1 should also be able to login to cron.ceda.ac.uk.

There are a few rules of the road to using this service:  

  1. Avoid process pile up: If a job has not finished before the cron starts the next instance of the same job then competition for resources probably means that job will also not finish. Eventually a mass of unfinished jobs will overwhelm the whole machine and it will crash. To avoid this jobs should test to see if the previous job is still running by using a lock file, or making sure the jobs timeout. The crontamer wrapper script, documented below, is available to help you implement this.
  2. Expect it to break occasionally: Regardless of any measure introduced by users to stop process pile up, you can expect it to go rouge at some point. We will reboot the machine when this happens. We may remove offending jobs from the cron table but persistent offenders may be barred from using the service.
  3. Don't do heavy processing on the cron machine itself. You can submit jobs to lotus (bsub is on cron.ceda.ac.uk) to offload the processing resource.

Common Cron Gotchas 

The bash shell environment when cron launches scripts is not identical to the one when working interactively. Annoyingly the path to common tools, like bsub, may not have been setup so that you get an error message from cron when it works perfectly well interactively. A way to get round this is to source the .bash_profile in the crontab file so that the interactive environment is used by cron. 

24 * * * * . $HOME/.bash_profile; bsub -W 12:0 mycmd.sh

Crontamer

Crontamer is a wrapper script to implement lock file and time out checking for cron jobs to avoid issues of ‘’’runaway’’’ jobs causing problems for everyone (e.g. where new jobs start before old ones finish, which can cause a snowball effect).

Download the crontamer script from  https://github.com/cedadev/crontamer.

Use ‘’cron’’ as normal, but include the ‘’crontamer.py’’ command before the users own script and arguments.
bash> crontab -l
#e.g. cron file entry to run job every day at 4am:
0 4 * * * /home/users/jblogs/bin/crontamer.py -t 2 /home/users/jblogs/bin/my_repeat_script.sh -opt1 arg1 arg2
Note: There are several crontamer.py reserved arguments should be avoided by the users code: ‘’’-t, -l, -u, -e, --email, -v’’’.
What it does:
The crontamer.py script can performs a number of checks before running your script.
  • Check for existing lock file to indicate if the script is already running (default). The script will exit if another instance of the script is already running (as the same user and with the same passed arguments).
  • Optionally unlock earlier instance (e.g. if earlier process exited before cleanup completed).
  • Check for (and kill) an existing instance has been running longer than specified timeout (default 12hr) to help catch errant processes.
  • Optionally email report to a specified address.
Lock files are created in the /tmp directory as:
/tmp/crontamer.{user}.{script}.{unique_id}
These files should be cleaned up automatically, but users may need occasional checks or find them helpful for identifying problems running their scripts. The {unique_id} is based on a combination of the username, passed script and arguments, enabling multiple calls of a script with different arguments to be handled separately.

Still need help? Contact Us Contact Us