Synda : Synchronise Data
Synda stands for Synchronise Data it is a command line utility that can manage the downloading of data through the Earth System Grid Federation (ESGF). Synda is composed of two components a transfer module (sdt) and a post-processing module (sdp). The process for installing on JASMIN the transfer module is outlined here. Full documentation on usage can be found in the main Synda documentation.
The Synda installation instructions can be found here. However, some useful additional installation instructions for the transfer module are provided below.
- Make a directory where you want the software installed
- Create a `setup_paths.sh` file in the synda directory with the following content (uses a general version of ) :
- Before running
syndathis file must be executed every time
$ mkdir /synda
You may wish to have multiple versions (3.9 is the current latest version) if so, make an "sdt" directory (the default) and symlink this to your version, from your Synda directory:
$ mkdir sdt_3.9 $ ln -s sdt_3.9 sdt
# Custom installation of Python # define a deps dir that contains a custom build of Python path below is a current working version deps_dir=/group_workspaces/jasmin3/esgf_repl/synda/deps/ python_dir=$deps_dir/deps/Python # Specify the top_dir as a location of synda once built top_dir=/group_workspaces/jasmin2/cp4cds1/synda export LD_LIBRARY_PATH=$python_dir/lib:$LD_LIBRARY_PATH export PATH=$python_dir/bin:$PATH export ST_HOME=$top_dir/sdt export PATH=$ST_HOME/bin:$PATH # If using the Synda post processing module #export SP_HOME=$top_dir/sdp #export PATH=$SP_HOME/bin:$PATH unset deps_dir unset top_dir unset python_dir
$ chmod u+x ./setup_paths.sh $ source setup_paths.sh
- wget the install script, and make it executable.
- run the install script:
$ ./install.sh -d <version-number> -t <directory path to where sdt is>
(1) In the sdt configurations file
- Default directory path for selection files: selection_path=<> default=synda/sdt/selection
- Default directory path for default selection files: default_path=<> default=synda/sdt/selection
- Default data directory path: data_path=<> default=synda/sdt/data; likely want to change this
- Default database directory: db_path=<> default=synda/sdt/db/
- Change default index to be CEDA by commenting out DKRZ and uncommenting CEDA
(2) In the sdt credentials file
- Supply a valid openid and username