Synda : Synchronise Data

Synda

Synda stands for Synchronise Data it is a command line utility that can manage the downloading of data through the Earth System Grid Federation (ESGF). Synda is composed of two components a transfer module (sdt) and a post-processing module (sdp). The process for installing on JASMIN the transfer module is outlined here. Full documentation on usage can be found in the main Synda documentation

The Synda installation instructions can be found here. However, some useful additional installation instructions for the transfer module are provided below. 

Before downloading

  1. Make a directory where you want the software installed 
  2. $ mkdir /synda
    	

    You may wish to have multiple versions (3.9 is the current latest version) if so, make an "sdt" directory (the default) and symlink this to your version, from your Synda directory:

    $ mkdir sdt_3.9
    $ ln -s sdt_3.9 sdt
  3. Create a `setup_paths.sh` file in the synda directory with the following content (uses a general version of ) :  
  4. # Custom installation of Python
    # define a deps dir that contains a custom build of Python path below is a current working version 
    deps_dir=/group_workspaces/jasmin3/esgf_repl/synda/deps/
    python_dir=$deps_dir/deps/Python
    
    # Specify the top_dir as a location of synda once built
    top_dir=/group_workspaces/jasmin2/cp4cds1/synda
    export LD_LIBRARY_PATH=$python_dir/lib:$LD_LIBRARY_PATH
    export PATH=$python_dir/bin:$PATH
    export ST_HOME=$top_dir/sdt
    export PATH=$ST_HOME/bin:$PATH
    
    # If using the Synda post processing module
    #export SP_HOME=$top_dir/sdp
    #export PATH=$SP_HOME/bin:$PATH
    
    unset deps_dir
    unset top_dir
    unset python_dir
    	
  5. Before running  synda  this file must be executed every time
$ chmod u+x ./setup_paths.sh 
$ source setup_paths.sh

Download 

Now following the online Synda instructions:
  1. wget the install script, and make it executable. 
  2. run the install script:
$ ./install.sh -d <version-number> -t <directory path to where sdt is>

Configurations

(1) In the sdt configurations file 
sdt/conf/sdt.conf:
Set:
  • Default directory path for selection files: selection_path=<> default=synda/sdt/selection
  • Default directory path for default selection files: default_path=<> default=synda/sdt/selection
  • Default data directory path: data_path=<> default=synda/sdt/data; likely want to change this
  • Default database directory: db_path=<> default=synda/sdt/db/
  • Change default index to be CEDA by commenting out DKRZ and uncommenting CEDA
(2) In the sdt credentials file
sdt/conf/credentials.conf
  • Supply a valid openid and username
Did this answer your question? Thanks for the feedback There was a problem submitting your feedback. Please try again later.