CEDA OPeNDAP: scripted interactions
As the CEDA Archive has various data access controls in place users wishing to interact with the CEDA OPeNDAP service will need to do so using security certificates. The following instructions will guide you through how to:
- Getting started
- Get a security certificate
- Download data from the CEDA Archive via Curl
- Connecting to a file in a Python script using the OpenDAP service
Further how-to guides will follow in due course covering the steps needed for Windows users and additional functions such as parameter sub-setting and including these calls within common scripting languages such as Python, IDL and MatLab.
- First create a directory where you will download the scripts to generate the security certificates you'll be using later
- Switch to that directory
- Next you'll need to download the code from the following Git Hub repository:
git clone https://github.com/cedadev/online_ca_client
- Then navigate to the scripts directory:
- Finally, run the following command to get a folder full of trustroots - these will be needed later when we generate your temporary certificate:
./onlineca-get-trustroots-wget.sh -U https://slcs.ceda.ac.uk/onlineca/trustroots/ -c ~/trustroots -b
Getting a Security Certificate.
Having obtained the necessary code from the git hub repository you now need a certificate associated with your myCEDA account in order to access restricted datasets
- Run the following command to get your temporary (3 day) certificate (remember to replace your ceda username below!:
./onlineca-get-cert-wget.sh -U https://slcs.ceda.ac.uk/onlineca/certificate/ -c ~/trustroots -l <ceda username> -o $PWD/creds.pem
- NOTE - You'll need to input your password after you submit the command
- This will pipe the output (your certificate) into a file of your choosing (above it will be a local file:
$PWD/creds.pem), you can alter this if you wish e.g.
Downloading a file with cURL
Finally, you can use your new certificate to download data using any tool that supports this feature. It should be noted that, due to an issue with our service, you may have trouble downloading data using Wget. For this reason, we are using cURL in our example.
- Use your certificate as a parameter to cURL to retrieve something from PyDAP:
curl --cert $PWD/creds.pem -L -c /dev/null curl --cert $PWD/creds.pem -L -c /dev/null http://data.ceda.ac.uk/badc/ukmo-midas-open/data/uk-daily-temperature-obs/dataset-version-201901/aberdeenshire/00145_cairnwell/qc-version-1/midas-open_uk-daily-temperature-obs_dv-201901_aberdeenshire_00145_cairnwell_qcv-1_1994.csv
The -L option is required for following links and the -c option tells cURL to store cookies. On Windows, you will need to use NULL rather than /dev/null for the -c option input.
Note that older versions of cURL may require an absolute path to your certificate file, as above.
Connecting to a file in a Python script using the OpenDAP service
Please see the example help page on Reading a NetCDF file from a Python Script using OpenDAP.