Depositing Data at CEDA: FTP and RSYNC

There are 3 ways you can send data to CEDA (HTTP, FTP and RSYNC). This help document will take you through steps to upload your data via FTP and RSYNC recommend for large amounts of data. To upload data via HTTP please refer to Depositing Data at CEDA: A step by step guide

FTP

Once you have collected/produced your atmospheric or earth observation data; processed, calibrated, validated and quality checked it and prepared it for archiving using correctly applied standard data formats (for example, NetCDF should adhere to the CF conventions and have good quality metadata including global attributes) and meaningful filenames, you are now ready to upload it to the CEDA archive, using the following steps. A video tutorial can be found here.

1. Register for a  CEDA account.

2. Go to arrivals.ceda.ac.uk and login with your CEDA username and password.

3. Click begin. You need to agree to the deposit agreement the first time you use the service.  

4. Click browse deliveries.

5. Choose an existing delivery directory or create a new delivery directory by clicking on 'new directory'. (Add a suitable title e.g. the project, instrument name, model name or what the data contains, etc).

7. Click on 'other upload methods' and take note of the URL string

8. Download FileZilla https://filezilla-project.org

9. Paste the URL connection string, e.g. ftp://username:password@arrivals.ceda.ac.uk/mydata  into the "Host" field and hit the "Quickconnect" button (the other fields will be filled in automatically, or you can fill them in yourself)

10.Browse and upload files using the explorer at the bottom of the screen

Once you have collected/produced your atmospheric or earth observation data; processed, calibrated, validated and quality checked it and prepared it for archiving using correctly applied standard data formats (for example, NetCDF should adhere to the CF conventions and have good quality metadata including global attributes) and meaningful filenames, you are now ready to upload it to the CEDA archive, using the following steps. A video tutorial can be found here.

1. Register for a  CEDA account.

2. Go to arrivals.ceda.ac.uk and login with your CEDA username and password.

3. Click begin. You need to agree to the deposit agreement the first time you use the service.  

4. Click browse deliveries.

5. Choose an existing delivery directory or create a new delivery directory by clicking on 'new directory'. (Add a suitable title e.g. the project, instrument name, model name or what the data contains, etc).

7. Click on 'other upload methods' and take note of the command to edit

8. You need to have the  rsync command installed (which comes by default on Linux/UNIX systems, Windows users will have to install it themselves)

9.Edit the command given on the Rsync instructions of the upload page e.g. rsync -av /path/to/source username@arrivals.ceda.ac.uk::username/mydata (change "/path/to/source" to the location of your data files, username to your CEDA username and mydata to the folder to upload data to)

10. Run the command, enter your Rsync password when prompted and your data will be synced with the upload directory

11. Create and upload a metadata.yaml text file along with the data.  The metadata.yaml file needs to contain the details of the dataset, project and instrument or model. Adding the information this way helps keep the record and the data together. To create the metadata file use this utility or edit one of the examples below.