Transfers to/from ARCHER/RDF

This article explains how to transfer data between JASMIN and ARCHER/RDF. It covers:

  • The choice of available tools / routes
  • How to request that the RDF transfer server "trusts" a credential issued by CEDA's short-term credential server

Choice of available Tools/Routes

See  JASMIN external connections and Data Transfer Tools  for general details.

It is important to choose the most appropriate route, method and tools to ensure you get the most efficient and reliable transfer experience. This can vary depending on system and network conditions, so it is worth experimenting with the different options available.

If you want to try  all the options available, you will need:

  • high-performance data transfer access on JASMIN
  • a login account at ARCHER/RDF, with access to the Data Transfer Nodes (dtn)
  • to have registered the subject of your CEDA-issued short-term credential with ARCHER support.

Check the examples in the linked documentation articles and ensure that you use them between the hosts used in the examples. Not all services connect over all routes to/from all hosts!

Table 1, below, shows recommended combinations of hosts & tools for transfers between RDF and JASMIN.

scp/rsync/sftp
Simple transfer using easy method but poor use of bandwidth
source dest notes
jasmin-xfer1 dtn02 over 1G light path
jasmin-xfer2 dtn0[12] over 10G JANET
bbcp
More sophisticated transfer tool, better use of bandwidth
jasmin-xfer1 dtn02 over 1G light path
jasmin-xfer2 dtn0[12] over 10G JANET
GridFTP over SSH   GridFTP performance with convenience of SSH, but only simple features of GridFTP
jasmin-xfer1 dtn02 over 1G light path
jasmin-xfer2 dtn0[12] over 10G JANET
GridFTP using certificate auth
Fully-featured GridFTP. Best performance.
jasmin-xfer1 dtn02 over 1G lightpath, client at JASMIN
jasmin-xfer2 dtn0[12] over JANET 10G, client at JASMIN
data-xfer1 dtn0[12] over JANET 10G, client at RDF
GridFTP using Globus Online
Managed GridFTP transfer using  web GUICLI or API. Best performance, reliable transfers.
data-xfer1 Globus endpoint
dtn03 Globus endpoint
over 10G JANET

Table 1: comparison of methods and routes for transferring data between RDF and JASMIN.

How to request that the RDF transfer node(s) trust a credential issued by CEDA's short-term credential server

  • Go to https://www.archer.ac.uk/safe
  • Go to "login accounts" and click your username
  • Click "Add certificate"
  • Enter the Distinguishing Name (DN) of your CEDA-issued short-term credential. You can obtain this by using the following commands:
$ myproxy-logon -s slcs1.ceda.ac.uk -l USERNAME -o credfile

The first time you do this, or in the case where host certificates have been updated, add the -b option to ensure that the relevant certificate files are copied to your local certificate store.

$ myproxy-logon -s slcs1.ceda.ac.uk -l USERNAME -o credfile -b

In this case, the output, i.e. the short-term credential, will be stored in credfile. Inspect this file and extract the subject:

$ openssl x509 -noout -in credfile -subject
subject= /DC=uk/DC=ac/DC=ceda/O=STFC RAL/CN=https://ceda.ac.uk/openid/Firstname.Lastname
  • Copy and paste the text beginning /DC= into the appropriate place in the form presented by the SAFE interface and submit the request.
  • Once you hear back from support@archer.ac.uk that your request has been completed, your credential should be trusted and you should be authorized to use the transfer server in this way. There is a chance that by this time, your short-term credential will have expired (!). If so, repeat the myproxy-logon step again, but when re-issued it will have the same subject and should be ready to use. Note also that there is sometimes a delay between the time when you receive the notification message from Archer support and the next time that their cron job runs to roll out the change to their servers. If in doubt, wait a few hours and try again.

Still need help? Contact Us Contact Us