00 TEMPLATE - more detailed ingestion workflows
NOTE - DO NOT PUBLISH THESE ARTICLES
just click on the tick to save them
What are the data
|MOLES collection page:|
Who is involved
|CEDA officer||Graham Parton|
|Data provider contact||See DMP links|
Data arrivals route
Data are prepared by ### staff which they push to their arrivals areas into various streams. Arrivals areas are:
Arrivals stream name(s):
- cfarr-radar : for all radar related products
Data pre-ingestion handling
Ingestion processes have to map between the filename components and directory locations. To do this the ingestion is done via calls to the
cfarr_ingester.py script. This script has a class that inherits from the standard Ingester class.
cfarr_ingester.py is then called for each ingest stream as defined in the
/home/badc/software/datasets/chilbolton/chilbolton_fromDeliveries.cfg config file
Runs on crontab.
Post Ingestion work
Files are only checked by CFARR staff. No other checking done at this stage. Format of files was confirmed to be CF-compliant ahead of setting up automatic processes.
Standard issues around ingest only.
Any other useful information need to know: