00 TEMPLATE - more detailed ingestion workflows
NOTE - DO NOT PUBLISH THESE ARTICLES
just click on the tick to save them
What are the data
Collection Title/description: | |
MOLES collection page: | |
DMP link: | |
Who is involved
CEDA officer | Graham Parton |
Data provider contact | See DMP links |
Ingestion process
Data arrivals route
Data are prepared by ### staff which they push to their arrivals areas into various streams. Arrivals areas are:
- .
- .
Arrivals stream name(s):
- cfarr-radar : for all radar related products
Data pre-ingestion handling
/home/badc/software/datasets//_fileProcessor.cfg
Data Ingestion
Ingestion processes have to map between the filename components and directory locations. To do this the ingestion is done via calls to the cfarr_ingester.py
script. This script has a class that inherits from the standard Ingester class. cfarr_ingester.py
is then called for each ingest stream as defined in the /home/badc/software/datasets/chilbolton/chilbolton_fromDeliveries.cfg
config file
Runs on crontab.
Post Ingestion work
None
QC checks:
Files are only checked by CFARR staff. No other checking done at this stage. Format of files was confirmed to be CF-compliant ahead of setting up automatic processes.
Common problems:
Standard issues around ingest only.
Any other useful information need to know:
None.