BADC-CSV Format for Data Exchange
This page gives a brief introduction to the BADC-CSV file format, which is covered in greater depth in the " BADC-CSV Text File Guide for Users and Produces" .
Note - for more details on how to read more generic comma- or -tab- (csv or tsv) separated data please see our ASCII data format page.
What is the BADC-CSV format?
A BADC-CSV formatted file is simply a csv file with a fancy file header full of information about the data.
It is used for simple 1-D data, such as:
- A collection of surface meteorological data from one or more stations over a period of time
- A time series of data from one instrument
- Simple model output
How do I format my data?
The BADC-CSV file is broken down into two sections; header and data. The header includes metadata, information about the data themselves and supplementary information that helps users to understand the data and how the data were produced, plus other useful information. The header has various components, some components are compulsory, while others are encouraged to make the data more understandable/usable.
The header includes various pieces of information about the data, known as metadata. Each metadata item contains a reference to the data column it applies to. A “G” indicates that the metadata applies globally - i.e. to all the data, while a “number” or “string” represents a particular column of data to which it applies. The links to specific columns could be, for example:
- numbers: 1, 2... etc
- letters: a, b, c, etc
- labels: temp, wind speed, etc.
These will correspond to the column headings at the start of the data section (see below for more information).
An example of a header section is shown below.
Below is a table of commonly used metadata labels that can be used in your data file. Note that the ‘Conventions’, ‘long_name’ and ‘coordinate_variable’ labels are compulsory. For a full list please see " BADC-CSV Text File Guide for Users and Produces".
|Conventions||G, BADC-CSV,version number||This is the "File Type Identifier" - a compulsory line for all data records. It indicates to any user the file format, as well as following the Climate and Forecasting Metadata Conventions (see here for more information about CF standards).|
|title||G||Title of the dataset|
|creator||G||Author name (<lastname>, <firstname> <middle initials>)|
|activity||G||Name of the activity sponsoring the collection of the data|
|coordinate_variable||G||Flag for coordinate variables, optionally a plotting axis suggestion and the name of the coord ref system.|
|long_name||Column number or name||Text description of the variable and unit|
|standard_name||Column number or name||Name from CF standard name list (name,unit, standard)|
|type||Column number or name||The value interpretation for the variable –( char (default), int, float)|
|flag_value||G, column number or name||Values used for flag table in data|
|flag_meanings||G, column number or name||Meanings of the flag_values in data|
|comments||G, column number or name||Any other useful information|
|last_revised_date||G||Date the data, file or metadata were last changed.|
|history||G||Free text description of the file history|
|source||G||Name of the instrument or model|
|observation_station||G||Name and address of the station|
|height||G, column number or name||Height(s) valid for data, plus unit. e.g. 36,m|
|location||G|| Coordinates of the station. Can be displayed with these number of values 1 - name of location 2 – (lat,long) 4 –bounding box given as N,W,S,E decimal degrees with respect to -180 to + 180 longitude, 90 to -90 latitude. Examples:
|date_valid||G||Date valid for the data (start,end)|
|rights||G||It is a statement about the ownership of the data and that they 'are made available under < a specific licence>|
All words within the label should be joined by underscores ("_"), should not have any white space and should be entirely lower case. Data providers may add their own labels, but additional labels will not be automatically interpreted by software designed to understand BADC-CSV formatted files.
The data section comes after the header and includes 3 markers to indicate the start and end of the data.
|data||The word (marked in red or box 1) is placed in the row immediately following the header section. This indicates the start of the data block of the report and must only be in lowercase.|
|column number or name||The following row contains the column numbers or names used in the header section above (marked in green or box 2). The data lines then follow this line.|
|end_data||The end of the data block and the record itself is then given by a single line with these words in lowercase (marked in orange or box 3)|
Simple Example: This simple example shows a basic BADC-CSV file to demonstrate the layout and also a simple sample of data. This has all the basic required elements in it to be acceptable.
Full Example 1:This is a sample taken from the UK Met Office MetDB dataset showing the AMDARS message type and also includes some additional metadata elements beyond the standard.
How do I get started?
A basic BADC-CSV template can be found here.
Uploading BADC-CSV files to CEDA
For programmes currently submitting BADC-CSV formatted files, CEDA provides a web-based file uploader BADC-CSV file checker service. In the process, files are checked for compliance with the BADC-CSV standard. Once your data have passed you can upload your data following the guide.
The BADC-CSV format has been structured to comply with a number of data standards. More information about these standards can be found through " BADC-CSV Text File Guide for Users and Producers".
For more information and guidance on formatting your data please visit our FAQs page.