MIDAS Meteorological Data: FAQ
The Met Office MIDAS dataset collection is one of the most popular dataset collections in CEDA's archives and often causes a number of questions for users.
The following Frequently Asked Questions (FAQs) have been put together to hopefully answer some common issues:
- When are station data updates available?
- Am I using the right station ID?
- Why doesn't this station have X type of data?
- My station says it was operational for my time period, but the data aren't available!
- Looks like my station should report the data I want, but I can't find them!
- Why does a site only report daily/monthly rain values and not both?
- Can I get a complete station listing?
1. When are station data updates available?
MIDAS data are collected by the Met Office as part of their ongoing long term database. Some of the data messages they receive in the MIDAS system may arrive sometime after they were originally observed due to the mechanism the information may be submitted and/or digitised. Such delays may mean that data may be a month or two behind, depending on the practices being followed.
In addition, CEDA only obtains a copy of the data from the MIDAS system once a month and usually covering the previous 6 months to pick up older data. Thus, in some cases there may be upto 2 months delay before the bulk of missing data are within the MIDAS collection at CEDA .
2. Am I using the right station ID?
A meteorological station may operate a suite of instruments each with their own instrument identifiers and even reporting within different networks. As such a site will typically have more than one identifier associated with it.
For example, Camborne has a MIDAS src_id of 1395 which helps to "join" all the data from this station together in the MIDAS collection. However, for it also has a range of station codes for the wide range of message types that it issues, as shown in this sample below:
|Station code||Message type||Message start date||Message end date|
3. Why doesn't this station have X type of data?
Different station covered by the MIDAS collection have differing operational uses - some are part of networks of rain gauges used by water companies to monitor rainfall levels, others are at airfields whilst others are part of meteorological agencies' monitoring networks. As such, the types of data that they collect and, therefore, report via "messages" they issue will vary. In the Camborne example above many message types can be seen, from "UATMP" which indicates that the site issues TEMP messages as part of an upper air network to "MODLERAD" which indicates that it also collects and reports irradiance data at the surface. As such attention to the different message types is also needed to ensure that the station is likely to be reporting the required data that you want. (e.g. if it only reports "WADRAIN" - i.e. daily rain - then don't expect to find wind data from the site).
4. My station says it is operational for my time period, but the data aren't available!
Though a station's start and end date may be a good indicator when data may be available, a station's reporting characteristics will often vary over time, so it's important that you also check the individual start and end dates for the message types related to the data that you are after for the station. For example, in the Camborne example above you can see that SREW, SYNOP messages are still ongoing, but CLM71 and HWND6910 messages have ceased. In other cases the site itself may have been operational for some time, but there may not be data available from that site until a later time.
5. Looks like my station should report the data I want, but I can't find them!
Sometimes the station information would suggest that you should be able to get the data that you want from the station, but it still doesn't appear to be available within MIDAS. This could be due to a variety of reasons such as:
- it's fairly recent data and they haven't been added to the MIDAS collection yet
- it's recent data that have arrived at the Met Office in a paper form, but haven't been digitised yet
- it's would have been old data, but no digitisation has happened
- the station information may not be correct - these are hand crafted by a team at the Met Office, based on available information and sometimes this doesn't reflect the actual MIDAS holdings
For all the above cases CEDA have very limited ability to get any missing data, except for recent data that is still in the process of being digitised (for these, CEDA still have to wait for that process to be complete and have no way of determining when those data will be available). Where we are aware of station information being incorrect, though, we do try and let the Met Office MIDAS team know about these so get this information checked and updated in due course.
6. Why does a site only report daily/monthly rain values and not both?
Within MIDAS all data from a given site can be found though search via its "src_id" - a unique identifier that the Met Office gives for the site itself. A site may, however, operate one or more raingauges - for example, it may operate a raingauge to record daily values or one to record monthly values, or one for both types of record. Each rain gauge will also have its own identifier (id, as opposed to src_id in the MIDAS data) and the Met Office stores the data all gauges for the site. The gauge IDs are listed on the station details page available through the MIDAS Station Search tool. Thus, by looking by "src_id" you will get data from all gauges on site, and specific rain gauge data can be further identified by its specific "id". However, whilst daily data may be recorded at a given site without an additional monthly rain gauge the Met Office does not derive monthly values from daily values reported from that site within MIDAS.
7. Can I get a complete station listing?
The MIDAS Station Search tool is a great way to find specific stations and access their metadata, but if you need access to station metadata from many sites (e.g. all the UK sites) to pull into a database, for example, then you can access the site station details a series of "metadata" files in the CEDA archive. You can access these files at: http://data.ceda.ac.uk/badc/ukmo-midas/metadata/. Here you will find a readme file detailing the contents of each sub-folder and the metadta files they contain. These provide details such as a station's name, location and river basin, as well as specific notes ("remarks") given for data from the sites and details on the geographic locations (e.g. county names). Note, these metadata files are updated periodically when CEDA obtains a new version of them from the Met Office, roughly monthly, so these files may change from time to time.