Elastic Search - Aircraft Indices

Original trac page

Naming Conventions

The indices in Elasticsearch are named according to the following convention:

[dataset_name]-[year]-[month]-[day], where the date is the date of processing.

For example, If one were indexing ARSF data from May 27th 2015, the formal index name would be "arsf-2015-05-27".

These index names can then be aliased to the dataset name using the "elasticsearch-head" tool (which is available here ) or as a chrome addon.

For example, one could create an alias for the "arsf-2015-05-27" at "arsf".

This is to allow multiple versions of the same index to exist on the same Elasticsearch cluster.

Haystack indices are created using the pattern:

cedamoles-haystack-prod_[date formatted using %d%m%Y_%H%M%S_%f]<br>
	

Overview of Production Indices

CEDA currently maintains several production indexes. The true name of these indices, created as above, is likely to change as indices evolve and is less important than the alias which is coded into services built on the indices. The names listed below are aliases.

ceda-eo 

Geospatial and temporal information about satellite observations from the Sentinel and Landsat missions.

arsf

Geospatial, temporal, and parameter metadata scanned from files in the CEDA archive under the "/neodc/arsf" directory.

eufar

Geospatial, temporal, and parameter metadata scanned from files in the CEDA archive under the "/badc/eufar" directory.

faam

Geospatial, temporal, and parameter metadata scanned from files in the CEDA archive under the "/badc/faam" directory.

ceda-level-1

Basic file level metadata for all files in the Archive.

ceda-level-2

File and parameter metadata scanned from files in the CEDA archive.

ceda-level-3

Geospatial, temporal, parameter and file level metadata for as many files as possible in the CEDA archive.

cedamoles-haystack-prod

Index created hourly from the MOLES database and is used in the MOLES catalogue search.

Querying Elasticsearch for the indices

We can ask Elasticsearch about the indices:

http://jasmin-es1.ceda.ac.uk:9200/_cat/indices

It reports something like:

green open um-2015-05-15    5 1 797435 0 103.8mb  51.9mb 
green open arsf-2015-05-27  5 1  49565 0 269.4mb 134.7mb 
green open faam-2015-04-27  5 1   2838 0  78.6mb  39.3mb 
green open aerial-photo     5 1  10899 0    10mb     5mb 
green open eufar-2015-04-27 5 1  13491 0  42.5mb  21.2mb
	
Did this answer your question? Thanks for the feedback There was a problem submitting your feedback. Please try again later.