Elasticsearch at CEDA

Elasticsearch at CEDA: Making data more discoverable

CEDA maintains an Elasticsearch cluster to index information which enables us to improve the searchability of our data holdings. This is both for us to improve the services we provide and allows external users to query our data holdings and build services using the response.

What is available?

Name Description
ceda-fbi Main index containing file level metadata about all files in the CEDA archive
ceda-dirs Index of all the directories in the CEDA archive. Includes information about the CEDA catalogue and 00README if there is one.
ceda-eo Earth observation index containing metadata about satellite scenes from Sentinel 1,2,3, Sentinel ARD and Landsat 5,7 and 8.
faam Metadata from flights flown by the Facility for Airborne Atmospheric Research
eufar Metadata from flights flown by the European Facility for Airborne Research
arsf Metadata from flights flown by the Airborne Research and Survey Facility
bas-masin Metadata from flights flown by the British Antarctic Survey – Meteorological Airborne Science Instrumentation

You can access the search and mappings API to get information.

The Elasticsearch API is extensively and thoroughly documented at: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-your-data.html

401 Unauthorised

Access to our Elasticsearch cluster is restricted. Unauthenticated requests will only allow you to perform search and get mapping operations on the indices specified above. 

How to search the indexes?


There is a very useful python client that will enable you to interact easily with elasticsearch at CEDA.

It can be installed using:

pip install python-elasticsearch

A basic example script which will return some results from the ceda-fbi index:

from elasticsearch import Elasticsearch

query = {
	"query": {"match_all": {}}

es = Elasticsearch(["https://elasticsearch.ceda.ac.uk"])
es.search(index="ceda-fbi", body=query)

Access via command-line tools: wget and curl

Here is a link to the URI search documentation from elasticsearch which describes all the allowed keywords.


Some examples using the wget and curl command-line tools:

wget https://elasticsearch.ceda.ac.uk/ceda-fbi/_search?size=1
wget https://elasticsearch.ceda.ac.uk/ceda-fbi/_mapping
curl https://elasticsearch.ceda.ac.uk/ceda-fbi/_search?size=1
curl https://elasticsearch.ceda.ac.uk/ceda-fbi/_mapping

Curl with body

curl https://elasticsearch.ceda.ac.uk/ceda-fbi/_search -H 'Content-Type: application/json' -d '{"query": {"match_all": {}}}'
Did this answer your question? Thanks for the feedback There was a problem submitting your feedback. Please try again later.