Developer Cloud Sandboxes solution

h1. ESGF CMIP5 tutorial

This tutorial provides guidance on how to apply for and access ESGF CMIP5 data.

h2. General Concept

The ESGF data discovery is based on the "OpenSearch":http://www.opensearch.org Catalogue Access Service . When a query is sent to the CAS, a software component called ESGFGateway provides a dynamic discovery of the resources hosted by the Earth System Grid Federation, through the access point of the "ENES archive at DKRZ":http://esgf-data.dkrz.de, with the search extended to all the peer nodes (distrib=true). Then the metadata so collected are cataloged in a CAS form .

h2. Catalogue Data and Output Formats

Actually a sample ESGF-CAS offers the following metadata :

|_<. Series name |_<. Description |_<. Example URL (ATOM) |

| fgco2 | Surface Downward Mass flux of carbon dioxide expressed as carbon (RCP, CMIP5) | http://sb-10-16-10-26.dev2.terradue.int/catalogue/esgf/fgco2/atom |

| o2min | Oxygen Minimum Concentration (RCP, CMIP5) | http://sb-10-16-10-26.dev2.terradue.int/catalogue/esgf/o2min/atom |

| ph | Sea Water pH reported on total scale  (RCP, CMIP5) | http://sb-10-16-10-26.dev2.terradue.int/catalogue/esgf/ph/atom|

| phyc | Phytoplankton Carbon Concentration (RCP, CMIP5) | http://sb-10-16-10-26.dev2.terradue.int/catalogue/esgf/phyc/atom |

| thetao | Sea Water Potential Temperature  (RCP, CMIP5) | http://sb-10-16-10-26.dev2.terradue.int/catalogue/esgf/thetao/atom |

| zos | Sea Surface Height Above Geoid (RCP, CMIP5)  | http://sb-10-16-10-26.dev2.terradue.int/catalogue/esgf/zos/atom |

The series name comes from the choice to have one series for each variable and fixed project = CMIP5 .

The following output formats are available (the *zos* series is used as an example in the remainder of this document, but everything explained below applies to the other series too):

|_<. Format |_<. URL (using ASA_IM__0P as example) |_<. Comment |

| RDF | http://sb-10-16-10-26.dev2.terradue.int/catalogue/esgf/zos/rdf | The most universal format, contains the complete product metadata |

| ATOM | http://sb-10-16-10-26.dev2.terradue.int/catalogue/esgf/zos/atom | ATOM feed |

| HTML | http://sb-10-16-10-26.dev2.terradue.int/catalogue/esgf/zos/html | Web-optimised representation, useful for quick verifications with a web browser |

h2. Viewing Product Information

The best way to get familiar with the ESGF CAS is making requests using a web browser. You can do this by choosing any of the above URLs (e.g. HTML or ATOM format).

At this point, you may refine your search by specifying search criteria. This is done by appending query string parameters to the base URL. Doing so, only information of the matching products is returned (below there is an example).

These are the search criteria you can specify:

|_<. Query string parameter |_<. Description |_<. Value format |_<. Value set |_<. Example |

| @bbox@ | *Rectangular area of interest* | @minlon,minlat,maxlon,maxlat@ | _none_ | @bbox=13.3,50.5,14.3,51.5@ |

|/2. @start@ |/2. *Start date* of the period to be covered | @YYYY-MM-DD@, or |/2. _none_ | @start=2010-09-12@ |

| @YYYY-MM-DDThh:mm:ssZ@ | @start=2010-09-12T13:58:26Z@ |

|/2. @stop@ |/2. *End date* of the period to be covered | @YYYY-MM-DD@, or |/2. _none_ | @start=2010-09-15@ |

| date and time in format @YYYY-MM-DDThh:mm:ssZ@ | @start=2010-09-15T21:24:30Z@ |

| @count@ | *Number of records* to be returned (default is @20@) | Integer value | _none_ | @count=100@ |

| @startIndex@ | *Index of first result record* to be returned (starting with @0@, default is @0@) | Integer value | _none_ | @startIndex=20@ |

| @startPage@ | *Result page* (starting with @0@, default is @0@), each page contains @count@ records (alternative to @startIndex@) | Integer value | _none_ | @startPage=1@ |

| @ensemble@ | *Ensemble* as defined from ESGF Portal | Text | _none_ | @ensemble=r1i1p1@ |

| @experiment@ | *Experiment* name as defined from ESGF Portal | Text | _none_ | @experiment=decadal1960@ |

| @model@ | *Model* name as defined from ESGF Portal | Text | _none_ | @model=GEOS-5@ |

| @cmor_table@ | *MIP Table* name as defined from ESGF Portal | Text | _none_ | @cmor_table=Omon@ |

| @realm@ | *Realm* name as defined from ESGF Portal | Text | _none_ | @realm=ocean@ |

By default the number of returned products is 20. To obtain all matching products, you can do the following:

* Provide a value for the @count@ query string parameter, e.g. @count=100@

* Do sequential requests providing the index of the first product to be returned until the returned result is empty (beyond the total number of matching products), e.g. @startIndex=0@, @startIndex=20@, @startIndex=40@, etc (note that the first product has the index @0@).

h2. Querying

The [[esgf-client|ESGFClient]] is a Command Line Interface to download products from the "ESGF Data Search".

h2. Conclusion

With this lesson you have learned:

- to query ESGF CMIP5 dataset series

- to query and downlaod ESGF CMIP5 datasets