Project

General

Profile

Esgf-client » History » Version 2

Herve Caumont, 2013-09-18 12:12

1 1 Herve Caumont
h1. ESGFClient
2
3 2 Herve Caumont
{{>toc}}
4
5 1 Herve Caumont
h2. Overview
6
7
ESGFClient is a Command Line Interface written in C# able to download products from the "ESGF Data Search" getting as input the location of a RDF file or a list of single product's urls.
8
9
h2. Installation
10
11
The user can install the ESGFClient following these easy steps:
12
13
<pre><code class="XML">
14
yum install esgf-tools
15
</code></pre>
16
17
and test the installation with
18
19
<pre><code class="XML">
20
ESGFClient --help
21
</code></pre>
22
23
h2. Usage
24
25
These are the options given:
26
<pre><code class="XML">
27
Options:
28
  -f, --file=VALUE           File RDF to parse
29
  -F, --Force                Force the whole rdf's resources
30
  -O, --OpenDAP              Download ONLY the OpenDAP resources
31
  -u, --url=VALUE            Url to download
32
  -o, --openid=VALUE         OpenId used to access.
33
  -p, --password=VALUE       Password OpenId used to access.
34
  -b, --bbox=VALUE            bounding box to restrict the file search and
35
                               the file size (only with OpenDAP service). Use
36
                               west, south, east, north .
37
  -s, --dtstart=VALUE         the beginning of the time query. YYYY-MM-
38
                               DDTHH:mm:ssZ .
39
  -e, --dtend=VALUE           the end of the time query. YYYY-MM-DDTHH:mm:ssZ
40
                               .
41
  -t, --basetime=VALUE        the start of time coverage of the Dataset. YYYY-
42
                               MM-DDTHH:mm:ssZ .
43
  -v, --variable=VALUE        variable
44
  -h, --help                 show this message and exit
45
</code></pre>
46
47
h2. Download from rdf url
48
49
The --file option allows the user to automatically parse an rdf response from the ESGFGateway, in order to find all downloadable online resources, HTTP or OpenDAP.
50
In the case of OpenDAP resources, the Client try to retrieve from the RDF file all the parameters needed for a query: a time start and a time stop (for temporal queries), a bounding box (for spatial queries) and the variable to query.
51
52
All these files are protected by OpenID, so the user has to give his credentials by specifying the "--openid" option an --password. If you don't have an OpenID account visit the page https://pcmdi9.llnl.gov/esgf-web-fe/createAccount. Conversely the security certificates are automatically downloaded by the WGET script.
53
54
Example:
55
<pre><code class="XML">
56
ESGFClient -f "http://sb-10-11-31-23.test.terradue.int/catalogue/cmip5rcp/zos/rdf?bbox=-111,42,-119,43&startTime=1962-10-01T12:00:00Z&stopTime=1963-01-01T12:00:00Z&startPage=0&count=3" -o "https://pcmdi9.llnl.gov/esgf-idp/openid/username" -p "password" -F
57
</code></pre>
58
59
In order to build these correct OpenDAP query, the input bounding box and the time search parameters are converted to a set of indexes needed to make a spatial and temporal query.
60
61
h2. Download user defined resources
62
63
The --url option is used to download a user defined resource, by specifying also the temporal coverage and spatial bounding box.
64
65
Example:
66
67
h3. HTTP resource:
68
69
<pre><code class="XML">
70
ESGFClient -u "http://bcccsm.cma.gov.cn/thredds/fileServer/cmip5_data/output/BCC/bcc-csm1-1/decadal2005/mon/ocean/zos/r1i1p1/zos_Omon_bcc-csm1-1_decadal2005_r1i1p1_200601-203512.nc" -o "https://pcmdi9.llnl.gov/esgf-idp/openid/username" -p "password"
71
</code></pre>
72
73
h3. OpenDAP resource:
74
75
<pre><code class="XML">
76
ESGFClient -u "http://bcccsm.cma.gov.cn/thredds/dodsC/cmip5.output1.BCC.bcc-csm1-1.decadal2005.mon.ocean.Omon.r1i1p1.zos.1.aggregation" --bbox "-111,42,-112,43" --dtstart "2006-02-16T12:00:00Z" --dtend "2006-05-16T12:00:00Z" --basetime "2006-01-16T12:00:00Z" -o "https://pcmdi9.llnl.gov/esgf-idp/openid/username" -p "password"
77
</code></pre>
78
The download is based on a revised wget script built upon a template called *wgetTemplate.sh*, filled with the URLs to be gotten.
79
80
Then the client run the WGET script, that saves the files into the file system.
81
82
The --Force option allows to specify if all the HTTP and OpenDAP online resources from the RDF should be download or simple the first.