Skip to content

Latest commit

 

History

History
188 lines (102 loc) · 7.36 KB

CONFIG.rst

File metadata and controls

188 lines (102 loc) · 7.36 KB

Configuration

Setting up the Credentials for EO Data Providers

The BDC Collection Builder uses BDC-Collectors to access and download data from remote providers.

Copernicus (Sentinel 2A and 2B)

Note

This step is required if you would like to change credential for SciHub/ESA. The commands make changes in database. Make sure you have exported SQLALCHEMY_DATABASE_URI=postgresql://<user>:<password>@<host>/<dbname> with proper values.

In order to search and obtain images from Copernicus SciHub (e.g. Sentinel-2A and 2B images), users must have a registered account at: https://scihub.copernicus.eu/dhus/#/self-registration and confirm validation through email. This account may take a few days to be operational when using it in scripts.

The BDC-Collection-Builder provide a command line :func:`bdc_collection_builder.cli.load_providers`. Edit the credentials key in the file examples/data/providers/scihub.json with username and password. After that, load or edit credentials with command:

bdc-collection-builder load-providers --ifile examples/data/providers/scihub.json --update

Once provider is created/updated, you must attach it into collection as following:

bdc-collection-builder set-provider --collection S2_L1C-1 --provider ESA

Note

You must have a collection S2_L1C-1. Please, check the Prepare the Database System in install step. The value ESA is the name of provider found in examples/data/providers/scihub.json.

The collection S2_L1C-1 will be marked as data origin from provider ESA. You can check it with command:

bdc-collection-builder overview --collection S2_L1C-1

The output:

-> title: Sentinel-2 - MSI - Level-1C
-> name: S2_L1C
-> version: 1
-> description: Level-1C product provides orthorectified Top-Of-Atmosphere (TOA) reflectance.
-> collection_type: collection
-> Providers:
  - ESA, driver=SciHub, priority=1, active=True

CREODIAS (Sentinel 2A and 2B)

Note

This section is optional. It is an alternative to avoid the Copernicus Long Term Archive.

Due the Copernicus Long Term Archive, you may face issues while requesting for download period higher than a year due Retention Policy. In this way, you can set a credential to allow bdc-collection-builder download from CREODIAS provider when scene is offline.

In order to search and obtain images from SciHub mirror CREODIAS, users must have a registered account at: https://creodias.eu/ and confirm validation through email.

Edit the credentials key in the file examples/data/providers/creodias.json with username and password. After that, load or edit credentials with command:

bdc-collection-builder load-providers --ifile examples/data/providers/creodias.json --update

Once provider is created/updated, you must attach it into collection as following:

bdc-collection-builder set-provider --collection S2_L1C-1 --provider CREODIAS --priority 2

Note

We are configuring CREODIAS with priority 2. In this case, it will be used as fallback if any error occurs in ESA. The lowest value is for priority.

Now, if you show overview for collection S2_L1C-1, the provider ESA and CREODIAS will be marked as data origin from provider:

bdc-collection-builder overview --collection S2_L1C-1

The output:

-> title: Sentinel-2 - MSI - Level-1C
-> title: Sentinel-2 - MSI - Level-1C
-> name: S2_L1C
-> version: 1
-> description: Level-1C product provides orthorectified Top-Of-Atmosphere (TOA) reflectance.
-> collection_type: collection
-> Providers:
  - ESA, driver=SciHub, priority=1, active=True
  - CREODIAS, driver=CREODIAS, priority=2, active=True

USGS (Landsat)

In order to search and obtain images from USGS Earth Explorer (e. g. Landsat-8 images), users must have a registered account at: https://ers.cr.usgs.gov/register/ and confirm validation through email.

Note

In the newest versions, the USGS provider account requires a Product Access Request which is available in User Profile Access Request. Fill out the form and wait for status Approved.

Edit the credentials key in the file examples/data/providers/nasa-usgs.json with username and password. After that, load or edit credentials with command:

bdc-collection-builder load-providers --ifile examples/data/providers/nasa-usgs.json --update

Once provider is created/updated, you must attach it into collection as following:

bdc-collection-builder set-provider --collection LC8_DN-1 --provider USGS --priority 1

Google Cloud Storage

Warning

Due unavailability of Landsat Collection 1, this section is obsolete. Use Landsat Collection 2 instead, which its only available in provider USGS (EarthExplorer)

Note

This section is optional and only works with product Landsat Collection 1.

You must have a Google Account in order to use any Google Cloud Services. In this way, you can create a new one in Create a new Google Account.

After that, you must also register an service account key in Create a Service Account Key and download the service key.

You must set the environment variable GOOGLE_APPLICATION_CREDENTIALS=/path/to/service_account_key.json in order to enable the Google Provider in Collection Builder application.

Edit the GOOGLE_APPLICATION_CREDENTIALS key in the file examples/data/providers/google-datasets.json. After that, load or edit it with command:

bdc-collection-builder load-providers --ifile examples/data/providers/google-datasets.json --update

Once provider is created/updated, you must attach it into collection as following:

bdc-collection-builder set-provider --collection LC8_DN-1 --provider Google --priority 2

Remove attached provider from collection

You may need to detach a provider from collection if you having errors in collector as following:

bdc-collection-builder set-provider --collection S2_L1C-1 --provider CREODIAS --remove