Landsat BIG DATA

The procedures shown in the previous section are sufficient if you want to acquire a few scenes. However, if you need many more scenes, e.g. if you want to do a time series analysis, the manual download becomes a real test of patience.
There are several ways to perform a mass, or bulk, download for Landsat 8 data. Let us take a look at an application for Level-1 and a method for Level-2 Landsat 8 scenes.

Section in a Box

In this section, the following content awaits you:

Bulk Download L8 Level-1 data
– use this when you need about 5-20 Level-1 scenes
Bulk Download L8 Level-2 data
– use this when you need about 5-20 Level-2 scenes
BIG DATA Download
– use this when you need a lot more scenes…

Bulk Download L8 Level-1 data

USGS provides an application for bulk-downloading Level-1 Landsat data with a very appropriate name: the Bulk Download Application. The Bulk Download Application is an easy-to-use tool for downloading large quantities of satellite imagery and geospatial data. In order to use it, you have to order your Landsat 8 scenes via the Add to Bulk Download option as mentioned in the Download section:

Order imagery by choosing bulk download option

Proceed to checkout and confirm your order as shown in the Download section. After that, a specific order number is given to you by email and on the USGS website:

Order number provided after submission

Now open the application by navigating to Other > Bulk Download Application in the start menu of our VM:

Location of Bulk Download Application in our VM

The user interface opens, in which you have to fill in your username and password used for the USGS Earth Explorer orders:

Log in for bulk downloading

Choose your order by its ID and click Select Order:

Choose the order ID for your download

Change the target directory to your liking by clicking on the right top icon to your. We recommend to set this path to “/media/sf_exchange” to not unnecessarily enlarge the VirtualBox storage. Begin your download by pressing Begin Download:

Begin your bulk download

Bulk Download L8 Level-2 data

Unfortunately, Level-2 data is not supported by the USGS bulk download application as shown in the previous section. Anyway, there is a nice python based script provided by USGS-EROS, which downloads automatically all scenes for you! The source code is open source and can be reviewed on GitHub via this link and can also be downloaded here. If you are using our VM, the script is locally stored in your document folder under /home/student/Documents/download_espa_order.py.

First of all, you should follow the guide for ordering Level-2 scenes (Download section) and receive the second email with the download confirmation.
Then open the linux terminal – it is linked in the taskbar of our VM:

Linux terminal shortcut in taskbar

To initiate the bulk download, you just need one command. Here is the blueprint of this command, in which you need to replace your email address, your username and password before executing:

/usr/bin/python /home/student/Documents/download_espa_order.py -r 10 -c -o ALL -d /media/sf_exchange/ -e john.doe@gmail.com -u user123 -p password123

Explanation: The command needs a python installation to work, so we define the location of python via
/usr/bin/python
. The download script itself is named “download_espa_order.py” and is stored in the documents folder of our VM, so we define its location with
/home/student/Documents/download_espa_order.py
. All individual letters preceded by a minus sign are arguments, which are assigned with the subsequent input. You have to change the email-address
-e john.doe@gmail.com
, the username -u user123, and the adequate password -p password123 to your own settings (USGS login data). For more information on this, have a look at the readme on GitHub.
Type in the whole command without any linebreaks into the terminal:

Terminal command for bulk downloading Landsat Level-2 data

Press the enter-key and watch the script downloading and saving all ordered scenes to your exchange folder -d /media/sf_exchange/ (you can change this destination folder to your liking):

Terminal command for bulk downloading Landsat Level-2 data

BIG DATA Download

You can not get enough of data and you have some Terrabyte of disk space available? So, if you just want to download ALL the scenes of a search query (up to 20,000 scenes at once), there is an easy way to do it! When you have completed a search query, as shown in the Perform a Search section, there is an button on top of the scene list which can be used to export your results. This button is only clickable when you are logged in:

Metadata Export button

A small window opens asking what format the metadata should be exported to. Choose “Non-Limited Results” and “CSV” in order to export the metadata of every single file found to a csv-file (which is a text file):

Metadata Export options

Click Export and you will receive an email with a link to the metadata export file. The processing time and thus the waiting time for the email will vary depending on number of search results requested.
Click the link in the mail and you will be redirected to a USGS page. Download the zipped csv-file to your harddrive (the exchange folder in our VM is recommended). Unzip the file by right-clicking on it and chose “Extract Here”. Next, open the following USGS website:

https://earthexplorer.usgs.gov/filelist

Select Single Data Set and choose the type of dataset you want to download via the drop-down menu Data Set:

  • Data Set for Level-1: Landsat Collection 1 Level-1: Landsat 8 OLI/TIRS C1 Level-1
  • Data Set for Level-2: Landsat Collection 1 Level-2 (On-Demand): Landsat 8 OLI/TIRS C1 Level-2

Click the button next to File and upload the unzipped .csv-file containing the metadata information. Confirm by pressing Submit File List:

Metadata Export options

If you have selected the Level-1 product, select Bulk Download on the next page and click on Submit Order. Please follow the instructions given in the bulk download section above to complete your download with the bulk download application.

Level-1 product Bulk Download order

If you have selected the Level-2 product, select Order on the next page and click on Submit Order, wait for the ESPA confirmation email (Download section) and follow the instructions given in the Bulk Download section above:

Level-2 product Bulk Download order