{
"cells": [
{
"cell_type": "markdown",
"id": "2fdffacb",
"metadata": {
"tags": []
},
"source": [
"# Exploring ECOSTRESS L2T LSTE \n",
"\n",
"**Summary** \n",
"\n",
"This notebook will show how to access [ECOsystem Spaceborne Thermal Radiometer Experiment on Space Station (ECOSTRESS)](https://ecostress.jpl.nasa.gov/) data programmatically using the [`earthaccess`](https://github.com/nsidc/earthaccess) python library leaveraging NASA's Common Metadata Repository (CMR) and enabling authentication, searching, downloading, and streaming of data with minimal coding. It also shows how to work with ECOSTRESS Tiled Land Surface Temperature and Emissivity ([`ECOSTRESS_L2T_LSTE`](https://doi.org/10.5067/ECOSTRESS/ECO_L2T_LSTE.002)) product hosted in the cloud and managed by the Land Processes Distributed Active Archive Center ([LP DAAC](https://lpdaac.usgs.gov/)). \n",
"\n",
"**Learning Objectives** \n",
"\n",
"- How to search ECOSTRESS data using `earthaccess`\n",
"- How to stream or download ECOSTRESS data\n",
"- How to clip ECOSTRESS data to a Region of Interest (ROI)\n",
"- How to quality filter ECOSTRESS data\n",
"- How to export the processed ECOSTRESS data\n",
"\n",
"\n",
"**Requirements** \n",
"\n",
"- NASA [Earthdata Login](https://urs.earthdata.nasa.gov/) account. If you do not have an Earthdata Account, you can create one [here](https://urs.earthdata.nasa.gov/users/new). "
]
},
{
"cell_type": "markdown",
"id": "c8756af0",
"metadata": {},
"source": [
"## Setup \n",
"\n",
"Import the required libraries."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e2a5c6a6",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Import Packages\n",
"import os\n",
"import earthaccess\n",
"import numpy as np\n",
"import rasterio as rio\n",
"import rioxarray as rxr\n",
"import xarray as xr\n",
"import hvplot.xarray\n",
"import hvplot.pandas\n",
"import geopandas as gp\n",
"from shapely.geometry import box\n",
"import pandas as pd\n",
"import panel as pn"
]
},
{
"cell_type": "markdown",
"id": "eeca1d96",
"metadata": {},
"source": [
"## Authentication\n",
"\n",
"Log into Earthdata using the `Auth` and `login` functions from the `earthaccess` library. The `persist=True` argument will create a local `.netrc` file if it doesn't exist, or add your login info to an existing `.netrc` file. If no Earthdata Login credentials are found in the `.netrc`, you'll be prompted for them.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d7d80eab",
"metadata": {},
"outputs": [],
"source": [
"auth = earthaccess.login(persist = True)\n",
"# are we authenticated?\n",
"print(auth.authenticated)"
]
},
{
"cell_type": "markdown",
"id": "16a5293b",
"metadata": {},
"source": [
"## Search for ECOSTRESS Data\n",
"\n",
"In this example, we will use the cloud-hosted `ECOSTRESS_L2T_LSTE` product but the searching process can be used with other EMIT or ECOSTRESS products, other collections, or different data providers, as well as across multiple catalogs with some modification. The Land Surface Temperature and Emissivity values from ECOSTRESS Level 2 Tiled Land Surface Temperature (ECO_L2T_LSTE) are derived from five thermal infrared (TIR) bands using a physics-based Temperature and Emissivity Separation (TES) algorithm. This tiled data product uses a modified version of the Military Grid Reference System (MGRS) which divides Universal Transverse Mercator (UTM) zones into square tiles that are 109.8 km by 109.8 km with a 70 meter (m) spatial resolution. \n",
"\n",
"**Define Your Query Parameters**\n",
"\n",
"We can search for granules using attributes such as collection short name, collection ID, acquisition time, and spatial footprint.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "72653eec",
"metadata": {},
"outputs": [],
"source": [
"# Define our study area: Boulder, Colorado\n",
"# These coordinates define a bounding box around Boulder\n",
"bbox = (-105.301, 39.957, -105.178, 40.094)\n",
"print(f\"🎯 Study area: Boulder, Colorado\")\n",
"print(f\"📍 Bounding box: {bbox}\")\n",
"\n",
"# Create a simple polygon for our study area (for visualization)\n",
"bbox_geom = box(*bbox)\n",
"study_area = gp.GeoDataFrame([1], geometry=[bbox_geom], crs='EPSG:4326')\n",
"study_area['name'] = 'Boulder, CO'\n",
"\n",
"# Let's visualize our study area\n",
"study_area.hvplot(\n",
" tiles='ESRI', geo=True, fill_alpha=0, line_color='red', line_width=2,\n",
" frame_height=400, frame_width=600, title='Study Area: Boulder, Colorado'\n",
")"
]
},
{
"cell_type": "markdown",
"id": "38c5a2a1",
"metadata": {},
"source": [
"Below, the parameters including `provider`, `short_name`, `version`, `bounding_box`, `temporal` and `count` are used for our query. "
]
},
{
"cell_type": "markdown",
"id": "4ee8f472",
"metadata": {},
"source": [
"Next, get the downloadable links for LST and quality layers using `data_links()` method from `earthaccess`. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7dd951ef",
"metadata": {},
"outputs": [],
"source": [
"# Search for data using this bbox\n",
"results = earthaccess.search_data(\n",
" provider='LPCLOUD',\n",
" short_name='ECO_L2T_LSTE',\n",
" version='002',\n",
" bounding_box=bbox,\n",
" temporal=('2023-07-01','2023-08-01'),\n",
" count=100\n",
")"
]
},
{
"cell_type": "markdown",
"id": "541af960",
"metadata": {},
"source": [
"Next, get the downloadable links for LSTE, quality, and cloud layers using data_links() method from earthaccess."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "eff1e96e",
"metadata": {},
"outputs": [],
"source": [
"# Get links\n",
"lst_links = [l for dl in results for l in dl.data_links() if 'LST.tif' in l]\n",
"# Show first 5\n",
"lst_links[:5]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "aadc9920",
"metadata": {},
"outputs": [],
"source": [
"qc_links = [l for dl in results for l in dl.data_links() if 'QC.tif' in l]\n",
"qc_links[:5]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7657f671",
"metadata": {},
"outputs": [],
"source": [
"cl_links = [l for dl in results for l in dl.data_links() if 'cloud.tif' in l]\n",
"cl_links[:5]"
]
},
{
"cell_type": "markdown",
"id": "dbcce044",
"metadata": {},
"source": [
"Let's take a look at the ECOSTRESS tiled data file name: \n",
"\n",
" Filename: **ECOv002_L2T_LSTE_28527_009_13TDE_20230718T081442_0710_01_LST.tif** \n",
"\n",
" ECO : Sensor \n",
" v002 : Product Version \n",
" L2T : Processing Level and Type (T = Tile) \n",
" LSTE : Geophysical Parameter \n",
" 28527 : Orbit Number \n",
" 009 : Scene ID \n",
" 13TDE : Military Grid Reference System (MGRS) Tile ID \n",
" 20230718 : Date of Acquisition (YYYYMMDD) \n",
" T081442 : Time of Acquisition (HHMMSS) (in UTC) \n",
" 0710 : Build ID of software that generated product, Major+Minor (2+2 digits) \n",
" 01 : Product Iteration Number \n",
" LST : Layer/band Name (each layer is a separate file) \n",
" .tif : Data Format for Tile \n",
"\n",
"\n",
"Looking at Military Grid Reference System (MGRS) Tile ID of the outputs, they are all all in UTM Zone 13. "
]
},
{
"cell_type": "markdown",
"id": "d8995a79",
"metadata": {},
"source": [
"## Accessing ECOSTRESS L2T Land Surface Temperature\n",
"\n",
"ECOSTRESS data is stored in NASA's Earthdata Cloud and can be accessed in different ways.\n",
"\n",
"**Downloading** – This has been a supported option since the inception of NASA's DAACs. Users can use the data link(s) to download files to their local working environment. This method works in both cloud and non-cloud environments.\n",
"\n",
"**Streaming** – Streaming enables on-the-fly reading of remote files (i.e., files not saved locally). However, the accessed data must fit into the workspace’s memory. Streaming works in both cloud and non-cloud environments. Streaming data stored in the cloud without downloading is called **in-place access or direct S3 access**, this is only available when working in a cloud environment deployed in AWS us-west-2.\n",
"\n",
"The Python libraries used to access COG files in Earthdata Cloud leverage GDAL's virtual file systems. Whether you are running this code in the Cloud or in a local workspace, GDAL configurations must be set in order to successfully access the ECOSTRESS COG files. For this exercise, we are going to open up a context manager for the notebook using the `rasterio.env` module to store these configurations. The context manager sends this information, including an authentication token or cookie when connecting to a file and can also customize how the file is handled locally. A list of all available config options can be found in the [GDAL config options documentation](https://gdal.org/en/stable/user/configoptions.html).\n",
"\n",
"While the context manager is open (env.enter()) we will be able to run the open or get data commands that would typically be executed within a \"with\" statement. Entering the context manager for multiple cells of the notebook allows us to more freely interact with the data. We’ll close the context manager (env.exit()) when we have all of the data loaded into memory.\n",
"\n",
"In this example, we will show how to stream the data. For that, the gdal configuration is set and the one of our LSTE files is read into the workspace using `open_rasterio` from the `rioxarray` library. Since the file consists of only 1 layer, we can `squeeze` it, removing the `band` dimension."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fc3683c4",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"rio_env = rio.Env(GDAL_DISABLE_READDIR_ON_OPEN='EMPTY_DIR',\n",
" GDAL_HTTP_COOKIEFILE=os.path.expanduser('~/cookies.txt'),\n",
" GDAL_HTTP_COOKIEJAR=os.path.expanduser('~/cookies.txt'),\n",
" GDAL_HTTP_MAX_RETRY = \"10\",\n",
" GDAL_HTTP_RETRY_DELAY = '0.5',\n",
" GDAL_HTTP_UNSAFESSL = 'YES'\n",
" )\n",
"rio_env.__enter__()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0cfaa14d",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"eco_lst_ds = rxr.open_rasterio(lst_links[14]).squeeze('band', drop=True)\n",
"eco_lst_ds"
]
},
{
"cell_type": "markdown",
"id": "932e9fa7",
"metadata": {},
"source": [
"As mentioned, the ECOSTRESS product we are using here is tiled and the CRS is dependent on UTM zone. For this tile, we can look at the `spatial_ref` variable through the interactive object above to see details such as the well-known-text (WKT) representation of the CRS and other attributes. \n",
"we are using `hvplot` for visualization here. For detailed information on available open-source Python tools and libraries for data visualization see
{key}: {value}
\"\n", " stats_html += \"