{ "cells": [ { "cell_type": "markdown", "id": "4c51d3fb", "metadata": {}, "source": [ "## Find Colocated PACE and EMIT Data\n", "Adapted from the NASA VITALS [\"Finding Concurrent ECOSTRESS and EMIT Data\"](https://nasa.github.io/VITALS/python/01_Finding_Concurrent_Data.html) tutorial.\n", "\n", "Original Author: Erik Bolch2\n", "\n", "Adapted for PACE by: Skye Caplan1 , Erik Bolch2\n", "\n", "1. Science Systems and Applications, Inc., NASA Goddard Space Flight Center.\n", "2. KBR Inc., contractor to the USGS Earth Observation Research and Science Center, NASA Land Processes Distributed Active Archive Center.\n", "\n", "Last updated: 06/24/2026\n", "\n", "
\n", "\n", "An [Earthdata Login][edl] account is required to access data from the NASA Earthdata system, including NASA PACE and EMIT data.\n", "\n", "
\n", "\n", "[edl]: https://urs.earthdata.nasa.gov/\n", "\n", "## Summary\n", "\n", "NASA's Plankton, Aerosol, Cloud, ocean Ecosystem (PACE) satellite mission and Earth Surface Mineral Dust Source Investigation (EMIT) sensor aboard the International Space Station (ISS) both make hyperspectral observations of the Earth's surface. Taking into account their different spectral, spatial, and temporal coverage, these instruments' complementary capabilities provide a wealth of new opportunities when used in concert. In this series of tutorials, we will demonstrate how to find, prepare, and use data from PACE and EMIT together, illustrating both missions' strengths and how they can help fill in each other's gaps. \n", "\n", "This first tutorial in the series will pull data from EMIT and PACE, specifically PACE's Ocean Color Instrument (OCI), using the `earthaccess` Python library, and demonstrate how to find colocated granules using file metadata. Ultimately, these data will be used in a case study analyzing the 2024-25 wildfires affecting Grampians National Park in Victoria, Australia.\n", "\n", "## Background\n", "\n", "In order to introduce the synergistic use of these two datasets, we will look at PACE OCI and EMIT observations from the same event, namely wildfires which affected Grampians (Gariwerd) National Park in Victoria, Australia. Two severe fires burned a large portion of the park in a short amount of time - The Yarram Gap Road fire which ignited ~December 16, 2024, and the Wallaby Rocks fire which ignited ~January 29, 2025. These fires started separately but merged into a large complex, burning roughly ~130,000-140,000 ha (State of Victoria, 2025). Before the fires, some areas had not burned in many years, contributing to the fuel load.\n", "\n", "The park is home to several endemic flora adapted to the landscape and dry, nutrient-poor (low nitrogen, phosphorus, and zinc) soil. Several categories of plant communities exist within the park, including sub-alpine, open forest/scrub, red gum forest, low open shrubland and heathy woodland. For more info about these you can see the [Grampians National Park Flora Guide](https://www.parks.vic.gov.au/-/media/project/pv/main/parks/documents/visitor-guides-and-publications/grampians/grampians-national-park_floraguide.pdf). Fire is common during the dry season, and as such, many species are dependent on fire as part of their survival and dispersal (Parks Victoria, nd).\n", "\n", "References \n", "\n", "1. State of Victoria. 12 September 2025. Fire History Records of Fires across Victoria. Dataset. Retrieved 28 April 2026. https://datashare.maps.vic.gov.au/search?md=d817f3be-a7a2-5847-9944-c0f2fdb39444.\n", "\n", "2. Parks Victoria. Grampians National Park (Gariwerd) Flora Guide. Retrieved 10 April 2026. https://www.parks.vic.gov.au/-/media/project/pv/main/parks/documents/visitor-guides-and-publications/grampians/grampians-national-park_floraguide.pdf\n", "\n", "In order to fully explore these fires and their impact on the National Park land, we'll want to use PACE OCI's and EMIT's complementary observation techniques. Referring to the instrument summary below, we can see that OCI has higher temporal revisit frequency, making near-daily, global observations, while EMIT has finer spatial resolution, providing sharp imagery of smaller slices of the Earth. The instruments also have distinct spectral characteristics, which can be used together for full coverage reflectances in the ultraviolet, visible, near infrared, and shortwave infrared regions.\n", "\n", "| Resolutions | EMIT | PACE OCI |\n", "| -------- | -------- | -------- |\n", "| Spatial | 60 m | 1.2 km (at nadir)|\n", "| Temporal | Variable | 1 - 2 days|\n", "| Spectral Range | 380 - 2500 nm | 350 - 895, with 7 multispectral SWIR bands from 935-2258 nm|\n", "| Spectral Sampling | 7.5 nm | 2.5 nm, with 1.25 nm in some spectral regions |\n", "\n", "## Learning Objectives\n", "\n", "At the end of this notebook, you will know how to:\n", "- Search for both PACE and EMIT data using the `earthaccess` package\n", "- Filter those search results to find concurrent datasets based on overpass time, bounding box, etc.\n", "- Download only the datasets which match your filter criteria. \n", "- Understand the differences in each instruments' orbit characteristics\n", "\n", "## Contents\n", "1. [Setup](#1-setup) \n", "2. [Searching for Data](#2-searching-for-data) \n", "3. [Filtering Data](#3-filtering-data) \n", "4. [Making the most of PACE's temporal coverage](#4-making-the-most-of-paces-temporal-coverage) \n", "5. [Downloading Data](#5-downloading-data) \n", "\n", "\n", "## 1. Setup \n", "\n", "Begin by importing all of the packages used in this notebook." ] }, { "cell_type": "code", "execution_count": null, "id": "af86b37f", "metadata": {}, "outputs": [], "source": [ "import folium\n", "import earthaccess\n", "import numpy as np \n", "import pandas as pd\n", "import geopandas as gpd\n", "import matplotlib.pyplot as plt\n", "from skimage import io\n", "from shapely import geometry\n", "from datetime import timedelta\n", "from branca.element import Figure\n", "from IPython.display import display\n", "from shapely.geometry import Polygon, box\n", "\n", "import warnings\n", "warnings.filterwarnings('ignore')" ] }, { "cell_type": "markdown", "id": "07f07819", "metadata": {}, "source": [ "## 2. Searching for Data \n", "\n", "One of the easiest ways to retrieve NASA data is through the `earthaccess` package, which allows us to search quickly through the multitude of datasets available on [NASA's Earthdata Cloud](https://earthdata.nasa.gov) for the ones which match our criteria. Please make sure you have the most updated version of the package (`earthaccess>=0.17.0`), as some of the functions in this notebook rely on newer utilities. In this tutorial, we'll be looking for observations of Grampians National Park from our two instruments of interest before and after the wildfires, to understand how each mission views the change in landscape. \n", "\n", "The following cell will allow you to log in to NASA's Earthdata Cloud catalogue just as you would through a browser with the `earthaccess.login()` method. Note that you can add `persist=True` as an argument to `earthaccess.login()` to save your credentials in a `.netrc` file, so you won't need to login using your Earthdata credentials each time you use this notebook. " ] }, { "cell_type": "code", "execution_count": null, "id": "316d62ef", "metadata": {}, "outputs": [], "source": [ "auth = earthaccess.login()" ] }, { "cell_type": "markdown", "id": "3b9613ea", "metadata": {}, "source": [ "Now that we're logged in, we can define parameters to begin our search for data using a specific timespan, a region of interest (ROI), and the products we are looking for.\n", "\n", "From the background section above, we know the location and time period over which these fires burned. We also know that EMIT will be the temporal limiting factor for the analyses in the next few notebooks due to its variable revisit time. Thankfully, our interest in the pre- and post-fire conditions of the park will allow us to use data from a month or two before and after the fires if necessary. We'll also be searching for surface reflectances from each mission to investigate the fires, looking for our pre-fire granules first, and the post-fire data after. \n", "\n", "The Yarram Gap Road fire began around December 16th, 2024, so we will set our search about a month out from that date. We will also need to know the `short_names` of the products we want to search for, essentially the names of the datasets in the Earthdata catalogue. " ] }, { "cell_type": "code", "execution_count": null, "id": "b90752cc", "metadata": {}, "outputs": [], "source": [ "tspan = ('2024-11-01','2024-12-16')\n", "roi_bounds = (141.9385, -37.74163, 142.77324, -36.79961)\n", "\n", "bbox = geometry.box(*roi_bounds, ccw=True)\n", "roi = list(bbox.exterior.coords)\n", "gdf = gpd.GeoDataFrame(index=[0], crs='epsg:4326', geometry=[bbox])\n", "\n", "prods = [\"PACE_OCI_L2_SFREFL\", \"EMITL2ARFL\"]" ] }, { "cell_type": "markdown", "id": "dc394a70", "metadata": {}, "source": [ "With the parameters set, we can use `earthaccess.search_data()` to look through our chosen collections." ] }, { "cell_type": "code", "execution_count": null, "id": "9f2bb2a7", "metadata": {}, "outputs": [], "source": [ "results = earthaccess.search_data(\n", " short_name=prods,\n", " polygon=roi,\n", " temporal=tspan,\n", ")\n", "\n", "len(results)" ] }, { "cell_type": "markdown", "id": "c7ffb258", "metadata": {}, "source": [ "Running the search gathered ~70 granules that match the parameters. That is a lot of data! Let's examine what the first two results from the `earthaccess` call look like. Running the cell below, we have 2 EMIT granules with all their associated metadata that matched the criteria in our query above. If you were to look through the whole `results` list, you'd see the same for however many PACE OCI granules fit the search parameters as well. These metadata will allow us to sort through all of the results to choose the best granules for our analyses." ] }, { "cell_type": "code", "execution_count": null, "id": "14f8f237", "metadata": {}, "outputs": [], "source": [ "results[:2]" ] }, { "cell_type": "markdown", "id": "43125768", "metadata": {}, "source": [ "## 3. Filtering Data \n", "\n", "In order to use the metadata efficiently, and to make it more human-readable, we can make our list of results into a `GeoDataFrame` by taking the spatial metadata and creating a shapely polygon representing the spatial coverage, and further customize which information we want to use from other metadata fields. These methods are explained more thoroughly in the VITALS ECOSTRESS-EMIT tutorial [`01_Finding_Concurrent_Data`](https://nasa.github.io/VITALS/python/01_Finding_Concurrent_Data.html). " ] }, { "cell_type": "code", "execution_count": null, "id": "cb9329d5", "metadata": {}, "outputs": [], "source": [ "# Functions to Build Dataframe\n", "def get_shapely_object(result: earthaccess.results.DataGranule):\n", " \"\"\"\n", " Create a shapely polygon of spatial coverage from results metadata.\n", " Will work for BoundingRectangles or GPolygons.\n", " \"\"\"\n", " # Get Geometry Keys\n", " geo = result[\"umm\"][\"SpatialExtent\"][\"HorizontalSpatialDomain\"][\"Geometry\"]\n", " keys = geo.keys()\n", "\n", " if \"BoundingRectangles\" in keys:\n", " bounding_rectangle = geo[\"BoundingRectangles\"][0]\n", " # Create bbox tuple\n", " bbox_coords = (\n", " bounding_rectangle[\"WestBoundingCoordinate\"],\n", " bounding_rectangle[\"SouthBoundingCoordinate\"],\n", " bounding_rectangle[\"EastBoundingCoordinate\"],\n", " bounding_rectangle[\"NorthBoundingCoordinate\"],\n", " )\n", " # Create shapely geometry from bbox\n", " shape = box(*bbox_coords, ccw=True)\n", " elif \"GPolygons\" in keys:\n", " points = geo[\"GPolygons\"][0][\"Boundary\"][\"Points\"]\n", " # Create shapely geometry from polygons\n", " shape = Polygon([[p[\"Longitude\"], p[\"Latitude\"]] for p in points])\n", " else:\n", " raise ValueError(\n", " \"Provided result does not contain bounding boxes/polygons or is incompatible.\"\n", " )\n", " return shape\n", "\n", "# Functions to Build Dataframe\n", "def get_png(result: earthaccess.results.DataGranule):\n", " \"\"\"\n", " Retrieve a png browse image if it exists or first jpg in list of urls\n", " \"\"\"\n", " https_links = [link for link in result.dataviz_links() if \"https\" in link]\n", " if len(https_links) == 1:\n", " browse = https_links[0]\n", " elif len(https_links) == 0:\n", " browse = \"no browse image\"\n", " else:\n", " browse = [png for png in https_links if \".png\" in png][0]\n", " if browse == 'no browse image':\n", " print(\"Some granules have no browse imagery.\")\n", " return browse\n", "\n", "def results_to_gdf(results: list, keep_cols: list = None, roi:gpd.GeoDataFrame=None):\n", " \"\"\"\n", " Takes a list of results from earthaccess and converts to a geodataframe.\n", " \"\"\"\n", " # Create Dataframe of Results Metadata\n", " results_df = pd.json_normalize(results)\n", " # Shorten column Names\n", " results_df.columns = [\n", " col.split(\".\")[-1] if \".\" in col else col for col in results_df.columns\n", " ]\n", " # Create shapely polygons for result\n", " geoms = gpd.GeoSeries(results, crs=4326)\n", " # Convert to GeoDataframe\n", " gdf = gpd.GeoDataFrame(results_df, geometry=geoms, crs=\"EPSG:4326\")\n", " # Add browse imagery and data links and product shortname\n", " gdf[\"browse\"] = [get_png(granule) for granule in results]\n", " gdf[\"shortname\"] = [\n", " result[\"umm\"][\"CollectionReference\"][\"ShortName\"] for result in results\n", " ]\n", " gdf[\"data\"] = [granule.data_links() for granule in results]\n", " \n", " # Drop Unwanted Columns\n", " default_cols = [\n", " \"native-id\",\n", " \"collection-concept-id\",\n", " \"BeginningDateTime\",\n", " \"EndingDateTime\",\n", " \"CloudCover\",\n", " \"DayNightFlag\",\n", " \"geometry\",\n", " \"browse\",\n", " \"shortname\",\n", " \"data\",\n", " ] \n", "\n", " if not keep_cols:\n", " keep_cols = default_cols\n", " else:\n", " keep_cols = list(set(keep_cols + default_cols))\n", "\n", " gdf = gdf[[col for col in keep_cols if col in gdf.columns]]\n", "\n", " # Convert Timestamps\n", " gdf['BeginningDateTime'] = pd.to_datetime(gdf['BeginningDateTime'])\n", " gdf['EndingDateTime'] = pd.to_datetime(gdf['EndingDateTime'])\n", " gdf['datetime_obj'] = pd.to_datetime(gdf['BeginningDateTime'], format='ISO8601')\n", "\n", " \n", " if roi is not None:\n", " gdf_eq = gdf.to_crs(\"EPSG:8857\")\n", " roi_eq = roi.to_crs(\"EPSG:8857\").union_all()\n", " \n", " gdf['overlap_frac'] = gdf_eq.geometry.intersection(roi_eq).area / gdf_eq.geometry.area\n", " \n", " return gdf" ] }, { "cell_type": "markdown", "id": "882a0c98", "metadata": {}, "source": [ "We can place our results in a geodataframe and look at the first few lines with the cell below. Each granule has various \"IDs\" related to the file or collection name, the host Distributed Active Archive Center (DAAC), and more. There are also other columns related to the boundaries, time of data collection, and more - not all of this information will be useful to us in finding collocated PACE and EMIT observations." ] }, { "cell_type": "code", "execution_count": null, "id": "9d5bd613", "metadata": {}, "outputs": [], "source": [ "gdf = results_to_gdf(results)\n", "\n", "gdf.head()" ] }, { "cell_type": "markdown", "id": "aae71d81", "metadata": {}, "source": [ "To view how many individual overpasses exist for the ROI from each instrument in time, we'll now create separate dataframes for PACE OCI and EMIT. This will also allow us to compare the time of overpass for each granule on the same day. " ] }, { "cell_type": "code", "execution_count": null, "id": "a40de4c1", "metadata": {}, "outputs": [], "source": [ "pace_gdf = gdf[gdf['native-id'].str.contains('OCI')]\n", "emit_gdf = gdf[gdf['native-id'].str.contains('EMIT')]\n", "print(f' PACE OCI Granules: {pace_gdf.shape[0]} \\n EMIT Granules: {emit_gdf.shape[0]}')" ] }, { "cell_type": "markdown", "id": "dc8d91bc", "metadata": {}, "source": [ "Clearly, most of our results were PACE granules - that makes sense given the orbit characteristics laid out in the background section of this tutorial. We want to find which of these many overpasses overlap spatially with the EMIT observations, so we'll subset PACE granules in `pace_gdf` by whether they intersect with EMIT granules. We'll create a new column based on intersection with the union of EMIT polygons." ] }, { "cell_type": "code", "execution_count": null, "id": "999e847b", "metadata": {}, "outputs": [], "source": [ "pace_gdf['intersects'] = pace_gdf.intersects(emit_gdf.union_all())\n", "\n", "# Subset by intersection\n", "pace_gdf = pace_gdf[pace_gdf['intersects'] == True]\n", "print(f' PACE OCI Granules: {pace_gdf.shape[0]} \\n EMIT Granules: {emit_gdf.shape[0]}')" ] }, { "cell_type": "markdown", "id": "3932e50f", "metadata": {}, "source": [ "After checking for spatial overlap, we still have a ton of PACE data to work with, thankfully. \n", "\n", "The next step is to make sure the measurements from both instruments are comparable in time to minimize any variation in cloud cover, solar angle, etc. PACE has a local overpass time of ~1:00 pm, whereas EMIT's overpasses are more variable, so we'll use a threshold of +/- 6 hours to match the datasets." ] }, { "cell_type": "code", "execution_count": null, "id": "28609ebf", "metadata": {}, "outputs": [], "source": [ "def concurrent_match(gdf_a:pd.DataFrame, gdf_b:pd.DataFrame, col_name:str, time_delta:timedelta):\n", " \"\"\"\n", " Cross references dataframes containing a datetime object column and keeps rows in \n", " each that fall within the provided timedelta of the other. Acceptable time_delta examples:\n", " \n", " months=1\n", " days=1\n", " hours=1\n", " minutes=1\n", " seconds=1\n", " \"\"\"\n", " # Match Timestamps from Dataframe A with Time-range of entries in Dataframe B\n", " # Create empty list\n", " a_list = []\n", " # Iterate results for product a based on index values\n", " for _n in gdf_b.index.to_list():\n", " # Find where product b is within the window of each product a result\n", " a_matches = (gdf_a[col_name] > gdf_b[col_name][_n]-time_delta) & (gdf_a[col_name] < gdf_b[col_name][_n]+time_delta)\n", " # Append list with values\n", " a_list.append(a_matches)\n", " # Match Timestamps from Dataframe B with Time-range of entries in Dataframe A\n", " # Create empty list\n", " b_list =[]\n", " for _m in gdf_a.index.to_list():\n", " # Find where product a is within the window of each product b result\n", " b_matches = (gdf_b[col_name] > gdf_a[col_name][_m]-time_delta) & (gdf_b[col_name] < gdf_a[col_name][_m]+time_delta)\n", " # Append list with values\n", " b_list.append(b_matches)\n", " # Filter Original Dataframes by summing list of bools, 0 = outside of all time-ranges\n", " a_filtered = gdf_a.loc[sum(a_list) > 0]\n", " b_filtered = gdf_b.loc[sum(b_list) > 0]\n", " return(a_filtered, b_filtered)\n", "\n", "pace_gdf, emit_gdf = concurrent_match(pace_gdf,emit_gdf, col_name='datetime_obj',time_delta=timedelta(hours=6))\n", "print(f'After concurrent match:\\n PACE OCI Granules: {pace_gdf.shape[0]} \\n EMIT Granules: {emit_gdf.shape[0]}')" ] }, { "cell_type": "markdown", "id": "21418d8d", "metadata": {}, "source": [ "The number of PACE granules has drastically decreased, which means that the list of PACE granules should only contain those within 6 hours of one of our 7 EMIT granules. Recall that PACE collects data approximately once a day, while EMIT could have several scenes in the same day, which is why the number of granules does not match. \n", "\n", "The final step in choosing the best observation day for our purposes is to make sure the ROI is clear of clouds or bad data. We can do this by using browse images, basically previews, of each granule. These browse images are simple RGB representations of the data which will give an idea of what the overpass looks like before we decide to download it. \n", "\n", "Let's plot the browse imagery for both PACE and EMIT below:" ] }, { "cell_type": "code", "execution_count": null, "id": "71f45e2b", "metadata": {}, "outputs": [], "source": [ "def plot_browse_images(gdf, cols=3):\n", " \"\"\"\n", " Plot browse images using a given GeoDataFrame with the applicable metadata.\n", " Args:\n", " gdf - GeoDataFrame including the \"browse\" metadata from earthaccess\n", " cols - the number of columns in the desired plot\n", " \n", " Returns:\n", " A plot of browse images\n", " \"\"\"\n", " rows = int(np.ceil(len(gdf)/cols))\n", " fig, ax = plt.subplots(rows, cols, figsize=(20,20))\n", " ax = ax.flatten()\n", "\n", " for _n, index in enumerate(gdf.index.to_list()):\n", " img = io.imread(gdf['browse'][index])\n", " ax[_n].imshow(img)\n", " ax[_n].set_title(f\"Index: {index} - {gdf['native-id'][index]}\")\n", " ax[_n].axis('off')\n", "\n", " for i in range(len(gdf), len(ax)): \n", " fig.delaxes(ax[i]) \n", "\n", " plt.tight_layout() \n", " plt.show() \n", "\n", "plot_browse_images(emit_gdf)" ] }, { "cell_type": "markdown", "id": "8bc2c84c", "metadata": {}, "source": [ "The EMIT granules at index 1 and 2 are taken on the same day and show Grampians National Park most clearly, with no clouds covering the ROI. The black regions are cloudy areas omitted by the on-instrument cloud screening process. \n", "\n", "At time of writing, PACE browse imagery is still being implemented. Once the implementation is complete, we'll be able to make the same plot with PACE. For this exercise, we'll assume our ROI in each OCI granule is free of clouds and isn't affected by any quality issues, which can be verified by this screenshot from [NASA Worldview]() for this scene:\n", "\n", "![An image of the PACE granule which overlaps with the EMIT granule chosen above. Although there are clouds present in the image, they do not affect the area over or around Grampians National Park, the ROI for this analysis](../../img/worldview_20241114T035224_browse.png)" ] }, { "cell_type": "code", "execution_count": null, "id": "335f0107", "metadata": {}, "outputs": [], "source": [ "# Plot PACE browse images - coming soon!\n", "#plot_browse_images(pace_gdf)" ] }, { "cell_type": "markdown", "id": "297ae40f", "metadata": {}, "source": [ "Before we get too excited about using those two granules, the last thing we need to make sure of is that our EMIT granule, and thus our ROI, is in a decent position relative to the entire OCI swath. Its relative location is important because Level 2 (L2) reflectances are in the swath resolution, and OCI's swath is ~2700 km. Because of its very wide swath, pixels toward the east and west edges of a L2 OCI granule tend to include greater angular effects. To minimize those effects, it is preferable to choose EMIT and PACE pairs where the ROI is closer to the center of OCI's swath. \n", "\n", "The browse images are great quick looks, but they do not demonstrate well just how broad OCI's swath is, or where the ROI falls within OCI's swath. Instead, we'll make a quick interactive plot with `folium`:" ] }, { "cell_type": "code", "execution_count": null, "id": "854b546a", "metadata": {}, "outputs": [], "source": [ "# Create Figure and Select Background Tiles\n", "fig = Figure(width=\"750px\", height=\"375px\")\n", "map1 = folium.Map(tiles='https://mt1.google.com/vt/lyrs=y&x={x}&y={y}&z={z}&hl=en', attr='Google')\n", "fig.add_child(map1)\n", "\n", "# Plot PACE OCI Results - note we must drop the datetime_obj columns for this to work\n", "pace_gdf.drop(columns=['datetime_obj']).explore(\n", " \"native-id\",\n", " categorical=True,\n", " tooltip=[\n", " \"native-id\",\n", " \"BeginningDateTime\",\n", " \"CloudCover\",\n", " ],\n", " popup=True,\n", " style_kwds=dict(fillOpacity=0.1, width=2),\n", " name=\"PACE OCI\",\n", " m=map1,\n", " legend=False\n", ")\n", "\n", "# Plot EMIT Results - note we must drop the datetime_obj columns for this to work\n", "emit_gdf.drop(columns=['datetime_obj']).explore(\n", " \"native-id\",\n", " categorical=True,\n", " tooltip=[\n", " \"native-id\",\n", " \"BeginningDateTime\",\n", " \"CloudCover\",\n", " ],\n", " popup=True,\n", " style_kwds=dict(fillOpacity=0.1, width=2),\n", " name=\"EMIT\",\n", " m=map1,\n", " legend=False\n", ")\n", "\n", "folium.GeoJson(bbox,\n", " name='bounding_box',\n", " ).add_to(map1)\n", "\n", "x1, y1, x2, y2 = gdf.union_all().bounds\n", "map1.fit_bounds(bounds=((y1, x1), (y2, x2)))\n", "map1.add_child(folium.LayerControl())\n", "display(fig)" ] }, { "cell_type": "markdown", "id": "0c240de2", "metadata": {}, "source": [ "The `folium` map above illustrates well just how different the area these instruments are observing can be. It also shows clearly where in OCI granule each respective EMIT granule falls - we can see that for the observations on November 14th, 2024, the EMIT granules fall closer to the center of the PACE OCI granule and cover the whole expanse of the park. Conversely, most of the other EMIT granules fall to the edges of their paired OCI granules. \n", "\n", "With these quick checks, we can be confident that the data from November 14th, 2024 is the right pre-fire data to use. Now we can grab those granules from their respective GeoDataFrames and put them in a single list for easy download later. Note that we will be grabbing two EMIT granules to cover the full footprint of Grampians National Park." ] }, { "cell_type": "code", "execution_count": null, "id": "0c32934a", "metadata": {}, "outputs": [], "source": [ "# Filter PACE using date of observation, since there is only one per day here\n", "good_pace_gran = pace_gdf[pace_gdf['native-id'].str.contains(\"20241114\")]\n", "\n", "# Filter EMIT using index from the browse imagery plot\n", "good_idxs = [1,2] \n", "good_emit_grans = emit_gdf[emit_gdf.index.isin(good_idxs)]\n", "keep_granules = good_pace_gran.index.to_list()+good_emit_grans.index.to_list()\n", "keep_granules.sort()\n", "\n", "prefire_results = [result for i, result in enumerate(results) if i in keep_granules]\n", "prefire_results" ] }, { "cell_type": "markdown", "id": "e8745e7a", "metadata": {}, "source": [ "### Search and filter for post-fire data\n", "\n", "We have our pre-fire results, and so can now repeat the process above to find post-fire data. We will combine several of the steps completed above to search and filter the data, especially because the only real change to make is in the time span we include in the `earthaccess.search_data()` call. The fires were controlled around mid-February 2025, so we will set our time span to ~1-2 months after that date." ] }, { "cell_type": "code", "execution_count": null, "id": "74f5b0cb", "metadata": {}, "outputs": [], "source": [ "tspan = ('2025-02-28','2025-03-30')\n", "\n", "results = earthaccess.search_data(\n", " short_name=prods,\n", " polygon=roi,\n", " temporal=tspan,\n", ")\n", "\n", "gdf = results_to_gdf(results)\n", "\n", "print(f'{gdf.shape[0]} granules total')\n", "\n", "pace_gdf = gdf[gdf['native-id'].str.contains('OCI')]\n", "emit_gdf = gdf[gdf['native-id'].str.contains('EMIT')]\n", "print(f' PACE OCI Granules: {pace_gdf.shape[0]} \\n EMIT Granules: {emit_gdf.shape[0]}')\n", "\n", "pace_gdf['intersects'] = pace_gdf.intersects(emit_gdf.union_all())\n", "\n", "pace_gdf = pace_gdf[pace_gdf['intersects'] == True]\n", "pace_gdf, emit_gdf = concurrent_match(pace_gdf,emit_gdf, col_name='datetime_obj',time_delta=timedelta(hours=6))\n", "print(f'After concurrent match:\\n PACE OCI Granules: {pace_gdf.shape[0]} \\n EMIT Granules: {emit_gdf.shape[0]}')" ] }, { "cell_type": "markdown", "id": "6a60510f", "metadata": {}, "source": [ "We have an even smaller number of colocated granules now, so let's again visualize the EMIT and PACE data." ] }, { "cell_type": "code", "execution_count": null, "id": "0c7ecae4", "metadata": {}, "outputs": [], "source": [ "plot_browse_images(emit_gdf)\n", "\n", "# PACE browse images coming soon!\n", "#plot_browse_images(pace_gdf)" ] }, { "cell_type": "markdown", "id": "e9c38db4", "metadata": {}, "source": [ "The EMIT data from March 19th, 2025 looks clearest, so let's create another `folium` map to visualize the PACE and EMIT overlaps." ] }, { "cell_type": "code", "execution_count": null, "id": "7fcc92cc", "metadata": {}, "outputs": [], "source": [ "fig = Figure(width=\"750px\", height=\"375px\")\n", "map1 = folium.Map(tiles='https://mt1.google.com/vt/lyrs=y&x={x}&y={y}&z={z}&hl=en', attr='Google')\n", "fig.add_child(map1)\n", "\n", "pace_gdf.drop(columns=['datetime_obj']).explore(\n", " \"native-id\",\n", " categorical=True,\n", " tooltip=[\n", " \"native-id\",\n", " \"BeginningDateTime\",\n", " \"CloudCover\",\n", " ],\n", " popup=True,\n", " style_kwds=dict(fillOpacity=0.1, width=2),\n", " name=\"PACE OCI\",\n", " m=map1,\n", " legend=False\n", ")\n", "\n", "emit_gdf.drop(columns=['datetime_obj']).explore(\n", " \"native-id\",\n", " categorical=True,\n", " tooltip=[\n", " \"native-id\",\n", " \"BeginningDateTime\",\n", " \"CloudCover\",\n", " ],\n", " popup=True,\n", " style_kwds=dict(fillOpacity=0.1, width=2),\n", " name=\"EMIT\",\n", " m=map1,\n", " legend=False\n", ")\n", "\n", "folium.GeoJson(bbox,\n", " name='bounding_box',\n", " ).add_to(map1)\n", "\n", "x1, y1, x2, y2 = gdf.union_all().bounds\n", "map1.fit_bounds(bounds=((y1, x1), (y2, x2)))\n", "map1.add_child(folium.LayerControl())\n", "display(fig)" ] }, { "cell_type": "markdown", "id": "008109a5", "metadata": {}, "source": [ "The EMIT granules from 03/19/2025 are still in a perfectly acceptable position relative to the OCI swath, so again we'll extract those granules from their respective `gdf`s and combine the two lists into a final granule list. Note that we'll need two EMIT granules to cover the park footprint this time, as well. " ] }, { "cell_type": "code", "execution_count": null, "id": "fcfb3ae1", "metadata": {}, "outputs": [], "source": [ "good_pace_gran = pace_gdf[pace_gdf['native-id'].str.contains(\"20250319\")]\n", "\n", "good_idxs = [3,4]\n", "good_emit_grans = emit_gdf[emit_gdf.index.isin(good_idxs)]\n", "keep_granules = good_pace_gran.index.to_list()+good_emit_grans.index.to_list()\n", "keep_granules.sort()\n", "\n", "postfire_results = [result for i, result in enumerate(results) if i in keep_granules]\n", "\n", "# Combine pre- and post-fire results into one list\n", "granules = prefire_results+postfire_results" ] }, { "cell_type": "markdown", "id": "ab8f9d75", "metadata": {}, "source": [ "## 4. Making the most of PACE's temporal coverage \n", "\n", "You'll notice that the dates we chose for our pre- and post-fire imagery, 11/14/2024 and 03/19/2025, have a span of several months between them that we are not currently capturing in our analysis. Although EMIT's temporal revisit is a limiting factor, PACE does see this area every day and can thus show us how the fires evolved over the days between our before and after imagery, adding some temporal context to EMIT's sharp spatial context." ] }, { "cell_type": "code", "execution_count": null, "id": "6da60f22", "metadata": {}, "outputs": [], "source": [ "results = earthaccess.search_data(\n", " short_name=[\"PACE_OCI_L2_SFREFL\"],\n", " polygon=roi,\n", " temporal=('2024-11-15','2025-03-18'),\n", ")\n", "\n", "print(f\"Number of PACE OCI granules: {len(results)}\")" ] }, { "cell_type": "markdown", "id": "9918fd3f", "metadata": {}, "source": [ "Remember that PACE overpasses occur once, and sometimes twice, a day in this region, giving a total of 172 results. That's a lot of data, much of which we know will be covered in clouds. We could include the `cloud_cover` search parameter in the `earthaccess.search_data()` call, but PACE's swath is very wide, and it is entirely possible that a granule with high cloud cover overall would be clear over our ROI. Some days may also have our ROI covered in smoke, since we know we're looking at fires in this tutorial series. \n", "\n", "The same browse imagery process as above would help visually sort through these granules, but for this volume of data, sometimes it can be useful (and more computer memory-friendly) to use tools like Worldview for quick searches. Worldview allows you create gifs of sequences of data, like the one at [this link](https://worldview.earthdata.nasa.gov/?v=135.69330009227843,-41.034094576269695,148.3782505015899,-33.93551410994782&as=2024-11-15-T00%3A00%3A00Z&ae=2025-03-19-T00%3A00%3A00Z&l=Reference_Labels_15m(hidden),Reference_Features_15m(hidden),Coastlines_15m,OCI_PACE_True_Color,VIIRS_NOAA21_CorrectedReflectance_TrueColor(hidden),VIIRS_NOAA20_CorrectedReflectance_TrueColor(hidden),VIIRS_SNPP_CorrectedReflectance_TrueColor(hidden),MODIS_Aqua_CorrectedReflectance_TrueColor(hidden),MODIS_Terra_CorrectedReflectance_TrueColor(hidden)&lg=true&av=1.5&ab=on&t=2025-03-19-T00%3A00%3A00Z) which we have used to isolate several good data days for PACE OCI imagery between 11/14/2024 and 03/19/2025. \n", "\n", "We can download these time series images to use in conjunction with our pre- and post-fire data from the cells above." ] }, { "cell_type": "code", "execution_count": null, "id": "5bcca3d1", "metadata": {}, "outputs": [], "source": [ "pace_ts_gdf = results_to_gdf(results)\n", "\n", "timestamps = [\"20241215T033822\", \"20250101T033916\", \"20250118T033938\", \"20250122T041511\", \n", " \"20250205T040925\", \"20250211T042542\", \"20250212T032254\",\n", " \"20250224T035613\", \"20250301T033714\", \"20250304T034526\",\n", " \"20250305T042058\", \"20250307T035339\", \"20250313T041003\",]\n", "\n", "pace_ts_idxs = [pace_ts_gdf[pace_ts_gdf['native-id'].str.contains(i)].index.to_list()[0] for i in timestamps]\n", "ts_results = [result for i, result in enumerate(results) if i in pace_ts_idxs]" ] }, { "cell_type": "markdown", "id": "4bb77c23", "metadata": {}, "source": [ "## 5. Downloading data \n", "\n", "
\n", "\n", "Note that the total size of the following downloads are 7.7 GB and 8.5 GB, respectively. If you are working in the cloud, we recommend using the cloud workflow included in 02_Process_PACE_EMIT_Data.\n", "\n", "
\n", "\n", "### Pre- and post-fire granules\n", "\n", "As a final step, we'll use the lists we just created to download (or stream if you are running this on a cloud-based compute platform) the data we want. \n", "\n", "If we inspected the EMIT elements in the `granules` list, we would see that the `Data` key is a list of three links, not just one as in the PACE datasets. Three links mean there are three datasets, which exist because the `EMITL2ARFL` collection contains separate files for the reflectances, uncertainties, and mask information. The next two tutorials in this series only require the reflectances and quality masks, so we will filter out the uncertainty files from the final download list. " ] }, { "cell_type": "code", "execution_count": null, "id": "62b4c6d2", "metadata": {}, "outputs": [], "source": [ "urls = []\n", "for i in range(len(granules)):\n", " links = [l for l in granules[i].data_links(access=\"external\") if \"UNCERT\" not in l]\n", " urls.extend(links)\n", "\n", "urls" ] }, { "cell_type": "markdown", "id": "ac90962f", "metadata": {}, "source": [ "Now that we have all the links for the data we want, we can either stream or download the files, depending on user preference. If you intend to download the files and work locally, be sure to comment out the second line (with `earthaccess.open()`). Alternatively, if you are going to stream the data, comment out the first line (with `earthaccess.download()`)." ] }, { "cell_type": "code", "execution_count": null, "id": "e78018ff", "metadata": {}, "outputs": [], "source": [ "paths = earthaccess.download(urls, local_path=\"data\")\n", "#paths = earthaccess.open(urls)" ] }, { "cell_type": "markdown", "id": "6e0300a3", "metadata": {}, "source": [ "markdown### Time series data\n", "\n", "Next, we'll download (or stream) the time series data we found. We don't need to extract specific URLs from the dataset list as we did above because each PACE granule only has one link. These are downloaded to the same `data` directory as the pre- and post-fire granules, since `02_Process_PACE_EMIT_Data` expects all PACE files together so it can distinguish the pre-/post-fire granules from the time series by position once sorted." ] }, { "cell_type": "code", "execution_count": null, "id": "a6e51ff5", "metadata": {}, "outputs": [], "source": [ "paths = earthaccess.download(ts_results, local_path=\"data\")\n", "#paths = earthaccess.open(ts_results)" ] }, { "cell_type": "markdown", "id": "92acdb03", "metadata": {}, "source": [ "
\n", "\n", "You have completed the first tutorial in this PACE+EMIT series. We suggest proceeding to the second notebook, `02_Process_PACE_EMIT_Data`, to learn how to open the data, do some processing, and align the grids of the two instrument datasets you just downloaded. \n", "\n", "
\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.14" } }, "nbformat": 4, "nbformat_minor": 5 }