Radiometrically Terrain Corrected Products for Biomass Mapping

In a European Space Agency funded project to generate annual estimates of the global spatial distribution of the critical climate indicator “biomass” (CCI BIOMASS project), Earth Big Data's software was used to scale radiometric terrain correction of millions of Sentinel-1 data for Gamma Remote Sensing employing the Copernicus 30 m DEM.

Annual global acquisitions of Sentinel-1 data sets acquired in the interferometric wide-swath mode (IW) were processed from the ground range detected format (GRD) into GeoTIFFs and tiled into 1x1 degree tiles for the time series modeling of biomass. As an example, for the year 2017, before the loss of Sentinel-1B, more than 305,000 RTC products at 720 pixel per degree resolution were produced. During processing, the Cloud infrastructure was scaled up to 2,000 AWS elastic compute cloud instances with 32 GB RAM and 4 virtual CPUs. With this fleet of compute instances, a throughput of 30 TB/hour was achieved for reading data from the Alaska Satellite Facility DAAC.

The entire processing job reduced an input volume of approximately 330 TB to an output of 4 TB in about 13 hours. The AWS cloud monitoring graphs for this processing run indicates a resource use of 2,000 ec2 instances.

CPU utilization rate at about 73%

An hourly network ingestion rate of 30TB

The hourly output rate from this processing was about 400 GB. All processing was performed in the AWS region us-west-2 where the ASF DAAC is holding all Sentinel-1 on an AWS s3 bucket. As Earth Big Data spun up 2,000 instances in us-west-2, data were accessed for processing at 20-30 MB/s. 

Data transfer from the ASF bucket into the ec2 instances incurred no cost since both storage bucket and ec2 instances were co-located in the same AWS region.

As can be seen in the “Instances in EBD_GAMMA_CCI Group” graph, for a short period of time, ec2 provision dropped to about 600 instances. Earth Big Data utilizes the AWS Spot marked for non-time critical processing, resulting in savings of 50% to 75% when compared to regular on-demand AWS pricing. Even as resources might fluctuate in availability, Earth Big Data automatically scales to the available desired maximum group size (orange line) when available.

A global mosaic of the resulting median Sentinel-1 backscatter data from C-band in vv (red) and vh (green) polarizations and their ration vv/vh (blue) is depicted below

If you would like more information on this Success Story, please contact us at info@earthbigdata.com