Data
High Resolution Electricity Access (HREA) indicator data are provided as rasters publicly for free. These data, which are derived from statistical processing of nightly satellite observations of the earth at night, are stored as Cloud Optimized GeoTIFFs (COGs) with associated Spatio-Temporal Asset Catalogue (STAC) metadata. The data are available from multiple sources and are easily obtained either through a web browser or command line interface. This page describes the data and how to obtain them.
HREA Data Overview
We provide HREA data for 115 countries that were not identified as fully electrified by the World Bank's Global Electrification Database by 2015. Data are currently available at the annual level from 2013 to 2020 and rely upon Meta's High Resolution Settlement Layer (HRSL) or the European Commission's Global Human Settlement Layer (GHSL).
Each country has a constant settlement population raster for the time period based on the underlying settlement source. Visible radiance composites and three HREA metrics are provided annually. HREA data based on HRSL are at the 1 arcsec resolution; those based on GHSL are at the 7.5 arcsec resolution. HREA estimates are only available for populated places; while they are calculated at the 15 arcsec level, they are provided at the original settlement layer resolution to represent the settlement distribution. Because VIIRS-DNB radiance values are at the 15 arcsec resolution, composites are provided in the same resolution as the 15 arcsec grid, which includes both settlements and non-settlements. COGs are stored as 32-bit floats with missing values corresponding to -999999. All COGs use the World Geodetic System unprojected longitude-latitude coordinate reference system (WGS 84, EPSG:4326). Each COG asset has a corresponding item json, and searchable STAC collections and catalogs organize the data following best practices.
On AWS, the latest HREA data (version 1.1) are stored in a top-level directory named HREAv1.1_COGs. Beneath that, files are separated into country-level directories using ISO ALPHA-3 country codes. The top-level directory contains a primary STAC catalog that lists the secondary country-level STAC catalogs found in the subdirectories. Each COG has an associated STAC item json with metadata about the file. Thus, by reading and navigating through the STAC catalogs, one can obtain the URLs for specific files according to the researcher's geographic and temporal interest. For an example of how to do this, please see the GitHub page on the topic.
Some countries are broken into multiple rasters either for the purpose of reducing file sizes or in the case of Fiji because it crosses the antimeridian. Countries and their constituent files may thus be broken up with directional indicators or by their first level administrative unit (ADM1) according to GADM. There are three countries in where this occurs: Brazil, India, and Fiji. Brazil is split into quadrants, where NW indicates northwest, NE indicates northeast, SW indicates southwest, and SE indicates southeast. Fiji is split into the part in the western hemisphere (W) and that in the eastern hemisphere (E). India is split into 36 states and territories, where state abbreviations follow the 2 letter abbreviations from the 2019-11-22 version of ISO 3166. For a full list of the country and sub-country codes used and their descriptions, please see the lookup table on GitHub here.
The table below provides a brief description of available annual HREA products. See the supplemental information for more detailed information on the VIIRS quality flags and Earth Observation Group (EOG) stable lights method as well as the HREA methodology including the grid creation. Value ranges in the table are approximate minima and maxima of non-missing data.
Description | File Stem | Resolution | Coverage | Value Range |
---|---|---|---|---|
Mean annual natural logged visible radiance (in nW/cm2/sr) + 2.5 following EOG procedure for dropping low quality observations | rade9lnmu | 15 arcsec | All cells | [0, 1] |
Mean z-score (standardized residuals) | set_zscore | 1 or 7.5 arcsec | Settlement cells | [-29, 218] |
Light score (transformation of mean z-score) | set_lightscore | 1 or 7.5 arcsec | Settlement cells | [0, 1] |
Proportion of nights a settlement is brighter than uninhabited areas | set_prplit | 1 or 7.5 arcsec | Settlement cells | [0, 1] |
Directory Structure and Filename Conventions
This section describes how files are organized relying upon the column variables in the lookup table described on the GitHub repository. The following list describes the meaning of the variables used below:
- country_iso3: ISO ALPHA-3 country code
- cog_id: specific identifier for country or sub-country boundary
- year: year in YYYY format
HREA rasters and metadata are stored in sub-directories below the top-level HREAv1.1_COGs folder, in which 115 directories exist named according to their country_iso3 country code. Inside each country_iso3 folder are directories for each cog_id belonging to that country. In most cases, there is only one cog_id which is equivalent to the country_iso3; however, some countries are split into multiple parts out of necessity. In such cases, the cog_id is formatted as country_iso3-X, where X may correspond to a compass direction indicator (as in Brazil and Fiji) or a two-letter state code as in India. For each cog_id, there exists a constant settlement population raster file HREAv1.1_COGs/country_iso3/cog_id/cog_id_set_pop.tif, as well as a STAC item with the same name but instead ending with .json. In addition, for each year in 2013–2020, there exist the following GeoTIFFs, which also have corresponding STAC items with names that substitute .json for the .tif extension:
- HREAv1.1_COGs/country_iso3/country_iso3/cog_id_rade9lnmu_year.tif: Nighttime light annual composite for all cells in a locale.
- HREAv1.1_COGs/country_iso3/country_iso3/cog_id_set_zscore_year.tif: Statistically estimated brightness levels for settlements. Higher levels indicate more robust usage of outdoor lighting, which is correlated with overall energy consumption.
- HREAv1.1_COGs/country_iso3/country_iso3/cog_id_set_lightscore_year.tif: Predicted likelihood that a settlement is electrified (0 to 1).
- HREAv1.1_COGs/country_iso3/country_iso3/cog_id_set_prplit_year.tif: Proportion of nights a settlement is statistically brighter than matched uninhabited areas.
How to Access
HREA data are available for download from Amazon Web Services (AWS) as part of the Light Every Night (LEN) repository. Please see the HREA project GitHub page for more documentation on how to access the data, including how to view and search the STAC catalogs.
One can obtain HREA COGs from AWS either by direct download using the object URLs or through the AWS command line interface (CLI). Using the CLI, one can list the contents of a directory with aws s3 ls s3://globalnightlight/path/to/dir. One must also use the --no-sign-request flag with the AWS CLI. For the top-level directory, one would run:
aws s3 ls s3://globalnightlight/HREAv1.1_COGs --no-sign-request
The root HREA STAC catalog, which lists links to all country_iso3 catalogs, can be downloaded with a tools like wget, curl, or aws-cli:
wget https://globalnightlight.s3.amazonaws.com/HREAv1.1_COGs/catalog.json
curl -O https://globalnightlight.s3.amazonaws.com/HREAv1.1_COGs/catalog.json
aws s3 cp s3://globalnightlight/HREAv1.1_COGs/catalog.json . --no-sign-request
One can check the cog_ids for a specific country_iso3 by listing the contents in the folder with:
aws s3 ls s3://globalnightlight/HREAv1.1_COGs/${country_iso3} --no-sign-request
One could read a specific country_iso3's catalog to determine linked catalogs and collections:
curl https://globalnightlight.s3.amazonaws.com/HREAv1.1_COGs/${country_iso3}/catalog.json
Individual assets can be downloaded using the path structure described above. As an example, one could obtain the mean standardized residuals for Ghana settlements in 2020 with:
aws s3 cp s3://globalnightlight/HREAv1.1_COGs/GHA/GHA/set_zscore/GHA_set_zscore_2020.tif . --no-sign-request
Alternatively, one can download the contents of an entire directory for some country_iso3 to some directory /path/to/dir/ like so:
aws s3 cp s3://globalnightlight/HREAv1.1_COGs/${country_iso3}/ /path/to/dir/ --recursive --no-sign-request
You may also download the files using the dropdown menus below:
How to Cite
If you use HREA data, please cite our paper:
Brian Min, Zachary P. O'Keeffe, Babatunde Abidoye, Kwawu Mensan Gaba, Trevor Monroe, Benjamin P. Stewart, Kim Baugh, Bruno Sánchez-Andrade Nuño, “Lost in the Dark: A Survey of Energy Poverty from Space,” Joule (2024), https://doi.org/10.1016/j.joule.2024.05.001.
Notes
We use the term “country” loosely, following the conventions established by ISO 3166-1 alpha-3. We use the Database of Global Administrative Areas (GADM) to define geographic boundaries across all areas of interest. Both decisions were made to enhance accessibility and usability so that researchers may more easily obtain the data and merge them with other data sources. We make no claims about rightful territorial ownership or political sovereignty.