Data Formats

This vignette illustrates cloud-native access of across a range of common formats, including netcdf, Zarr, hdf5, hdf4, and COGs.

library(earthdatalogin)
edl_unset_token()

(Right now these are somewhat generic examples, not necessarily based on NASA’s EarthData sources and therefore not requiring authentication, but the workflow should be identical after calling edl_set_token())

library(stars)
library(terra)

netcdf, 1 GB file

Cloud-read data from a public AWS S3 bucket, over https:

url <- paste0("https://era5-pds.s3.amazonaws.com",
              "/1979/01/data/air_pressure_at_mean_sea_level.nc")
r <- terra::rast(url, vsi=TRUE)
r
#> class       : SpatRaster 
#> dimensions  : 721, 1440, 744  (nrow, ncol, nlyr)
#> resolution  : 1, 1  (x, y)
#> extent      : 0, 1440, 0, 721  (xmin, xmax, ymin, ymax)
#> coord. ref. :  
#> source      : air_pressure_at_mean_sea_level.nc 
#> names       : air_p~vel_1, air_p~vel_2, air_p~vel_3, air_p~vel_4, air_p~vel_5, air_p~vel_6, ...

Cloud read of the same data, over S3 protocol:

Sys.setenv("AWS_NO_SIGN_REQUEST"=TRUE)
s3uri <- "s3://era5-pds/1979/01/data/air_pressure_at_mean_sea_level.nc"
r <- terra::rast(s3uri)
Sys.unsetenv("AWS_NO_SIGN_REQUEST")

Zarr, 31 TB

Cloud read of a 31 Terrabyte Zarr archive (S3, public https)

url <- "https://mur-sst.s3.us-west-2.amazonaws.com/zarr-v1"
prefixes <- 'ZARR:\"/vsicurl/'
slice <- '\":/analysed_sst:0"'
addr <- paste0(prefixes, url, slice)
y = terra::rast(addr)
library(tmap)
data(World)

india <-st_bbox( World$geometry[World$name == "India"] )
st_as_stars(y) |>
  st_crop(india) |>
  plot()
plot of chunk zarr_sst
plot of chunk zarr_sst