NASA SMD FY2024 Science Data Repository Metrics

One of 11 ESD facilities managing ingest, archival, and distribution of Earth science data by domain. Examples: LP DAAC (land processes, USGS), GES DISC (atmosphere, GSFC), NSIDC DAAC (cryosphere), ASF DAAC (SAR/NISAR, Alaska). Each DAAC is operationally independent but unified under a single EOSDIS discovery portal at Earthdata.gov.

NASA's unified Earth science data infrastructure managed at GSFC. Spans 11 DAACs + 11 SIPSs providing ingest, preservation, access, and distribution. Single discovery portal at Earthdata.gov. Current holdings: 127,665 TB growing to 400,000 TB by 2030. Serves 21M cloud + 7.4M on-prem users annually.

Persistent identifier enabling datasets to be cited in peer-reviewed publications and tracked for usage impact. Key gap: PSD on-prem has only 11% DOI coverage (1,565 of 13,620 datasets). APD cloud has only 12 of 46. Without DOIs, datasets cannot participate in citation networks or AI-driven data discovery pipelines.

Machine-readable interface for programmatic data access. Datasets without APIs cannot participate in automated workflows or AI/ML pipelines. ESD and BPS achieve near-100% API coverage across cloud holdings. Critical gap: PSD on-prem has only 640 of 13,620 datasets API-accessible (4.7%), limiting its integration with modern science platforms.

Geospatial raster format enabling HTTP range requests — partial file reads without full download. Key enabler of ESD's 228× access-to-storage ratio. Users retrieve only the spatial extent or time step needed, dramatically reducing egress costs and latency. Used alongside Zarr for multidimensional EO data across EOSDIS DAACs.

Chunked, compressed, multidimensional array format for cloud object storage (S3). Enables dimension-sliced access (e.g., a single pixel's complete time series) without reading full files. Critical for AI/ML workflows on large EO datacubes. Increasingly used alongside COG across EOSDIS DAACs as cloud-native analysis replaces bulk downloads.

ESD's flagship cloud analysis environment enabling computation co-located with data — eliminating egress costs and download latency. Designed for cross-mission workflows (e.g., combining GEDI + ICESat-2 + NISAR for biomass estimation). Key driver of ESD's 21M cloud user base and 228× access-to-storage ratio.

NASA HEC facility at GSFC housing the Discover supercomputer. HPD uniquely copies 57% of its data (1,800 TB) to NCCS for HEC access — a "bring data to computation" strategy. Also supports GMAO weather/climate modeling, where GES DISC observational data feeds operational model runs and outputs are archived back to EOSDIS.

NASA ESD program purchasing commercial satellite data (Planet, Maxar, Capella, ICEYE, GHGSat, Satellogic, Tomorrow.io) via a $476M IDIQ contract. Vendor data evaluated by PI teams (scientific utility across all 6 ESD R&A focus areas) and SME teams (radiometric/geometric quality) before sustained purchase decisions. On-ramp #4 evaluated PlanetiQ and Umbra.

ESD's 11 PI-operated processing facilities producing higher-level data products from raw mission telemetry before submission to DAACs. Examples: MODAPS (MODIS/VIIRS processing), OBDAAC (ocean biology), LANCE (near-real-time products for FIRMS/VIIRS fire alerts). Distinct from DAACs, which focus on preservation and user access rather than production.

Advanced computational resources provided through NASA Advanced Supercomputing (NAS) at Ames (Pleiades) and NCCS at GSFC (Discover). Most SMD divisions provide limited adjacent compute resources. HPD is the only division with a systematic data-co-location-with-HEC strategy (57% of data at NCCS). APD and PSD route HEC access via ROSES proposal allocations.

ESD tile-based satellite imagery service via OGC WMTS/WMS protocols, powering NASA Worldview and third-party GIS applications. Accounts for 20M of ESD's 28.4M unique users per year — making it the largest single ESD access pathway. Distinct from Earthdata.gov's 7.4–8.4M download-oriented users. Critical for rapid EO data browsing and situational awareness.

AWS program waiving data egress charges for qualifying public datasets. APD (us-east-1) and HPD store data under ODR, enabling free access for all researchers. ESD stores in standard AWS us-west-2 (not ODR), so egress costs apply — a friction point explicitly noted in the report. ESD's 649 PB/yr egress at AWS commercial rates would represent substantial costs without institutional arrangements.

NASA SMD's omnibus competitive grant solicitation, published annually. Used by CSDA to recruit PI evaluation teams (e.g., ROSES-2023 A.51 for PlanetiQ and Umbra, 70-day proposal window). Also the mechanism allocating NAS HEC resources to researchers with selected proposals. New program elements for commercial data evaluation appear as ROSES amendments.

BPS-managed cloud platform hosting analysis tools — Multi-Study Data Visualization, Environmental Data Application, and RadLab. Provides CPU compute adjacent to BPS S3 data. A model for other divisions considering how to provide analysis infrastructure without requiring users to independently manage cloud architecture. GeneLab workflows also run on HEC via SMCE coordination.

Formal plan specifying how a division or mission manages data through its lifecycle — from creation through long-term archiving. Three divisions (ESD, HPD, PSD) have division-level standardized DMPs. APD and BPS use repository- or mission-specific approaches — trading cross-mission consistency for flexibility. The report flags inconsistent DMP coverage as a cross-division interoperability gap.

HPD's unified metadata and discovery layer federating SDAC and SPDF into a single search interface. Provides standardized high-level metadata across both heliophysics repositories. HelioCloud extends HDRL into the cloud with GPU and CPU compute, enabling scalable heliophysics analysis without local data download. HPD is the only division providing GPU cloud compute adjacent to data.

PSD's federated archive: 6 science discipline nodes (Atmospheres, Cartography, Geosciences, Imaging, Rings, Small Bodies) plus NAIF ancillary data facility. Each node independently curates and serves its domain. Largest dataset count in SMD (27,460) due to per-mission granular archiving. Archives all NASA planetary mission data from Voyager and Pioneer through Perseverance and New Horizons.

SMD office responsible for making NASA science data accessible, preserved, and ready for future discovery. Manages cross-division data policy, DOI/API/DMP standards, cloud transition strategy, and the annual repository metrics reporting. Kevin Murphy (CSDO) authored this report; Chelle Gentemann chairs the Steering Team. Also governs the CSDA commercial data acquisition policy framework.

NASA Goddard Space Flight Center's internal AI assistant platform. Used in preparing this FY2024 report to combine, clarify, and summarize division-contributed metrics. Disclosed transparently in the document's AI Usage Disclosure per agency reporting standards (first NASA SP-series report to include such a disclosure). Content was reviewed and edited by the human author team led by Kevin Murphy.

NASA's next-generation publication discovery and search platform, planned to succeed ADS (Astrophysics Data System) agency-wide. APD, HPD, and PSD identify Sci-X as their primary publication discovery mechanism. Aims to increase interlinking between repository data DOIs and related publications, enabling provenance tracking and AI-assisted cross-mission literature mining.