The Dunleavy Organization

Data · Analytics · Digital Delivery

Climate Record Platform

An observational climate data platform on public NOAA daily station records: a quality-controlled pipeline, a dimensional warehouse, automated tests, and a multi-chart explorer for long-record stations across the United States.

Role
Sole developer
Timeline
2026
Stations
~6,265
Daily observations
~514 million, quality-checked
Type
Data platform
Status
Live demo

Problem & Outcome

Public daily weather archives are large, station by station, and carry quality flags. Heating and cooling totals, extremes, and freeze-season summaries need a pipeline that keeps those flags and can be run again.

Outcome: NOAA daily files are cleaned, checked, and modeled as a star schema with summaries. dbt tests the models. The live explorer on this page reads those summaries for long-record stations nationwide: degree-days, extremes, freeze season, climate series, completeness, ranks, and a station map.

Live Explorer

The charts read one station’s summaries at a time. Filter by state, then station and years.

Heating & cooling degree-days

Base 18 °C · quality-controlled temperatures only

Hot days & freeze days by year

Days with high ≥ 32 °C and low ≤ 0 °C in the selected range

Wet days ≥ 25.4 mm by year

Days with daily precipitation ≥ 1 inch (25.4 mm)

Annual max high & min low

Hottest daily maximum and coldest daily minimum (°C) each year

Max daily precipitation by year

Largest single-day total (mm) in each year

Precipitation

Monthly totals or yearly sums

Average max & min temperature

Average daily max and min · °C

How complete is the daily high-temp record?

For each year: usable daily maximum temperatures ÷ days in the year (after quality control). Low % means averages/extremes for that year are less trustworthy.

Freeze days & growing-season length

Freeze day = daily low ≤ 0 °C

Hottest / coldest / wettest years (selected station)

For the year range selected above

Rank Hottest years
by max high °C
Coldest years
by min low °C
Wettest years
by max daily rain mm
Load a station to see ranks…

Network snapshot — hot days (latest year available)

All long-record sample stations · one year · top 40 by hot-day count. Independent of the station dropdown above.

Station map

Hover for day count · click to select. Color follows the metric below.

Architecture

Python loads the NOAA files, applies quality control, and calculates the metrics. The checked daily data is stored as a Parquet star schema. dbt builds the summaries in DuckDB and tests them. The charts on this page read the exported JSON.

Hosting & Operations

This page is on Cloudflare Pages. The station charts read published files stored on Cloudflare R2. The warehouse stays private. Only those exported files are on the public site.

Cloudflare Pages Cloudflare R2 Medallion pipeline Star schema + marts Static explorer JSON

Tech Stack

Layer Choice Why
Source NOAA GHCNd daily bulk Public, documented daily station archive
Pipeline / QC Python, Pandas, Parquet Reproducible transforms; quality flags retained
SQL layer dbt + DuckDB Models and data tests
Analytics model Star schema + marts Flexible warehouse; fast product summaries
Web demo Static HTML + Chart.js + Leaflet Charts on this page from the published JSON.

Methods

Loading summary…

  • Scope: long-record USW/USC stations nationwide (all states + D.C.) with ≥50-year overlapping TMAX + TMIN + PRCP inventory coverage — the same rules used for the earlier SC/NC/GA regional sample.
  • Quality control: missing values, NOAA quality flags, temperature range gates, and same-day high < low pairs are excluded from gold.
  • Degree-days: daily mean ≈ (maximum + minimum) / 2; heating/cooling degree-days vs 18 °C; summed by month.
  • Extremes: days with maximum ≥ 32 °C / 35 °C; minimum ≤ 0 °C; precipitation ≥ 25.4 mm.
  • Freeze season: freeze day if minimum ≤ 0 °C; growing-season length between last spring freeze and first fall freeze when both exist.

Takeaways

  • About 6,265 long-record stations, with quality checks on the daily observations
  • A star schema, with tested summaries for degree-days, extremes, and freeze season
  • A live explorer on this page, reading published JSON one station at a time
Want a data platform like this?