Climate Record Platform
An observational climate data platform on public NOAA daily station records: a quality-controlled pipeline, a dimensional warehouse, automated tests, and a multi-chart explorer for long-record stations across the United States.
Problem & Outcome
Public daily weather archives are large, station by station, and carry quality flags. Heating and cooling totals, extremes, and freeze-season summaries need a pipeline that keeps those flags and can be run again.
Outcome: NOAA daily files are cleaned, checked, and modeled as a star schema with summaries. dbt tests the models. The live explorer on this page reads those summaries for long-record stations nationwide: degree-days, extremes, freeze season, climate series, completeness, ranks, and a station map.
Live Explorer
The charts read one station’s summaries at a time. Filter by state, then station and years.
Heating & cooling degree-days
Base 18 °C · quality-controlled temperatures only
Hot days & freeze days by year
Days with high ≥ 32 °C and low ≤ 0 °C in the selected range
Wet days ≥ 25.4 mm by year
Days with daily precipitation ≥ 1 inch (25.4 mm)
Annual max high & min low
Hottest daily maximum and coldest daily minimum (°C) each year
Max daily precipitation by year
Largest single-day total (mm) in each year
Precipitation
Monthly totals or yearly sums
Average max & min temperature
Average daily max and min · °C
How complete is the daily high-temp record?
For each year: usable daily maximum temperatures ÷ days in the year (after quality control). Low % means averages/extremes for that year are less trustworthy.
Freeze days & growing-season length
Freeze day = daily low ≤ 0 °C
Hottest / coldest / wettest years (selected station)
For the year range selected above
| Rank | Hottest years by max high °C |
Coldest years by min low °C |
Wettest years by max daily rain mm |
|---|---|---|---|
| Load a station to see ranks… | |||
Network snapshot — hot days (latest year available)
All long-record sample stations · one year · top 40 by hot-day count. Independent of the station dropdown above.
Station map
Hover for day count · click to select. Color follows the metric below.
Architecture
Python loads the NOAA files, applies quality control, and calculates the metrics. The checked daily data is stored as a Parquet star schema. dbt builds the summaries in DuckDB and tests them. The charts on this page read the exported JSON.
flowchart TB A["NOAA GHCNd daily files"] --> B["Cleaned rows and quality control"] B --> C["Parquet star schema"] C --> D["dbt and DuckDB summaries"] D --> E["JSON on this explorer"]
Hosting & Operations
This page is on Cloudflare Pages. The station charts read published files stored on Cloudflare R2. The warehouse stays private. Only those exported files are on the public site.
Tech Stack
| Layer | Choice | Why |
|---|---|---|
| Source | NOAA GHCNd daily bulk | Public, documented daily station archive |
| Pipeline / QC | Python, Pandas, Parquet | Reproducible transforms; quality flags retained |
| SQL layer | dbt + DuckDB | Models and data tests |
| Analytics model | Star schema + marts | Flexible warehouse; fast product summaries |
| Web demo | Static HTML + Chart.js + Leaflet | Charts on this page from the published JSON. |
Methods
Loading summary…
- Scope: long-record USW/USC stations nationwide (all states + D.C.) with ≥50-year overlapping TMAX + TMIN + PRCP inventory coverage — the same rules used for the earlier SC/NC/GA regional sample.
- Quality control: missing values, NOAA quality flags, temperature range gates, and same-day high < low pairs are excluded from gold.
- Degree-days: daily mean ≈ (maximum + minimum) / 2; heating/cooling degree-days vs 18 °C; summed by month.
- Extremes: days with maximum ≥ 32 °C / 35 °C; minimum ≤ 0 °C; precipitation ≥ 25.4 mm.
- Freeze season: freeze day if minimum ≤ 0 °C; growing-season length between last spring freeze and first fall freeze when both exist.
Takeaways
- About 6,265 long-record stations, with quality checks on the daily observations
- A star schema, with tested summaries for degree-days, extremes, and freeze season
- A live explorer on this page, reading published JSON one station at a time