Split the pipeline documentation by purpose so each fact has one home: - docs/pipeline-plan.md keeps the plan, checklist, tracker, and guardrails - docs/decisions.md holds open decisions and the dated decision log - docs/reviews/ holds findings and tasks: one file per filter, plus 00-cross-filter.md for findings that span filters - scripts/common/README.md holds the shared-helper rules (formerly Phase 2) - filter-calculations.md now describes calculations only Filed findings 12-22 from a consistency audit of the app, docs, and scripts. Filter 1 (Köppen-Geiger): use "Köppen" with the umlaut in all prose, labels, docstrings, help text, and checker messages (finding 21), and correct the base build's "majority" docstring (finding 22). Filter 2 (annual avg temperature): record the adopted definition in filter-calculations.md §2: equally weighted 1991-2020 monthly normals, per WMO-No. 1203 and NOAA's 2020 methodology; area-weighted county means; blank unless all 12 months exist. Code changes for this filter are still pending. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Temperature-Based Analysis
Temperature-Based Analysis is an exploratory project for studying how local climate conditions vary across the United States and, eventually, whether those conditions show any relationship with mood-based metrics.
The project currently provides the US County Climate Explorer, an interactive county-level map for viewing and filtering climate data. The mood dataset and climate-to-mood correlation analysis are not implemented yet. At this stage, the project is focused on building, validating, and presenting the climate side of the analysis.
Current Features
- Interactive Leaflet map with county boundaries and county selection
- Metric groups for climate classification, temperature, precipitation, moisture, and solar resource
- Numeric range filters and categorical filters
- County detail panel showing all available climate metrics
- Legends and in-app source descriptions
- Local CSV-based data loading with no application backend or build step
- Offline Python scripts for assembling and updating county climate data
The browser currently loads 3,221 county-level records from
data/climate-data.csv.
Available Climate Metrics
| Group | Metrics |
|---|---|
| Köppen-Geiger Classification | Predominant county climate class (covering at least 50% of the county's land and leading the runner-up by at least 5 points), or Mixed, drawn as stripes of the county's top two classes |
| Temperature & Extremes | Annual average temperature, diurnal temperature range, annual extreme temperature days, and annual 90 F+ heat-index days |
| Precipitation & Moisture | Annual precipitation, precipitation seasonality, wettest month, driest month, and summer specific humidity |
| Solar Resource | Mean daily global horizontal radiation (GHI) and clear-sky GHI reduction index |
Most long-term climate metrics use a 1991-2020 reference period. In the
checked-in CSV, the annual extreme-temperature-days metric uses 1991-2025 data
and is the average annual count of days with Tmax >= 95 F or Tmin <= 0 F.
The build script defaults its analysis end year to the latest likely complete
calendar year. Definitions and time periods are shown in the application's
Sources dialog and documented in more detail in
scripts/county_data_sources.md.
Run the Explorer
Requirements
- A modern web browser
- Python 3
- PowerShell for the included convenience script
- An internet connection for Leaflet, map tiles, and hosted fonts
No Node.js installation or frontend build is required.
From the project root, run:
.\serve.ps1
Then open:
http://localhost:8000/
To use another port:
.\serve.ps1 -Port 8080
The site must be served over HTTP because the browser loads the climate CSV and
county GeoJSON with fetch(). Opening index.html directly as a local file
will not work.
Using the Map
- Choose a metric group and metric from the control panel.
- Adjust the value range or category filter to highlight matching counties.
- Click a county to zoom to it and inspect all available metrics.
- Use Reset Country View or Clear Selection to return to the broader map.
- Open Sources for metric definitions and data provenance.
Project Structure
| Path | Purpose |
|---|---|
index.html |
Application structure and controls |
styles.css |
Layout and visual styling |
app.js |
Map rendering, filtering, county details, and source metadata |
serve.ps1 |
Local HTTP server launcher |
data/climate-data.csv |
Browser-ready county climate records |
data/metrics/ |
Per-metric county outputs, starting with koppen.csv |
data/geojson-counties-fips.json |
County geometry keyed by FIPS code |
scripts/ |
Climate-data download, aggregation, and update tools |
scripts/county_data_sources.md |
Detailed metric definitions, data provenance, and pipeline examples |
scripts/requirements_county_etl.txt |
Python dependencies for the offline data pipeline |
docs/ |
Filter calculations, the pipeline plan, decisions, per-filter reviews (docs/reviews/), and design notes |
tests/ |
Tests for the Köppen metric, the climate CSV check, NSRDB request/download wrappers, cloud-metric merging, and gridMET heat-index calculations |
Climate Data Pipeline
The checked-in browser assets are the final outputs needed to run the explorer. The scripts directory contains the larger offline workflow used to derive those outputs from sources including:
- Beck et al. Köppen-Geiger climate classification data
- NOAA NCEI Climate Normals and nClimGrid data
- gridMET humidity data
- NREL National Solar Radiation Database data
- Plotly's county GeoJSON, keyed by U.S. Census county FIPS codes
The Python tooling is configured for Python 3.11. To work on the data pipeline, create a virtual environment and install the ETL dependencies:
python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install -r scripts\requirements_county_etl.txt
Some data-generation workflows download large files or require NREL/NSRDB API credentials. Generated source datasets are intentionally excluded from Git; only the browser-ready CSV and GeoJSON are tracked.
Run script commands from the project root so their default data/... paths
resolve correctly. The pipeline is split into a base generator, source-specific
builders, and small scripts that merge the resulting metrics into
data/climate-data.csv. Most apply_*.py commands update that CSV in place by
default.
Script Inventory
| Script | Current role |
|---|---|
build_county_climate_data.py |
Builds the base app CSV from county geometry, Köppen-Geiger data, NOAA temperature/precipitation data, and optional solar inputs. Its older extremeDays output is replaced by the current absolute-threshold stage below, and its largest-share koppenZone by the Köppen stage. It writes only the base columns, so do not run it over the live CSV. |
build_county_koppen_metric.py / apply_koppen_metric_to_climate_data.py |
Builds area-weighted Köppen class shares per county in data/metrics/koppen.csv, classifies each county as predominant or Mixed, and writes koppenZone plus the two stripe-class columns for Mixed counties. |
apply_precipitation_month_metrics_to_climate_data.py |
Recomputes and merges the 1991-2020 wettest- and driest-month categories from monthly nClimGrid precipitation. |
build_county_locally_extreme_data.py |
Downloads or reads cached nClimGrid-Daily county Tmax/Tmin files, calculates county-percentile diagnostics, and calculates the app-facing absolute 95 F / 0 F day counts. |
apply_locally_extreme_metric_to_climate_data.py |
Writes absoluteExtremeDays, removes retired locally extreme/legacy fields, and selects polygon GHI with representative-point GHI as fallback. The filename is retained from the earlier pipeline. |
build_county_diurnal_temperature_range.py / apply_diurnal_temperature_range_to_climate_data.py |
Builds the 1991-2020 county mean daily Tmax-minus-Tmin artifact and merges avgDiurnalTempRangeF. |
download_gridmet_data.py |
Downloads 1991-2020 sph, rmax, and rmin NetCDF files by default. |
summarize_county_gridmet_humidity.py / apply_gridmet_humidity_metric_to_climate_data.py |
Produces county summer specific humidity and a 90 F+ Heat Index day proxy, then merges those metrics and FIPS audit fields. |
build_county_representative_points.py |
Creates interior county points used by the lightweight NSRDB workflows. |
fetch_nsrdb_representative_point_ghi.py / rebuild_nsrdb_representative_point_ghi_summary.py |
Fetches point-based NSRDB GHI or rebuilds its summary from cached responses without another API call. |
fetch_nsrdb_representative_point_cloud_metrics.py |
Fetches point-based GHI, clear-sky GHI, and cloud type, then summarizes the clear-sky GHI reduction index. |
request_nsrdb_county_polygon_archives.py |
Shared, resumable NSRDB polygon-request engine with tiling, site-count checks, pacing, and Polar fallback. |
request_nsrdb_county_polygon_ghi_archives.py / request_nsrdb_county_polygon_cloud_archives.py |
Recommended wrappers around the shared request engine, with separate GHI and cloud attributes and output paths. |
download_nsrdb_county_polygon_archives.py |
Shared state-machine downloader for completed NSRDB archive jobs. |
download_nsrdb_county_polygon_ghi_archives.py / download_nsrdb_county_polygon_cloud_archives.py |
Recommended wrappers around the shared downloader, keeping GHI and cloud archives separate. |
summarize_nsrdb_county_polygon_archives.py |
Combines county/tile GHI archives into area-weighted county summaries. |
summarize_nsrdb_county_polygon_cloud_archives.py |
Combines county/tile cloud archives into area-weighted clear-sky GHI reduction summaries. |
apply_nsrdb_cloud_metric_to_climate_data.py |
Merges the clear-sky GHI reduction index, preferring polygon summaries and falling back to representative points. |
check_climate_data.py |
Validates data/climate-data.csv, including Köppen codes and the stripe-class columns. |
common/ |
Shared helpers: county loading, the Köppen legend, and area-weighted raster shares. |
The metric-specific NSRDB request and download wrappers are the normal entry points. The shared engines remain available for custom attributes or artifact paths. Representative-point results provide a faster first pass; polygon summaries are the preferred county-area result when available.
Common local enrichment stages, after their source files have been downloaded, are:
.venv\Scripts\python.exe scripts\build_county_koppen_metric.py
.venv\Scripts\python.exe scripts\apply_koppen_metric_to_climate_data.py
.venv\Scripts\python.exe scripts\build_county_locally_extreme_data.py --skip-download
.venv\Scripts\python.exe scripts\apply_locally_extreme_metric_to_climate_data.py
.venv\Scripts\python.exe scripts\build_county_diurnal_temperature_range.py
.venv\Scripts\python.exe scripts\apply_diurnal_temperature_range_to_climate_data.py
.venv\Scripts\python.exe scripts\summarize_county_gridmet_humidity.py --years 1991-2020
.venv\Scripts\python.exe scripts\apply_gridmet_humidity_metric_to_climate_data.py
.venv\Scripts\python.exe scripts\apply_nsrdb_cloud_metric_to_climate_data.py
The order matters when rebuilding from scratch: run the Köppen apply step after
the base build, generate the locally extreme comparison and solar summaries
before running their apply step, and summarize gridMET or NSRDB downloads
before merging them. See the data-source document linked above for acquisition
commands, expected artifacts, FIPS handling, and the representative-point and
polygon NSRDB workflows. The full rebuild order is in
docs/pipeline-plan.md.
After updating the CSV, validate it with:
.venv\Scripts\python.exe scripts\check_climate_data.py
Run the current automated tests with:
python -m unittest discover -s tests
Planned Mood Analysis
The longer-term goal is to add mood-based metrics and investigate whether patterns in those metrics are associated with climate characteristics such as temperature, sunlight availability, clear-sky GHI reduction, humidity, precipitation, or extreme-weather frequency.
That phase still requires decisions about:
- How mood data will be collected or sourced
- Geographic and temporal granularity
- Privacy, consent, and aggregation requirements
- Confounding variables and missing-data handling
- Appropriate statistical and visualization methods
No mood records, mood visualizations, or correlation results are currently part of the application. Any future relationship found by the project should be treated as an association to investigate, not evidence that climate alone causes changes in mood.
Current Limitations
- The application presents long-term county summaries rather than live weather.
- Climate values aggregate conditions within county boundaries and do not represent every location inside a county.
- Source datasets use different methods and, in some cases, different time periods.
- Some solar metrics use representative-point values where county polygon summaries are unavailable.
- The current interface is an exploratory visualization, not a completed climate-and-mood research analysis.