Knou/Climate-Mood-Analysis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

README.md has been updated to reflect project changes.

Browse Source
This commit is contained in:
Knou
2026-09-11 16:30:15 -04:00
parent 4e2d878e5b
commit 92fbbfb2e9
1 changed files with 67 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+67 -11
View File
@@ -33,11 +33,13 @@ The browser currently loads 3,221 county-level records from
| Precipitation & Moisture | Annual precipitation, precipitation seasonality, wettest month, driest month, and summer specific humidity |
| Solar Resource | Mean daily global horizontal radiation (GHI) and clear-sky GHI reduction index |
Most long-term climate metrics use a 1991-2020 reference period. The annual
extreme-temperature-days metric currently uses 1991-2025 data and includes days
meeting either the extreme heat or extreme cold threshold. Definitions and time
periods are shown in the application's **Sources** dialog and documented in more
detail in [`scripts/county_data_sources.md`](scripts/county_data_sources.md).
Most long-term climate metrics use a 1991-2020 reference period. In the
checked-in CSV, the annual extreme-temperature-days metric uses 1991-2025 data
and is the average annual count of days with `Tmax >= 95 F` or `Tmin <= 0 F`.
The build script defaults its analysis end year to the latest likely complete
calendar year. Definitions and time periods are shown in the application's
**Sources** dialog and documented in more detail in
[`scripts/county_data_sources.md`](scripts/county_data_sources.md).
## Run the Explorer
@@ -92,7 +94,9 @@ will not work.
| `data/climate-data.csv` | Browser-ready county climate records |
| `data/geojson-counties-fips.json` | County geometry keyed by FIPS code |
| `scripts/` | Climate-data download, aggregation, and update tools |
| `tests/` | Tests for the NSRDB polygon request and download workflow |
| `scripts/county_data_sources.md` | Detailed metric definitions, data provenance, and pipeline examples |
| `scripts/requirements_county_etl.txt` | Python dependencies for the offline data pipeline |
| `tests/` | Tests for NSRDB request/download wrappers, cloud-metric merging, and gridMET heat-index calculations |
## Climate Data Pipeline
@@ -104,10 +108,10 @@ outputs from sources including:
- NOAA NCEI Climate Normals and nClimGrid data
- gridMET humidity data
- NREL National Solar Radiation Database data
- U.S. Census Bureau county geometry
- Plotly's county GeoJSON, keyed by U.S. Census county FIPS codes
To work on the Python data pipeline, create a virtual environment and install
the ETL dependencies:
The Python tooling is configured for Python 3.11. To work on the data pipeline,
create a virtual environment and install the ETL dependencies:
```powershell
python -m venv .venv
@@ -119,6 +123,58 @@ Some data-generation workflows download large files or require NREL/NSRDB API
credentials. Generated source datasets are intentionally excluded from Git;
only the browser-ready CSV and GeoJSON are tracked.
Run script commands from the project root so their default `data/...` paths
resolve correctly. The pipeline is split into a base generator, source-specific
builders, and small scripts that merge the resulting metrics into
`data/climate-data.csv`. Most `apply_*.py` commands update that CSV in place by
default.
### Script Inventory
| Script | Current role |
| --- | --- |
| `build_county_climate_data.py` | Builds the base app CSV from county geometry, Koppen-Geiger data, NOAA temperature/precipitation data, and optional solar inputs. Its older `extremeDays` output is replaced by the current absolute-threshold stage below. |
| `apply_precipitation_month_metrics_to_climate_data.py` | Recomputes and merges the 1991-2020 wettest- and driest-month categories from monthly nClimGrid precipitation. |
| `build_county_locally_extreme_data.py` | Downloads or reads cached nClimGrid-Daily county Tmax/Tmin files, calculates county-percentile diagnostics, and calculates the app-facing absolute 95 F / 0 F day counts. |
| `apply_locally_extreme_metric_to_climate_data.py` | Writes `absoluteExtremeDays`, removes retired locally extreme/legacy fields, and selects polygon GHI with representative-point GHI as fallback. The filename is retained from the earlier pipeline. |
| `build_county_diurnal_temperature_range.py` / `apply_diurnal_temperature_range_to_climate_data.py` | Builds the 1991-2020 county mean daily Tmax-minus-Tmin artifact and merges `avgDiurnalTempRangeF`. |
| `download_gridmet_data.py` | Downloads 1991-2020 `sph`, `rmax`, and `rmin` NetCDF files by default. |
| `summarize_county_gridmet_humidity.py` / `apply_gridmet_humidity_metric_to_climate_data.py` | Produces county summer specific humidity and a 90 F+ Heat Index day proxy, then merges those metrics and FIPS audit fields. |
| `build_county_representative_points.py` | Creates interior county points used by the lightweight NSRDB workflows. |
| `fetch_nsrdb_representative_point_ghi.py` / `rebuild_nsrdb_representative_point_ghi_summary.py` | Fetches point-based NSRDB GHI or rebuilds its summary from cached responses without another API call. |
| `fetch_nsrdb_representative_point_cloud_metrics.py` | Fetches point-based GHI, clear-sky GHI, and cloud type, then summarizes the clear-sky GHI reduction index. |
| `request_nsrdb_county_polygon_archives.py` | Shared, resumable NSRDB polygon-request engine with tiling, site-count checks, pacing, and Polar fallback. |
| `request_nsrdb_county_polygon_ghi_archives.py` / `request_nsrdb_county_polygon_cloud_archives.py` | Recommended wrappers around the shared request engine, with separate GHI and cloud attributes and output paths. |
| `download_nsrdb_county_polygon_archives.py` | Shared state-machine downloader for completed NSRDB archive jobs. |
| `download_nsrdb_county_polygon_ghi_archives.py` / `download_nsrdb_county_polygon_cloud_archives.py` | Recommended wrappers around the shared downloader, keeping GHI and cloud archives separate. |
| `summarize_nsrdb_county_polygon_archives.py` | Combines county/tile GHI archives into area-weighted county summaries. |
| `summarize_nsrdb_county_polygon_cloud_archives.py` | Combines county/tile cloud archives into area-weighted clear-sky GHI reduction summaries. |
| `apply_nsrdb_cloud_metric_to_climate_data.py` | Merges the clear-sky GHI reduction index, preferring polygon summaries and falling back to representative points. |
The metric-specific NSRDB request and download wrappers are the normal entry
points. The shared engines remain available for custom attributes or artifact
paths. Representative-point results provide a faster first pass; polygon
summaries are the preferred county-area result when available.
Common local enrichment stages, after their source files have been downloaded,
are:
```powershell
.venv\Scripts\python.exe scripts\build_county_locally_extreme_data.py --skip-download
.venv\Scripts\python.exe scripts\apply_locally_extreme_metric_to_climate_data.py
.venv\Scripts\python.exe scripts\build_county_diurnal_temperature_range.py
.venv\Scripts\python.exe scripts\apply_diurnal_temperature_range_to_climate_data.py
.venv\Scripts\python.exe scripts\summarize_county_gridmet_humidity.py --years 1991-2020
.venv\Scripts\python.exe scripts\apply_gridmet_humidity_metric_to_climate_data.py
.venv\Scripts\python.exe scripts\apply_nsrdb_cloud_metric_to_climate_data.py
```
The order matters when rebuilding from scratch: generate the locally extreme
comparison and solar summaries before running their apply step, and summarize
gridMET or NSRDB downloads before merging them. See the data-source document
linked above for acquisition commands, expected artifacts, FIPS handling, and
the representative-point and polygon NSRDB workflows.
Run the current automated tests with:
```powershell
@@ -129,8 +185,8 @@ python -m unittest discover -s tests
The longer-term goal is to add mood-based metrics and investigate whether
patterns in those metrics are associated with climate characteristics such as
temperature, sunlight availability, clear-sky GHI reduction, humidity, precipitation, or extreme-weather
frequency.
temperature, sunlight availability, clear-sky GHI reduction, humidity,
precipitation, or extreme-weather frequency.
That phase still requires decisions about: