Climate-Mood-Analysis/docs/reviews/00-cross-filter.md

Cross-filter review

Findings that affect more than one filter. The numbering conventions and an index of every finding are in README.md.

Findings

2. Base NOAA aggregation is not area-weighted. Every touched raster cell receives equal weight, including cells that intersect only a small portion of a county. This can matter most for small or narrow counties and along coastlines. Resolved for Köppen on 2026-09-13: class shares are now area-weighted (filter-calculations.md §1).

Affects filters 1 (resolved), 2, 6, and 7.

11. Spatial weighting is inconsistent across metric families. Base NOAA normals use equal touched-cell weights, Köppen uses area-weighted class shares, gridMET humidity uses \cos(\phi) weights on cell centers, and NSRDB polygon metrics use estimated overlap areas. Cross-metric comparisons should account for these different county aggregation methods.

Affects filters 1, 2, 6, 7, 10, 11, and 12. The planned fix is item 3 of the target design in pipeline-plan.md: one shared county-aggregation module used by every raster-based metric.

12. Alaska and Hawaii are not covered by the NOAA and gridMET sources. NOAA nClimGrid and gridMET cover the contiguous U.S. only; the nClimGrid grid spans latitude 24.56 to 49.35 and longitude −124.69 to −67.02. All 29 Alaska and 5 Hawaii counties are therefore blank in filters 2–10. Solar GHI and clear-sky reduction (filters 11 and 12) cover both states. check_climate_data.py allows these blanks (OUTSIDE_CONUS). The notes on the base build in scripts/county_data_sources.md say "fallback values are applied" for counties outside NOAA coverage, but build_county_climate_data.py leaves them blank (build_county_records).

The README's "Current Limitations" states the gap (added 2026-09-15), but the app's county panel shows only "No data". If these counties stay blank, the app should give the reason, for example "Not covered: source data is contiguous U.S. only".

Affects filters 2–10. Open decision: "Alaska and Hawaii coverage" in decisions.md.

13. Lexington, VA (51678) is missing from the NOAA daily county files. Diurnal temperature range and extreme temperature days are blank for it, and check_climate_data.py allows those blanks (NOAA_DAILY_MISSING_FIPS). Heat-index days instead use the surrounding Rockbridge County (51163) as a proxy, recorded in humidHeatSourceFips and humidHeatFipsAdjustment. The three filters handle the same gap differently.

Affects filters 3, 4, and 5.

14. Helper functions are duplicated across scripts, and some copies have drifted. A 2026-09-13 survey found 19 functions with identical copies in several scripts and 19 with copies that have drifted apart. Most identical copies are NSRDB helpers, which belong in an NSRDB module rather than common/; read_csv_rows (4 identical copies in apply_* scripts) is cross-source. Drifted copies need a decision on which version is correct before merging. Notable drifts: summarize_county_gridmet_humidity.py has its own county loader and FIPS normalizer, and the state FIPS table is also copied in build_county_representative_points.py, summarize_county_gridmet_humidity.py, and request_nsrdb_county_polygon_archives.py.

Affects every script that holds a copy. The rules for moving shared code are in scripts/common/README.md.

15. The app's source text for three NOAA metrics names the wrong product. In app.js, avgTempF, annualPrecipIn, and seasonalityIndex credit "NOAA NCEI 1991-2020 U.S. Climate Normals" and link the station-based Normals page. Their values are computed from the nClimGrid-Monthly series averaged over 1991–2020, which is NCEI's gridded-normals method rather than the station product. Source 2 in scripts/county_data_sources.md likewise says the build reads monthly normals files, while the build command passes data/noaa/nclimgrid/nclimgrid_tavg.nc and nclimgrid_prcp.nc. Found during the filter 2 review (2026-09-15).

Affects filters 2, 6, and 7. Filter 2's task list covers the avgTempF text.

16. Source links point to retired or missing pages. Checked 2026-09-15. In app.js, the "NOAA nClimGrid Monthly" link used by wettest and driest month (https://www.ncei.noaa.gov/products/land-based-station/nclimgrid) returns 404, and the "NREL National Solar Radiation Database" link used by solar GHI and clear-sky GHI reduction (https://nsrdb.nrel.gov/) no longer resolves. NREL's sites have moved to nlr.gov: https://nsrdb.nlr.gov/ loads, and the NSRDB scripts already call developer.nlr.gov. In Source 4 of scripts/county_data_sources.md, https://developer.nrel.gov/docs/solar/nsrdb/ also fails (its developer.nlr.gov counterpart loads), and https://www.nrel.gov/gis/solar-resource-maps fails with no counterpart at the same path on nlr.gov. Every other link in app.js, README.md, and scripts/county_data_sources.md loads.

Affects filters 8, 9, 11, and 12.

17. The data-sources metric list is out of date. The "Metric definitions in generated output" list in scripts/county_data_sources.md describes extremeDays / oldExtremeDays as an audit column kept in the app CSV, but neither column is in data/climate-data.csv. The list has no entry for avgDiurnalTempRangeF or absoluteExtremeDays, 2 of the 12 app metrics.

Affects filters 3 and 4.

23. The metric files and metric_sources.json were not in git. .gitignore excluded everything under data/ except the app CSV and the county GeoJSON, so data/metrics/koppen.csv and data/metric_sources.json existed only locally. On a fresh clone, check_climate_data.py failed its metric sources check, and reproduction tier 2 in pipeline-plan.md (reassemble from committed metric files) was impossible. Found 2026-09-15. Resolved on 2026-09-15: .gitignore now keeps both; see "Committing metric files" in decisions.md.

Affects every filter, since each adds a metric file.

24. The data-sources doc does not cover several pipeline steps. scripts/county_data_sources.md has source sections for Köppen, the NOAA gridded normals, county geometry, NSRDB solar, and nClimGrid-Daily, but none for gridMET: no dataset link, license, variables, or commands for download_gridmet_data.py → summarize_county_gridmet_humidity.py → apply_gridmet_humidity_metric_to_climate_data.py. gridMET appears only in two metric-definition lines. The doc also has no commands for build_county_diurnal_temperature_range.py / apply_diurnal_temperature_range_to_climate_data.py or apply_precipitation_month_metrics_to_climate_data.py. Found 2026-09-15.

Affects filters 3, 5, 8, 9, and 10; each filter's review adds its section or commands.

25. Wettest and driest month are computed in two places. build_county_climate_data.py writes wettestPrecipMonth and driestPrecipMonth (_precip_month_extremes), and apply_precipitation_month_metrics_to_climate_data.py recomputes and overwrites them (build_precip_month_lookup), reusing the base build's climatology and zonal-mean helpers. The live values are correct only if the second script runs after the base build. Noted in the original review; numbered 2026-09-15.

Affects filters 8 and 9. This is part of problem 3 in pipeline-plan.md §1.

26. Solar GHI is finalized by the extreme-temperature apply step. build_county_climate_data.py can write meanDailyGlobalHorizontalRadiationKwhM2Day, but the live value is written by apply_locally_extreme_metric_to_climate_data.py, which prefers polygon GHI and falls back to representative-point GHI. Updating GHI therefore means rerunning the extreme-temperature apply step, and nothing in that script's name says it owns the solar column. Noted in the original review; numbered 2026-09-15.

Affects filters 4 and 11. This is part of problem 3 in pipeline-plan.md §1.

Decisions

In decisions.md: Shared helpers (2026-09-13), and the open decision on Alaska and Hawaii coverage.

Tasks

Fixes for these findings are carried out in the affected filters' reviews.

Original review

filter-calculations.md was derived from the checked-in calculation and merge scripts, not solely from UI descriptions. No calculation code was changed. The automated test suite was not executed during this review because pytest is not installed in either the system Python environment or the project virtual environment. The suite uses unittest and runs without pytest: .venv\Scripts\python.exe -m unittest discover -s tests. As of 2026-09-15, all 74 tests pass.

Section 1 and findings 2, 4, and 11 were updated on 2026-09-14, after the Köppen classification was reworked and applied.