Knou/Climate-Mood-Analysis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Refine project documentation and track metric files

Browse Source 
Close gaps found in a review of the documentation:
- Track data/metrics/ and data/metric_sources.json in git so the data
  checker passes on a fresh clone (finding 23; decision logged)
- State the Alaska and Hawaii coverage gap in the README limitations and
  extend finding 12
- File findings 24-26: the data-sources doc lacks gridMET and several
  pipeline commands; wettest/driest month are computed twice; solar GHI
  is written by the extreme-temperature apply step
- Add the stale "fallback values" note to filter 2's tasks

Tidy the document system:
- Add docs/reviews/README.md with the numbering rules and a finding index
- Rename koppen-mixed-display-plan.md to koppen-mixed-display.md and fix
  its stale Puerto Rico and "stage 5" text
- Add the precipitation-month step to the README enrichment list
- Describe the Current method / Previous method pattern in plan section 5
- Add CLAUDE.md with the project guardrails and doc layout

Format filter-calculations.md so it renders on GitHub and in VS Code:
inline math uses $...$, ranges use en dashes, and implementation
references name functions instead of line numbers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Knou
2026-09-15 17:31:26 -04:00
co-authored by Claude Opus 5
parent e855d583e3
commit 867a07cecb
20 changed files with 3519 additions and 121 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/reviews/00-cross-filter.md
+61 -8
View File
@@ -1,10 +1,7 @@
# Cross-filter review
Findings that affect more than one filter. Each finding lives in exactly one
review file; the filter reviews it affects link here and record only how it
applied to them. Findings keep their numbers across all review files, and new
findings take the next number wherever they are filed. A finding that turns out
to affect other filters moves here, leaving a link behind.
Findings that affect more than one filter. The numbering conventions and an
index of every finding are in [README.md](README.md).
## Findings
@@ -19,7 +16,7 @@ Affects filters 1 (resolved), 2, 6, and 7.
**11. Spatial weighting is inconsistent across metric families.** Base NOAA
normals use equal touched-cell weights, Köppen uses area-weighted class
shares, gridMET humidity uses
\(\cos(\phi)\) weights on cell centers, and NSRDB polygon metrics use
$\cos(\phi)$ weights on cell centers, and NSRDB polygon metrics use
estimated overlap areas. Cross-metric comparisons should account for these
different county aggregation methods.
@@ -35,7 +32,12 @@ clear-sky reduction (filters 11 and 12) cover both states.
`check_climate_data.py` allows these blanks (`OUTSIDE_CONUS`). The notes on
the base build in `scripts/county_data_sources.md` say "fallback values are
applied" for counties outside NOAA coverage, but `build_county_climate_data.py`
leaves them blank (lines 499–520).
leaves them blank (`build_county_records`).
The README's "Current Limitations" states the gap (added 2026-09-15), but the
app's county panel shows only "No data". If these counties stay blank, the app
should give the reason, for example "Not covered: source data is contiguous
U.S. only".
Affects filters 2–10. Open decision: "Alaska and Hawaii coverage" in
[decisions.md](../decisions.md).
@@ -99,6 +101,56 @@ neither column is in `data/climate-data.csv`. The list has no entry for
Affects filters 3 and 4.
**23. The metric files and `metric_sources.json` were not in git.**
`.gitignore` excluded everything under `data/` except the app CSV and the
county GeoJSON, so `data/metrics/koppen.csv` and `data/metric_sources.json`
existed only locally. On a fresh clone, `check_climate_data.py` failed its
metric sources check, and reproduction tier 2 in
[pipeline-plan.md](../pipeline-plan.md) (reassemble from committed metric
files) was impossible. Found 2026-09-15. *Resolved on 2026-09-15: `.gitignore`
now keeps both; see "Committing metric files" in
[decisions.md](../decisions.md).*
Affects every filter, since each adds a metric file.
**24. The data-sources doc does not cover several pipeline steps.**
`scripts/county_data_sources.md` has source sections for Köppen, the NOAA
gridded normals, county geometry, NSRDB solar, and nClimGrid-Daily, but none
for gridMET: no dataset link, license, variables, or commands for
`download_gridmet_data.py` → `summarize_county_gridmet_humidity.py` →
`apply_gridmet_humidity_metric_to_climate_data.py`. gridMET appears only in two
metric-definition lines. The doc also has no commands for
`build_county_diurnal_temperature_range.py` /
`apply_diurnal_temperature_range_to_climate_data.py` or
`apply_precipitation_month_metrics_to_climate_data.py`. Found 2026-09-15.
Affects filters 3, 5, 8, 9, and 10; each filter's review adds its section or
commands.
**25. Wettest and driest month are computed in two places.**
`build_county_climate_data.py` writes `wettestPrecipMonth` and
`driestPrecipMonth` (`_precip_month_extremes`), and
`apply_precipitation_month_metrics_to_climate_data.py` recomputes and
overwrites them (`build_precip_month_lookup`), reusing the base build's
climatology and zonal-mean helpers. The live values are correct only if the
second script runs after the base build. Noted in the original review;
numbered 2026-09-15.
Affects filters 8 and 9. This is part of problem 3 in
[pipeline-plan.md](../pipeline-plan.md) §1.
**26. Solar GHI is finalized by the extreme-temperature apply step.**
`build_county_climate_data.py` can write
`meanDailyGlobalHorizontalRadiationKwhM2Day`, but the live value is written by
`apply_locally_extreme_metric_to_climate_data.py`, which prefers polygon GHI
and falls back to representative-point GHI. Updating GHI therefore means
rerunning the extreme-temperature apply step, and nothing in that script's name
says it owns the solar column. Noted in the original review; numbered
2026-09-15.
Affects filters 4 and 11. This is part of problem 3 in
[pipeline-plan.md](../pipeline-plan.md) §1.
## Decisions
In [decisions.md](../decisions.md): Shared helpers (2026-09-13), and the open
@@ -115,7 +167,8 @@ scripts, not solely from UI descriptions. No calculation code was changed. The
automated test suite was not executed during this review because `pytest` is
not installed in either the system Python environment or the project virtual
environment. The suite uses `unittest` and runs without pytest:
`python -m unittest discover -s tests`.
`.venv\Scripts\python.exe -m unittest discover -s tests`. As of 2026-09-15,
all 74 tests pass.
Section 1 and findings 2, 4, and 11 were updated on 2026-09-14, after the
Köppen classification was reworked and applied.
docs/reviews/01-koppen.md
+2 -2
View File
@@ -6,7 +6,7 @@ updated.
**Data keys:** `koppenZone`, `koppenPrimaryClass`, `koppenSecondaryClass`.
Calculation: [filter-calculations.md](../filter-calculations.md) §1. Display
design and history:
[koppen-mixed-display-plan.md](../koppen-mixed-display-plan.md).
[koppen-mixed-display.md](../koppen-mixed-display.md).
## Findings
@@ -69,7 +69,7 @@ covers at least 50% of the county's land and leads the runner-up by at least
- [x] Allow `Mixed` in `check_climate_data.py`.
- [x] Add a Mixed climate category to `app.js`, drawn as stripes of the
county's top two classes; see
[koppen-mixed-display-plan.md](../koppen-mixed-display-plan.md).
[koppen-mixed-display.md](../koppen-mixed-display.md).
- [x] Replace the plurality description in `filter-calculations.md` §1 and mark
review findings 2 and 4 resolved for Köppen (2026-09-14).
- [x] Update the Köppen descriptions and script lists in `README.md` and
docs/reviews/02-annual-avg-temperature.md
+3
View File
@@ -76,6 +76,9 @@ Climate Normals*). Alaska and Hawaii stay blank; see
- [ ] Correct `scripts/county_data_sources.md`: Source 2 says the build reads
monthly normals files, but it reads the nClimGrid monthly series and
averages 1991–2020. Update the `avgTempF` definition line to match.
Also remove the "Run the generator" note that fallback values are
applied to counties outside NOAA coverage; they are left blank
(finding 12).
- [ ] Add the area-weighted `avgTempF` to the §7 guardrail on rerunning
`build_county_climate_data.py` in
[pipeline-plan.md](../pipeline-plan.md).
docs/reviews/03-diurnal-temperature-range.md
+2 -1
View File
@@ -11,7 +11,8 @@ No filter-specific findings yet.
Cross-filter findings that affect this filter, in
[00-cross-filter.md](00-cross-filter.md): 12 (Alaska and Hawaii not covered),
13 (Lexington, VA blank), and 17 (missing from the data-sources metric list).
13 (Lexington, VA blank), 17 (missing from the data-sources metric list), and
24 (no build or apply commands in the data-sources doc).
## Decisions
docs/reviews/04-extreme-temperature-days.md
+2 -1
View File
@@ -28,7 +28,8 @@ included in the app CSV.
Cross-filter findings that affect this filter, in
[00-cross-filter.md](00-cross-filter.md): 12 (Alaska and Hawaii not covered),
13 (Lexington, VA blank), and 17 (missing from the data-sources metric list).
13 (Lexington, VA blank), 17 (missing from the data-sources metric list), and
26 (this filter's apply step also writes solar GHI).
## Decisions
docs/reviews/05-heat-index-days.md
+3 -2
View File
@@ -17,8 +17,9 @@ Tmax/RH pair is included in the equal-year average. A minimum valid-day rule
would reduce low-biased partial-year counts.
Cross-filter findings that affect this filter, in
[00-cross-filter.md](00-cross-filter.md): 12 (Alaska and Hawaii not covered)
and 13 (Lexington, VA uses Rockbridge County as a proxy).
[00-cross-filter.md](00-cross-filter.md): 12 (Alaska and Hawaii not covered),
13 (Lexington, VA uses Rockbridge County as a proxy), and 24 (no gridMET
section in the data-sources doc).
## Decisions
docs/reviews/08-wettest-month.md
+5 -4
View File
@@ -7,12 +7,13 @@
## Findings
No numbered findings yet. Known issue to review: computed in both the base
build and the precipitation-month script.
No filter-specific findings yet.
Cross-filter findings that affect this filter, in
[00-cross-filter.md](00-cross-filter.md): 12 (Alaska and Hawaii not covered)
and 16 (the app's nClimGrid source link returns 404).
[00-cross-filter.md](00-cross-filter.md): 12 (Alaska and Hawaii not covered),
16 (the app's nClimGrid source link returns 404), 24 (no precipitation-month
command in the data-sources doc), and 25 (computed in both the base build and
the precipitation-month script).
## Decisions
docs/reviews/09-driest-month.md
+5 -5
View File
@@ -7,13 +7,13 @@
## Findings
No numbered findings yet. Known issue to review: same as wettest month, which
is computed in both the base build and the precipitation-month script; see
[08-wettest-month.md](08-wettest-month.md).
No filter-specific findings yet.
Cross-filter findings that affect this filter, in
[00-cross-filter.md](00-cross-filter.md): 12 (Alaska and Hawaii not covered)
and 16 (the app's nClimGrid source link returns 404).
[00-cross-filter.md](00-cross-filter.md): 12 (Alaska and Hawaii not covered),
16 (the app's nClimGrid source link returns 404), 24 (no precipitation-month
command in the data-sources doc), and 25 (computed in both the base build and
the precipitation-month script).
## Decisions
docs/reviews/10-summer-specific-humidity.md
+4 -4
View File
@@ -7,12 +7,12 @@
## Findings
No filter-specific findings yet. Known issue to review: cell-center
cos(latitude) aggregation differs from other metrics (finding 11).
No filter-specific findings yet.
Cross-filter findings that affect this filter, in
[00-cross-filter.md](00-cross-filter.md): 11 (inconsistent aggregation) and
12 (Alaska and Hawaii not covered).
[00-cross-filter.md](00-cross-filter.md): 11 (inconsistent aggregation),
12 (Alaska and Hawaii not covered), and 24 (no gridMET section in the
data-sources doc).
## Decisions
docs/reviews/11-solar-ghi.md
+3 -4
View File
@@ -25,11 +25,10 @@ representative-point fallback, and the per-row `source` tag is always
`data/nrel/county_polygon_ghi_summary.csv`. The apply step later replaces both
the value and the tag, so only the base build's output is mislabeled.
Known issue to review: finalized by the extreme-temperature apply script.
Cross-filter findings that affect this filter, in
[00-cross-filter.md](00-cross-filter.md): 11 (inconsistent aggregation) and
16 (the app's NSRDB source link no longer resolves).
[00-cross-filter.md](00-cross-filter.md): 11 (inconsistent aggregation),
16 (the app's NSRDB source link no longer resolves), and 26 (finalized by the
extreme-temperature apply step).
## Decisions
docs/reviews/README.md
+53
View File
@@ -0,0 +1,53 @@
# Filter reviews
Each filter has one review file (`01`–`12`) holding its findings, links to its
decisions, and its task checklist. [00-cross-filter.md](00-cross-filter.md)
holds findings that affect more than one filter. Review status is tracked in
[pipeline-plan.md](../pipeline-plan.md) §6, and every review follows the
checklist in §5.
## Conventions
- Each finding lives in exactly one review file. The other filter reviews it
affects link to it and record only how it applied to them.
- Findings keep one number across all review files. A new finding takes the
next number (27 as of 2026-09-15) wherever it is filed, and is added to the
index below.
- A finding that turns out to affect other filters moves to `00`, leaving a
link behind.
- A resolved finding stays where it is, with its resolution in italics.
- A finding accepted as a limitation becomes a caveat in
[filter-calculations.md](../filter-calculations.md).
## Finding index
Status is kept in each finding, not here.
| # | Finding | File |
| --- | --- | --- |
| 1 | Annual temperature weights months equally | [02](02-annual-avg-temperature.md) |
| 2 | Base NOAA aggregation is not area-weighted | [00](00-cross-filter.md) |
| 3 | Partial precipitation years are accepted | [06](06-annual-precipitation.md) |
| 4 | The Köppen fallback can create false data | [01](01-koppen.md) |
| 5 | Extreme-day counts are not completeness-normalized | [04](04-extreme-temperature-days.md) |
| 6 | The absolute-extreme metric depends on unrelated percentile thresholds | [04](04-extreme-temperature-days.md) |
| 7 | Heat Index days are a daily-extrema proxy | [05](05-heat-index-days.md) |
| 8 | Heat-year completeness is permissive | [05](05-heat-index-days.md) |
| 9 | The GHI formula assumes hourly, 365-day input | [11](11-solar-ghi.md) |
| 10 | Clear-sky reduction averages ratios rather than energy totals | [12](12-clear-sky-ghi-reduction.md) |
| 11 | Spatial weighting is inconsistent across metric families | [00](00-cross-filter.md) |
| 12 | Alaska and Hawaii are not covered by the NOAA and gridMET sources | [00](00-cross-filter.md) |
| 13 | Lexington, VA (51678) is missing from the NOAA daily county files | [00](00-cross-filter.md) |
| 14 | Helper functions are duplicated across scripts | [00](00-cross-filter.md) |
| 15 | The app's source text for three NOAA metrics names the wrong product | [00](00-cross-filter.md) |
| 16 | Source links point to retired or missing pages | [00](00-cross-filter.md) |
| 17 | The data-sources metric list is out of date | [00](00-cross-filter.md) |
| 18 | The data-sources doc describes the retired locally extreme metric | [04](04-extreme-temperature-days.md) |
| 19 | The data-sources doc recommends a GHI raster the pipeline does not use | [11](11-solar-ghi.md) |
| 20 | The base build labels any solar CSV as representative-point | [11](11-solar-ghi.md) |
| 21 | The documented Köppen labels differ from the app | [01](01-koppen.md) |
| 22 | The base build's docstring calls its Köppen value a majority class | [01](01-koppen.md) |
| 23 | The metric files and `metric_sources.json` were not in git | [00](00-cross-filter.md) |
| 24 | The data-sources doc does not cover several pipeline steps | [00](00-cross-filter.md) |
| 25 | Wettest and driest month are computed in two places | [00](00-cross-filter.md) |
| 26 | Solar GHI is finalized by the extreme-temperature apply step | [00](00-cross-filter.md) |