Refine project documentation and track metric files
Close gaps found in a review of the documentation: - Track data/metrics/ and data/metric_sources.json in git so the data checker passes on a fresh clone (finding 23; decision logged) - State the Alaska and Hawaii coverage gap in the README limitations and extend finding 12 - File findings 24-26: the data-sources doc lacks gridMET and several pipeline commands; wettest/driest month are computed twice; solar GHI is written by the extreme-temperature apply step - Add the stale "fallback values" note to filter 2's tasks Tidy the document system: - Add docs/reviews/README.md with the numbering rules and a finding index - Rename koppen-mixed-display-plan.md to koppen-mixed-display.md and fix its stale Puerto Rico and "stage 5" text - Add the precipitation-month step to the README enrichment list - Describe the Current method / Previous method pattern in plan section 5 - Add CLAUDE.md with the project guardrails and doc layout Format filter-calculations.md so it renders on GitHub and in VS Code: inline math uses $...$, ranges use en dashes, and implementation references name functions instead of line numbers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -1,10 +1,7 @@
|
||||
# Cross-filter review
|
||||
|
||||
Findings that affect more than one filter. Each finding lives in exactly one
|
||||
review file; the filter reviews it affects link here and record only how it
|
||||
applied to them. Findings keep their numbers across all review files, and new
|
||||
findings take the next number wherever they are filed. A finding that turns out
|
||||
to affect other filters moves here, leaving a link behind.
|
||||
Findings that affect more than one filter. The numbering conventions and an
|
||||
index of every finding are in [README.md](README.md).
|
||||
|
||||
## Findings
|
||||
|
||||
@@ -19,7 +16,7 @@ Affects filters 1 (resolved), 2, 6, and 7.
|
||||
**11. Spatial weighting is inconsistent across metric families.** Base NOAA
|
||||
normals use equal touched-cell weights, Köppen uses area-weighted class
|
||||
shares, gridMET humidity uses
|
||||
\(\cos(\phi)\) weights on cell centers, and NSRDB polygon metrics use
|
||||
$\cos(\phi)$ weights on cell centers, and NSRDB polygon metrics use
|
||||
estimated overlap areas. Cross-metric comparisons should account for these
|
||||
different county aggregation methods.
|
||||
|
||||
@@ -35,7 +32,12 @@ clear-sky reduction (filters 11 and 12) cover both states.
|
||||
`check_climate_data.py` allows these blanks (`OUTSIDE_CONUS`). The notes on
|
||||
the base build in `scripts/county_data_sources.md` say "fallback values are
|
||||
applied" for counties outside NOAA coverage, but `build_county_climate_data.py`
|
||||
leaves them blank (lines 499–520).
|
||||
leaves them blank (`build_county_records`).
|
||||
|
||||
The README's "Current Limitations" states the gap (added 2026-09-15), but the
|
||||
app's county panel shows only "No data". If these counties stay blank, the app
|
||||
should give the reason, for example "Not covered: source data is contiguous
|
||||
U.S. only".
|
||||
|
||||
Affects filters 2–10. Open decision: "Alaska and Hawaii coverage" in
|
||||
[decisions.md](../decisions.md).
|
||||
@@ -99,6 +101,56 @@ neither column is in `data/climate-data.csv`. The list has no entry for
|
||||
|
||||
Affects filters 3 and 4.
|
||||
|
||||
**23. The metric files and `metric_sources.json` were not in git.**
|
||||
`.gitignore` excluded everything under `data/` except the app CSV and the
|
||||
county GeoJSON, so `data/metrics/koppen.csv` and `data/metric_sources.json`
|
||||
existed only locally. On a fresh clone, `check_climate_data.py` failed its
|
||||
metric sources check, and reproduction tier 2 in
|
||||
[pipeline-plan.md](../pipeline-plan.md) (reassemble from committed metric
|
||||
files) was impossible. Found 2026-09-15. *Resolved on 2026-09-15: `.gitignore`
|
||||
now keeps both; see "Committing metric files" in
|
||||
[decisions.md](../decisions.md).*
|
||||
|
||||
Affects every filter, since each adds a metric file.
|
||||
|
||||
**24. The data-sources doc does not cover several pipeline steps.**
|
||||
`scripts/county_data_sources.md` has source sections for Köppen, the NOAA
|
||||
gridded normals, county geometry, NSRDB solar, and nClimGrid-Daily, but none
|
||||
for gridMET: no dataset link, license, variables, or commands for
|
||||
`download_gridmet_data.py` → `summarize_county_gridmet_humidity.py` →
|
||||
`apply_gridmet_humidity_metric_to_climate_data.py`. gridMET appears only in two
|
||||
metric-definition lines. The doc also has no commands for
|
||||
`build_county_diurnal_temperature_range.py` /
|
||||
`apply_diurnal_temperature_range_to_climate_data.py` or
|
||||
`apply_precipitation_month_metrics_to_climate_data.py`. Found 2026-09-15.
|
||||
|
||||
Affects filters 3, 5, 8, 9, and 10; each filter's review adds its section or
|
||||
commands.
|
||||
|
||||
**25. Wettest and driest month are computed in two places.**
|
||||
`build_county_climate_data.py` writes `wettestPrecipMonth` and
|
||||
`driestPrecipMonth` (`_precip_month_extremes`), and
|
||||
`apply_precipitation_month_metrics_to_climate_data.py` recomputes and
|
||||
overwrites them (`build_precip_month_lookup`), reusing the base build's
|
||||
climatology and zonal-mean helpers. The live values are correct only if the
|
||||
second script runs after the base build. Noted in the original review;
|
||||
numbered 2026-09-15.
|
||||
|
||||
Affects filters 8 and 9. This is part of problem 3 in
|
||||
[pipeline-plan.md](../pipeline-plan.md) §1.
|
||||
|
||||
**26. Solar GHI is finalized by the extreme-temperature apply step.**
|
||||
`build_county_climate_data.py` can write
|
||||
`meanDailyGlobalHorizontalRadiationKwhM2Day`, but the live value is written by
|
||||
`apply_locally_extreme_metric_to_climate_data.py`, which prefers polygon GHI
|
||||
and falls back to representative-point GHI. Updating GHI therefore means
|
||||
rerunning the extreme-temperature apply step, and nothing in that script's name
|
||||
says it owns the solar column. Noted in the original review; numbered
|
||||
2026-09-15.
|
||||
|
||||
Affects filters 4 and 11. This is part of problem 3 in
|
||||
[pipeline-plan.md](../pipeline-plan.md) §1.
|
||||
|
||||
## Decisions
|
||||
|
||||
In [decisions.md](../decisions.md): Shared helpers (2026-09-13), and the open
|
||||
@@ -115,7 +167,8 @@ scripts, not solely from UI descriptions. No calculation code was changed. The
|
||||
automated test suite was not executed during this review because `pytest` is
|
||||
not installed in either the system Python environment or the project virtual
|
||||
environment. The suite uses `unittest` and runs without pytest:
|
||||
`python -m unittest discover -s tests`.
|
||||
`.venv\Scripts\python.exe -m unittest discover -s tests`. As of 2026-09-15,
|
||||
all 74 tests pass.
|
||||
|
||||
Section 1 and findings 2, 4, and 11 were updated on 2026-09-14, after the
|
||||
Köppen classification was reworked and applied.
|
||||
|
||||
Reference in New Issue
Block a user