KnouandClaude Opus 5
e855d583e3
Split the pipeline documentation by purpose so each fact has one home: - docs/pipeline-plan.md keeps the plan, checklist, tracker, and guardrails - docs/decisions.md holds open decisions and the dated decision log - docs/reviews/ holds findings and tasks: one file per filter, plus 00-cross-filter.md for findings that span filters - scripts/common/README.md holds the shared-helper rules (formerly Phase 2) - filter-calculations.md now describes calculations only Filed findings 12-22 from a consistency audit of the app, docs, and scripts. Filter 1 (Köppen-Geiger): use "Köppen" with the umlaut in all prose, labels, docstrings, help text, and checker messages (finding 21), and correct the base build's "majority" docstring (finding 22). Filter 2 (annual avg temperature): record the adopted definition in filter-calculations.md §2: equally weighted 1991-2020 monthly normals, per WMO-No. 1203 and NOAA's 2020 methodology; area-weighted county means; blank unless all 12 months exist. Code changes for this filter are still pending. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
82 lines
4.2 KiB
Markdown
82 lines
4.2 KiB
Markdown
# Filter 2: Annual avg temperature
|
||
|
||
**Status:** In progress: method decided (2026-09-15).
|
||
|
||
**Data key:** `avgTempF`. Calculation:
|
||
[filter-calculations.md](../filter-calculations.md) §2.
|
||
|
||
## Findings
|
||
|
||
**1. Annual temperature weights months equally.** February has the same weight
|
||
as January or July. If the intended label means an average across all days,
|
||
monthly normals should instead be weighted by the number of days in each
|
||
month. *Resolved on 2026-09-15: equal weighting is the WMO and NOAA standard
|
||
for annual normals and is kept; see "Annual avg temperature month weighting"
|
||
in [decisions.md](../decisions.md).*
|
||
|
||
Cross-filter findings that affect this filter, in
|
||
[00-cross-filter.md](00-cross-filter.md): 2 and 11 (county aggregation),
|
||
12 (Alaska and Hawaii not covered), and 15 (source text cites the station
|
||
Normals).
|
||
|
||
## Decisions
|
||
|
||
In [decisions.md](../decisions.md), all 2026-09-15: Annual avg temperature
|
||
month weighting, aggregation, completeness, and in Alaska and Hawaii.
|
||
|
||
## Tasks
|
||
|
||
Adopted definition: the equally weighted mean of the 12 monthly 1991–2020
|
||
normals of nClimGrid-Monthly `tavg`, area-weighted to each county; blank
|
||
unless all 12 months are present. The per-cell normals follow NCEI's own
|
||
gridded-normals method, "a simple 30-year average of monthly grids"
|
||
(Rennie and Palecki, *U.S. Monthly Gridded Precipitation and Temperature
|
||
Climate Normals*). Alaska and Hawaii stay blank; see
|
||
[decisions.md](../decisions.md).
|
||
|
||
- [x] Verify the calculation against the source data and the WMO and NOAA
|
||
normals definitions (2026-09-15).
|
||
- [x] Decide month weighting, aggregation, completeness, and Alaska/Hawaii
|
||
handling (2026-09-15; see [decisions.md](../decisions.md)).
|
||
- [x] Rewrite `filter-calculations.md` §2 with the adopted definition, citing
|
||
WMO-No. 1203 §4.3.3 and NOAA's 1991–2020 Normals methodology. State that
|
||
average temperature is (Tmax + Tmin)/2 and that the grid covers the
|
||
contiguous U.S. only (2026-09-15; the current touched-cell method is kept
|
||
as "Current method" until the new values are applied).
|
||
- [ ] Add a continuous-value area-weighted mean to
|
||
`scripts/common/county_zonal_stats.py`. It takes an array and transform,
|
||
because nClimGrid is read from netCDF rather than a rasterio file, and
|
||
reuses `cell_coverage_fractions`, the cos(latitude) scaling, and the
|
||
padded window. NaN cells are excluded.
|
||
- [ ] Build `scripts/build_county_avg_temp_metric.py`, writing
|
||
`data/metrics/avg_temp.csv` with `avgTempF` and a valid-month count. The
|
||
1991–2020 climatology step is NOAA-specific, so it stays with the NOAA
|
||
code rather than `common/`.
|
||
- [ ] Single-column apply step
|
||
`scripts/apply_avg_temp_metric_to_climate_data.py` with `--dry-run`.
|
||
Move `read_csv_rows` (4 identical copies in `apply_*` scripts) into
|
||
`common/` as part of this step.
|
||
- [ ] `check_climate_data.py`: keep the 20–85 °F rule and the Alaska/Hawaii
|
||
blank allowance until that open decision in
|
||
[decisions.md](../decisions.md) is made; confirm no new blanks in the
|
||
contiguous U.S.
|
||
- [ ] Tests (`tests/test_avg_temp_metric.py`): partial-cell weights,
|
||
cos(latitude), NaN exclusion, the 12-month rule, equal month weights,
|
||
Celsius-to-Fahrenheit conversion and rounding, and the apply step.
|
||
- [ ] Apply to `data/climate-data.csv`, run the checker, and compare: the
|
||
spread of changes, the largest shifts (expected in small, narrow, and
|
||
coastal counties), and no new blanks among the 3,109 contiguous-U.S.
|
||
counties.
|
||
- [ ] Mark finding 2 resolved for this metric in
|
||
[00-cross-filter.md](00-cross-filter.md).
|
||
- [ ] Correct the `avgTempF` source text in `app.js`, which cites the
|
||
station-based U.S. Climate Normals; describe it as 1991–2020 normals
|
||
computed from nClimGrid-Monthly and link nClimGrid. The "(Normals)"
|
||
label stays.
|
||
- [ ] Correct `scripts/county_data_sources.md`: Source 2 says the build reads
|
||
monthly normals files, but it reads the nClimGrid monthly series and
|
||
averages 1991–2020. Update the `avgTempF` definition line to match.
|
||
- [ ] Add the area-weighted `avgTempF` to the §7 guardrail on rerunning
|
||
`build_county_climate_data.py` in
|
||
[pipeline-plan.md](../pipeline-plan.md).
|