Add project README

README.md

@@ -0,0 +1,157 @@
+# Temperature-Based Analysis
+
+Temperature-Based Analysis is an exploratory project for studying how local
+climate conditions vary across the United States and, eventually, whether those
+conditions show any relationship with mood-based metrics.
+
+The project currently provides the **US County Climate Explorer**, an
+interactive county-level map for viewing and filtering climate data. The mood
+dataset and climate-to-mood correlation analysis are **not implemented yet**.
+At this stage, the project is focused on building, validating, and presenting
+the climate side of the analysis.
+
+## Current Features
+
+- Interactive Leaflet map with county boundaries and county selection
+- Metric groups for climate classification, temperature, precipitation,
+  moisture, and solar exposure
+- Numeric range filters and categorical filters
+- County detail panel showing all available climate metrics
+- Legends and in-app source descriptions
+- Local CSV-based data loading with no application backend or build step
+- Offline Python scripts for assembling and updating county climate data
+
+The browser currently loads 3,221 county-level records from
+`data/climate-data.csv`.
+
+## Available Climate Metrics
+
+| Group | Metrics |
+| --- | --- |
+| Koppen-Geiger Classification | Majority county climate class |
+| Temperature & Extremes | Annual average temperature, diurnal temperature range, absolute extreme days per year, and heat-index days |
+| Precipitation & Moisture | Annual precipitation, precipitation seasonality, wettest month, driest month, and summer specific humidity |
+| Solar Exposure | Average daily global horizontal irradiance (GHI) and cloudiness index |
+
+Most long-term climate metrics use a 1991-2020 reference period. The absolute
+extreme-days metric currently uses 1991-2025 data. Definitions and time periods
+are shown in the application's **Sources** dialog and documented in more detail
+in [`scripts/county_data_sources.md`](scripts/county_data_sources.md).
+
+## Run the Explorer
+
+### Requirements
+
+- A modern web browser
+- Python 3
+- PowerShell for the included convenience script
+- An internet connection for Leaflet, map tiles, and hosted fonts
+
+No Node.js installation or frontend build is required.
+
+From the project root, run:
+
+```powershell
+.\serve.ps1
+```
+
+Then open:
+
+```text
+http://localhost:8000/
+```
+
+To use another port:
+
+```powershell
+.\serve.ps1 -Port 8080
+```
+
+The site must be served over HTTP because the browser loads the climate CSV and
+county GeoJSON with `fetch()`. Opening `index.html` directly as a local file
+will not work.
+
+## Using the Map
+
+1. Choose a metric group and metric from the control panel.
+2. Adjust the value range or category filter to highlight matching counties.
+3. Click a county to zoom to it and inspect all available metrics.
+4. Use **Reset Country View** or **Clear Selection** to return to the broader
+   map.
+5. Open **Sources** for metric definitions and data provenance.
+
+## Project Structure
+
+| Path | Purpose |
+| --- | --- |
+| `index.html` | Application structure and controls |
+| `styles.css` | Layout and visual styling |
+| `app.js` | Map rendering, filtering, county details, and source metadata |
+| `serve.ps1` | Local HTTP server launcher |
+| `data/climate-data.csv` | Browser-ready county climate records |
+| `data/geojson-counties-fips.json` | County geometry keyed by FIPS code |
+| `scripts/` | Climate-data download, aggregation, and update tools |
+| `tests/` | Tests for the NSRDB polygon request and download workflow |
+
+## Climate Data Pipeline
+
+The checked-in browser assets are the final outputs needed to run the explorer.
+The scripts directory contains the larger offline workflow used to derive those
+outputs from sources including:
+
+- Beck et al. Koppen-Geiger climate classification data
+- NOAA NCEI Climate Normals and nClimGrid data
+- gridMET humidity data
+- NREL National Solar Radiation Database data
+- U.S. Census Bureau county geometry
+
+To work on the Python data pipeline, create a virtual environment and install
+the ETL dependencies:
+
+```powershell
+python -m venv .venv
+.\.venv\Scripts\Activate.ps1
+python -m pip install -r scripts\requirements_county_etl.txt
+```
+
+Some data-generation workflows download large files or require NREL/NSRDB API
+credentials. Generated source datasets are intentionally excluded from Git;
+only the browser-ready CSV and GeoJSON are tracked.
+
+Run the current automated tests with:
+
+```powershell
+python -m unittest discover -s tests
+```
+
+## Planned Mood Analysis
+
+The longer-term goal is to add mood-based metrics and investigate whether
+patterns in those metrics are associated with climate characteristics such as
+temperature, sunlight, cloudiness, humidity, precipitation, or extreme-weather
+frequency.
+
+That phase still requires decisions about:
+
+- How mood data will be collected or sourced
+- Geographic and temporal granularity
+- Privacy, consent, and aggregation requirements
+- Confounding variables and missing-data handling
+- Appropriate statistical and visualization methods
+
+No mood records, mood visualizations, or correlation results are currently part
+of the application. Any future relationship found by the project should be
+treated as an association to investigate, not evidence that climate alone
+causes changes in mood.
+
+## Current Limitations
+
+- The application presents long-term county summaries rather than live weather.
+- Climate values aggregate conditions within county boundaries and do not
+  represent every location inside a county.
+- Source datasets use different methods and, in some cases, different time
+  periods.
+- Some solar metrics use representative-point values where county polygon
+  summaries are unavailable.
+- The current interface is an exploratory visualization, not a completed
+  climate-and-mood research analysis.

app.js

@@ -360,25 +360,7 @@ const METRICS = {
   }
 };
 
-const DETAIL_ONLY_METRICS = {
-  oldExtremeDays: {
-    label: "Previous Extreme Days / Year",
-    type: "numeric",
-    unit: "days",
-    source: {
-      description:
-        "Older app proxy preserved for comparison. It came from NOAA 1991-2020 normals using monthly proxy thresholds before the daily nClimGrid replacement was added.",
-      links: [
-        {
-          label: "NOAA NCEI U.S. Climate Normals",
-          url: "https://www.ncei.noaa.gov/products/land-based-station/us-climate-normals"
-        }
-      ]
-    }
-  }
-};
-
-const DETAIL_METRICS = { ...METRICS, ...DETAIL_ONLY_METRICS };
+const DETAIL_METRICS = METRICS;
 const DATA_METRIC_KEYS = Object.keys(METRICS);
 const COUNTY_DETAIL_METRIC_KEYS = [...DATA_METRIC_KEYS];
 const METRIC_GROUPS = [
@@ -390,7 +372,7 @@ const METRIC_GROUPS = [
   {
     key: "temperatureExtremes",
     label: "Temperature & Extremes",
-    metricKeys: ["avgTempF", "avgDiurnalTempRangeF", "extremeDays", "absoluteExtremeDays", "humidHeatDays"]
+    metricKeys: ["avgTempF", "avgDiurnalTempRangeF", "absoluteExtremeDays", "humidHeatDays"]
   },
   {
     key: "precipitationMoisture",
@@ -534,12 +516,9 @@ function getSourceEntries() {
   return [...BASE_SOURCE_ENTRIES, ...metricEntries];
 }
 
-// Keep retired internal audit tags out of the selected-county panel.
+// Normalize source text for the selected-county panel.
 function getVisibleSourceTag(sourceText) {
-  return String(sourceText)
-    .split(" + ")
-    .filter((tag) => !tag.includes("locally-extreme") && !tag.startsWith("extremeDays=") && !tag.startsWith("oldExtremeDays="))
-    .join(" + ");
+  return String(sourceText);
 }
 
 // Render the scrollable sources dialog without injecting source text as HTML.
@@ -914,10 +893,6 @@ function sanitizeOverrideRecord(rawRecord) {
     }
   });
 
-  Object.keys(DETAIL_ONLY_METRICS).forEach((metricKey) => {
-    sanitizedRecord[metricKey] = parseNullableNumber(rawRecord[metricKey]);
-  });
-
   return sanitizedRecord;
 }
 

scripts/apply_locally_extreme_metric_to_climate_data.py

@@ -1,11 +1,10 @@
 #!/usr/bin/env python3
 """
-Apply NOAA daily extreme-day and polygon solar metrics to the app CSV.
+Apply NOAA absolute-temperature threshold and polygon solar metrics to the app CSV.
 
-The browser app already reads the `extremeDays` column, so this script updates
-that column to the average locally extreme days/year while preserving the older
-proxy value in `oldExtremeDays`. It also writes an absolute daily heat/cold
-bucket metric from the same NOAA nClimGrid-Daily county data.
+This script writes an absolute daily heat/cold threshold metric from NOAA
+nClimGrid-Daily county data. Locally percentile-based extreme-day metrics are
+not included in the app CSV.
 
 When an NSRDB polygon archive summary is available, this also replaces the
 older representative-point solar GHI value with the county polygon value. Rows
@@ -25,32 +24,17 @@ DEFAULT_COMPARISON = REPO_ROOT / "data" / "noaa" / "county_locally_extreme_days_
 DEFAULT_POLYGON_SOLAR_GHI = REPO_ROOT / "data" / "nrel" / "county_polygon_ghi_summary.csv"
 DEFAULT_REPRESENTATIVE_POINT_SOLAR_GHI = REPO_ROOT / "data" / "nrel" / "county_representative_point_ghi_summary.csv"
 
-BASE_FIELDS = [
-    "countyFips",
-    "countyName",
-    "state",
-    "koppenZone",
-    "avgTempF",
-    "annualPrecipIn",
-    "seasonalityIndex",
-    "wettestPrecipMonth",
-    "driestPrecipMonth",
+LEGACY_FIELDS = {
+    "absoluteExtremeHeatDays",
+    "absoluteExtremeColdDays",
     "extremeDays",
     "locallyExtremeDays",
     "locallyExtremeHotDays",
     "locallyExtremeColdDays",
-    "absoluteExtremeDays",
     "oldExtremeDays",
     "locallyExtremeAnalysisYears",
     "locallyExtremeSourceFips",
     "locallyExtremeFipsAdjustment",
-    "avgSolarGhiKwhM2Day",
-    "source",
-]
-
-LEGACY_FIELDS = {
-    "absoluteExtremeHeatDays",
-    "absoluteExtremeColdDays",
 }
 
 SOLAR_SOURCE_TAGS = {
@@ -132,26 +116,28 @@ def load_solar_ghi_lookup(solar_ghi_csv: Path) -> dict[str, str]:
     return lookup
 
 
-def append_source_tag(source: str, has_local_value: bool) -> str:
-    """Add the locally extreme provenance tags without duplicating old tags."""
-    old_source = source.strip() or "unknown-source"
-    old_source = old_source.replace(" + absoluteExtremeHeatDays=tmax-gte-95f", "")
-    old_source = old_source.replace(" + absoluteExtremeColdDays=tmin-lte-0f", "")
-    if not has_local_value:
-        return f"{old_source} + missing-noaa-locally-extreme-days"
-
-    has_local_tag = "extremeDays=locally-extreme-noaa-nclimgrid-daily-1991-2025" in old_source
-    has_absolute_tag = "absoluteExtremeDays=tmax-gte-95f-or-tmin-lte-0f" in old_source
-    if has_local_tag and has_absolute_tag:
-        return old_source
-    if has_local_tag:
-        return f"{old_source} + absoluteExtremeDays=tmax-gte-95f-or-tmin-lte-0f"
-
-    return (
-        f"{old_source} + extremeDays=locally-extreme-noaa-nclimgrid-daily-1991-2025 "
-        "+ absoluteExtremeDays=tmax-gte-95f-or-tmin-lte-0f "
-        "+ oldExtremeDays=previous-monthly-proxy"
+def replace_extreme_source_tags(source: str, has_absolute_value: bool) -> str:
+    """Remove retired extreme-day tags and retain absolute-threshold provenance."""
+    source = source.replace("noaa-nclimgrid-1991-2020 (monthly-proxy)", "noaa-nclimgrid-1991-2020")
+    retired_prefixes = (
+        "extremeDays=",
+        "oldExtremeDays=",
+        "missing-noaa-locally-extreme-days",
+        "missing-noaa-absolute-extreme-days",
     )
+    parts = [
+        part.strip()
+        for part in (source.strip() or "unknown-source").split("+")
+        if part.strip()
+        and "locally-extreme" not in part
+        and not part.strip().startswith(retired_prefixes)
+        and part.strip() != "absoluteExtremeDays=tmax-gte-95f-or-tmin-lte-0f"
+    ]
+    if has_absolute_value:
+        parts.append("absoluteExtremeDays=tmax-gte-95f-or-tmin-lte-0f")
+    else:
+        parts.append("missing-noaa-absolute-extreme-days")
+    return " + ".join(parts)
 
 
 def replace_solar_source_tag(source: str, solar_source_tag: str) -> str:
@@ -175,52 +161,35 @@ def apply_locally_extreme_metric(
     representative_point_solar_ghi_csv: Path,
     out: Path,
 ) -> None:
-    """Replace app extreme-day fields and solar GHI with improved metrics."""
+    """Replace the app absolute threshold and solar GHI metrics."""
     original_fields, climate_rows = read_csv_rows(climate_data)
     local_lookup = load_locally_extreme_lookup(comparison_csv)
     polygon_solar_lookup = load_solar_ghi_lookup(polygon_solar_ghi_csv)
     representative_solar_lookup = load_solar_ghi_lookup(representative_point_solar_ghi_csv)
 
-    extra_fields = [
-        field
-        for field in original_fields
-        if field not in BASE_FIELDS and field not in LEGACY_FIELDS
-    ]
-    fieldnames = BASE_FIELDS + extra_fields
+    fieldnames = [field for field in original_fields if field not in LEGACY_FIELDS]
+    if "absoluteExtremeDays" not in fieldnames:
+        insert_after = "driestPrecipMonth"
+        insert_at = fieldnames.index(insert_after) + 1 if insert_after in fieldnames else len(fieldnames)
+        fieldnames.insert(insert_at, "absoluteExtremeDays")
 
-    updated_count = 0
-    missing_count = 0
+    absolute_updated_count = 0
+    absolute_missing_count = 0
     polygon_solar_count = 0
     representative_solar_count = 0
     missing_solar_count = 0
     for row in climate_rows:
         county_fips = normalize_fips(row.get("countyFips", ""))
         local_row = local_lookup.get(county_fips)
-        old_extreme_days = (row.get("oldExtremeDays") or row.get("extremeDays") or "").strip()
 
-        row["oldExtremeDays"] = old_extreme_days
         if local_row:
-            row["extremeDays"] = local_row.get("avgLocallyExtremeDays", "")
-            row["locallyExtremeDays"] = local_row.get("avgLocallyExtremeDays", "")
-            row["locallyExtremeHotDays"] = local_row.get("avgHotExtremeDays", "")
-            row["locallyExtremeColdDays"] = local_row.get("avgColdExtremeDays", "")
             row["absoluteExtremeDays"] = local_row.get("avgAbsoluteExtremeDays", "")
-            row["locallyExtremeAnalysisYears"] = local_row.get("analysisYears", "")
-            row["locallyExtremeSourceFips"] = local_row.get("countyFips", "")
-            row["locallyExtremeFipsAdjustment"] = local_row.get("fipsAdjustment", "")
-            row["source"] = append_source_tag(row.get("source", ""), has_local_value=True)
-            updated_count += 1
+            row["source"] = replace_extreme_source_tags(row.get("source", ""), has_absolute_value=True)
+            absolute_updated_count += 1
         else:
-            row["extremeDays"] = ""
-            row["locallyExtremeDays"] = ""
-            row["locallyExtremeHotDays"] = ""
-            row["locallyExtremeColdDays"] = ""
             row["absoluteExtremeDays"] = ""
-            row["locallyExtremeAnalysisYears"] = ""
-            row["locallyExtremeSourceFips"] = ""
-            row["locallyExtremeFipsAdjustment"] = ""
-            row["source"] = append_source_tag(row.get("source", ""), has_local_value=False)
-            missing_count += 1
+            row["source"] = replace_extreme_source_tags(row.get("source", ""), has_absolute_value=False)
+            absolute_missing_count += 1
 
         polygon_solar_value = polygon_solar_lookup.get(county_fips)
         representative_solar_value = representative_solar_lookup.get(county_fips)
@@ -250,8 +219,8 @@ def apply_locally_extreme_metric(
         writer.writerows(climate_rows)
 
     print(f"Wrote {len(climate_rows)} rows to {out}")
-    print(f"Updated active extremeDays from locally extreme data for {updated_count} rows.")
-    print(f"Left active extremeDays blank for {missing_count} rows without locally extreme data.")
+    print(f"Updated absoluteExtremeDays for {absolute_updated_count} rows.")
+    print(f"Left absoluteExtremeDays blank for {absolute_missing_count} rows without NOAA data.")
     print(f"Updated avgSolarGhiKwhM2Day from polygon archives for {polygon_solar_count} rows.")
     print(f"Kept representative-point solar fallback for {representative_solar_count} rows.")
     print(f"Left avgSolarGhiKwhM2Day blank for {missing_solar_count} rows without solar data.")
@@ -260,7 +229,7 @@ def apply_locally_extreme_metric(
 def parse_args() -> argparse.Namespace:
     """Parse command-line paths for the climate-data update."""
     parser = argparse.ArgumentParser(
-        description="Apply locally extreme days/year and polygon solar GHI to climate-data.csv."
+        description="Apply absolute threshold days/year and polygon solar GHI to climate-data.csv."
     )
     parser.add_argument("--climate-data", type=Path, default=DEFAULT_CLIMATE_DATA)
     parser.add_argument("--comparison-csv", type=Path, default=DEFAULT_COMPARISON)