Weather Whisperer v5
ML Nowcasting for Ride Operations

vw_storm_state_current is a single-row, 5-state storm severity engine (CLEAR → WATCH → APPROACH → ACTIVE → SEVERE) fusing lightning proximity and NWS alerts — the primary go/no-go signal for outdoor ride operations. ride_weather_sensitivity stores historical storm/wind closure sensitivity percentages per individual ride, joined against live conditions by weather-ml-predictor and weather-outage-tracker. v_rides_currently_down joins rides currently down during operating hours with live weather context, and tbl_weather_ride_outage_log records weather-attributed closures written by the outage tracker.

Thunderstorm, precipitation, high-wind, fog, venue-impact, and high-impact-venue prediction have different feature relevance and different cost-of-error profiles. Keeping them as 6 separately trained scikit-learn models (retrained on-demand by weather-ml-trainer from 44K+ observations, pickled and versioned in GCS) lets weather-ml-predictor report a distinct confidence per hazard rather than one blended score that obscures which risk is actually driving a recommendation. BigQuery remains the single source of truth for features and labels — the models just live outside it, where model choice and explainability are easier to control.

CAPE, CIN, and Lifted Index from upper-air soundings are leading indicators of convective development — they signal thunderstorm potential 3–6 hours before surface conditions deteriorate. METAR alone would only react to thunderstorms after they're already overhead. The atmospheric component of the ride impact score gives operations staff advance warning: a high CAPE + low CIN sounding at 12Z means the afternoon convective risk is elevated even if current surface conditions look fine. The University of Wyoming archives soundings at 00Z and 12Z — the 12-hour ingestion schedule matches this availability window exactly.

The consuming application is hosted on Azure, and now also an AWS Bedrock agent. A GCP API Gateway in front of BigQuery would add ~$50–100/month in API call costs and a round-trip to GCP on every request. Instead, the publisher job runs every 5–15 minutes (1 minute during storms) and writes pre-computed JSON to a CORS-enabled public GCS bucket. Azure web apps fetch the JSON directly — no GCP API calls at read time. GCS standard storage for ~200 objects costs under $1/month. This architecture decouples the compute (GCP) from the consumption (Azure, AWS) with no persistent connection required.

METAR observations accumulate at ~144 records/day per station (every 10 min). At 17 monitored stations, that's ~2,450 rows/day, ~894,000 rows/year — small in absolute terms, but the curated views are queried on every predictor/publisher run (every 10–20 min). Without partitioning, BigQuery would scan the full table on every query. DAY partitioning on observation_time with clustering on station_id means each view only scans the relevant partition and skips other stations via clustering — reducing per-query bytes scanned from GB to KB. BigQuery charges $5/TB scanned; partition + clustering drops the query cost to near zero.

Cloud Run Jobs are the right primitive for batch/scheduled workloads that run for seconds to minutes and then exit. Each ingestion job runs for <2 minutes every 15–60 minutes — an always-on container or Cloud Function with a long timeout would be idle >95% of the time. Cloud Run Jobs bill per CPU-second of execution, not per allocation time. Cloud Functions have a 9-minute max timeout — the atmospheric ingestion job parses complex sounding data and could approach this. Cloud Run Jobs have a 1-hour timeout with no per-invocation startup overhead once the container is cached. Cloud Scheduler triggers each job on its own schedule independently.