Architecture Overview · Park Whisperer Platform

Live Theme Park Data Pipeline

A 3-phase ETL job running every 10 minutes on Azure Container Apps that ingests live Walt Disney World attraction data — syncing 286 entities, schedules, and real-time status to 4 PostgreSQL tables with idempotent upsert semantics.

● Production · 10-min cadence286 Park EntitiesIdempotent · Geo-redundant

Click any component to explore it in the full case study.

Input
ThemeParks.wiki API — Walt Disney World
entity_id: e957da41-3552-4cf6-b636-5babc5cbc4e5. Three endpoints: /entity/{id}/children (roster), /entity/{id}/schedule (hours), /entity/{id}/live (real-time status). Polled every 10 minutes.
ThemeParks.wiki · 3 endpoints · 10 min
3-Phase ETL Pipeline — Azure Container Apps Jobs · v6
Phase 1
Entity Sync
UPSERT 286 attractions/restaurants/shows → park_entities_v2. Maintains entity roster with classifications, area mappings, and operational flags.
›
Phase 2
Schedule Sync
UPSERT park operating hours + Lightning Lane purchase options → park_schedules_v2 + ll_purchase_options_v2
›
Phase 3
Live Status Poll
Real-time wait/status → park_live_status_logs_v2. Detect downtime, sellout, and Lightning Lane release events. clip_to_park_hours() for accurate operating_minutes.
🔄
Idempotent Design + Event Detection
ON CONFLICT DO UPDATE semantics throughout — every run is safe to re-run. RETIRED_ENTITY_IDS blocklist prevents stale data. clip_to_park_hours() corrects operating_minutes for closure periods. Blob Storage 35-day geo-redundant backup.
ON CONFLICT DO UPDATE (idempotent)RETIRED_ENTITY_IDS blocklistclip_to_park_hours() accuracy35-day backup retentionGeo-redundant (Blob Storage)10 min run interval
Output A
park_entities_v2
286 park entities with classifications, area mappings, popularity tiers, and hidden gem scores. Foundation for all downstream consumers.
286 rows
Output B
park_live_status_logs_v2
High-volume real-time status log: wait times, operational status, queue types, and Lightning Lane availability per entity per interval.
High-volume · 10-min cadence
Output C
park_schedules_v2
Operating hours, park open/close times, Lightning Lane purchase windows. Used by clip_to_park_hours() for accurate analytics.
Operating context
Output D
selloutEventv2
Event documents: release_duration_minutes, operating_minutes, sellout timestamps. Used by Knowledge RAG and Content Pipelines.
Event analytics
10m
Run interval
286
Park entities
4
PostgreSQL tables
35d
Backup retention
StackAzure Container Apps JobsPython 3.11psycopg2PostgreSQL Flexible ServerThemeParks.wiki APIAzure Blob StorageScheduled trigger