Architecture Overview · Park Whisperer Platform

Database Architecture Decision: A Cost Inflection Story

Three-era architectural evolution from a single Cosmos DB container ($394/mo floor, hot partition at 62.5%) through a dual-write split to full PostgreSQL-only decommission — saving $394/month and cutting pipeline runtime by 60%.

Architecture EvolutionCosmos DB → PostgreSQL$394/mo → $0 Cosmos Cost

Click any component to explore it in the full case study.

Input
ThemeParks.wiki Live Feed
57,600 docs/day peak at full polling rate — park status updates, wait times, schedules. Every 10 minutes across 286 entities.
57,600 docs/day peak
Era 1: Cosmos DB (v1–v5)
Single-Container Hot Partition
All data in one Cosmos DB container partitioned by /doctype. 5.9M documents accumulated. waitSnapshot dominated 62.5% of reads — hot partition degraded performance. $394/month floor cost regardless of usage.
5.9M docs · $394/mo floor
Era 2: PostgreSQL + Cosmos Split (v6 early)
Ingest Separated, Publish Continued
PostgreSQL handles ingestion. Data republished to 3 Cosmos containers: park_timeseries, park_aggregates, park_config. Improved partition distribution but added operational complexity and retained Cosmos cost.
Dual-write · Cost persists
Era 3: PostgreSQL Only (v6 current)
Full Cosmos Decommission
PostgreSQL-only: aggregates computed at query time via covering indexes. 6 PG covering indexes replace all Cosmos RU costs. Batch writes with ThreadPoolExecutor×4. Runtime: 2–3 min (was 6–9 min).
$0 Cosmos · 6 indexes
💡
Architecture Decision Record — Three Cost Inflection Points
Era 1 → 2: Hot partition failure forced separation. Era 2 → 3: Discovered Cosmos costs never fell below $394/month floor regardless of optimization. Hierarchical partition key [doctype, date] was analyzed but never built — cost-benefit analysis drove full decommission instead.
$394/mo Cosmos floor → $0 todayHot partition: 62.5% waitSnapshotBatch writes (ThreadPoolExecutor×4)6 PG covering indexes replace Cosmos RURuntime 6–9 min → 2–3 minLessons: floor cost traps, query-time compute
5.9M
Peak Cosmos documents
$394
Monthly Cosmos floor (before)
$0
Cosmos cost today
62.5%
Hot partition share (waitSnapshot)
StackCosmos DB (deprecated)PostgreSQL Flexible ServerThreadPoolExecutor batch writespsycopg2Covering indexesAzure Container Apps JobsPython