Automated Traffic Forecasting for Capacity Planning

Business Context

Disney’s digital properties — including Walt Disney World, Disneyland, Disney Cruise Line, and other Parks and Experiences websites — experience massive traffic fluctuations driven by marketing campaigns, seasonal demand, and holiday booking patterns. Infrastructure teams needed advance warning to spin up compute resources before traffic spikes hit, preventing system overloads during critical revenue-driving periods.

Challenge: No forecasting capability existed. Server teams were reacting to traffic in real-time, leading to either over-provisioning (wasted cost) or under-provisioning (risk of outages during major campaigns).

Technical Approach

Model Architecture

I developed eight distinct forecasting models (one per endpoint) using multi-seasonal ARIMA with Fourier terms to capture complex periodic patterns:

• Primary seasonality: Day-of-week patterns (weekday vs. weekend behavior)
• Secondary seasonality: Week-of-year patterns (seasonal booking cycles)
• Holiday effects: Individual holidays modeled via binary indicators using a lookup table for exact dates
• Exogenous variables: Ad spend and marketing activity

The sine/cosine Fourier approach allowed simultaneous fitting of multiple periodicities without over-parameterization — a critical requirement given the relatively short training windows available for some endpoints.

Prototyping & Validation

Models were initially prototyped in R (using forecast and tsibble ecosystems) to rapidly test configurations. I benchmarked against:

• Facebook Prophet: ARIMA outperformed on our specific multi-seasonality structure
• Deep learning approaches (LSTM/Seq2Seq): ARIMA provided comparable accuracy with superior interpretability and faster training times

Final models were productionized in Python for deployment consistency.

Automated MLOps Pipeline

Built an automated “build, check, redeploy” system using Dataiku:

• Weekly retraining: Models automatically refit on rolling windows
• Accuracy monitoring: Continuous 90-day MAPE validation against holdout periods
• Model promotion: New fits automatically replace production models only if they exceed accuracy thresholds
• Zero manual intervention: Full automation abstracted via Dataiku’s orchestration layer

This avoided the complexity of standing up Airflow or CI/CD infrastructure while achieving equivalent operational reliability.

Impact

Business Outcomes:

• Enabled infrastructure teams to proactively scale compute resources days ahead of predicted traffic spikes
• Eliminated reactive firefighting during major ad campaigns
• Reduced risk of customer-facing outages during peak booking periods
• Optimized cloud resource utilization through data-driven capacity planning

Key Technical Decisions

1. ARIMA over Prophet/Deep Learning: While neural methods showed promise in experiments, ARIMA’s interpretability was essential for infrastructure teams who needed to understand why traffic was predicted to spike. The 5-10% MAPE met accuracy requirements without black-box complexity.

2. Fourier Terms for Multiple Seasonality: Rather than using Prophet’s built-in multi-seasonality (which struggled with our specific holiday structures) or building separate models per seasonality, fitting sine/cosine coefficients simultaneously within ARIMA preserved model coherence and improved convergence.

3. Dataiku over Traditional MLOps: Given organizational constraints and existing Dataiku licensing, leveraging its built-in scheduling and model management accelerated time-to-production by months versus building Airflow pipelines from scratch. Trade-off was minor vendor lock-in, but operational reliability was equivalent.

4. Endpoint-Specific Models vs. Global Model: Eight separate models allowed each Disney property to have distinct seasonal patterns (cruise booking cycles differ from theme park cycles) while sharing the same underlying architecture and automation framework.