Agent-Based Marketing Mix Simulator for COVID-era Planning
Business Context
For many years, Disney had relied on linear regression-based marketing mix models to optimize advertising spend across parks and experiences. When COVID-19 forced park closures in 2020, attendance dropped to zero — creating a structural break that invalidated historical relationships between ad spend and visitation. Traditional models could not be used for planning during reopening or recovery phases.
Critical Constraint: Marketing teams needed to optimize ad spend for the post-COVID recovery without reliable historical data. Waiting 2-3 years to rebuild training data was not viable.
Technical Approach
From Regression to Simulation
I designed and built an agent-based simulation from first principles to model visitor behavior as emergent from individual family decision-making, rather than relying on aggregate historical patterns.
Agent Architecture:
The simulation generates a synthetic population of families based on US Census distributions:
- Family size, age composition, household income, geographic location (zip code)
- Each agent represents a potential visiting family with realistic demographic variation
Weekly Simulation Dynamics:
-
Ad Exposure (Poisson Process): Each week, families are exposed to advertising impressions drawn from a Poisson distribution where λ (exposure rate) is parameterized by actual ad spend levels
-
Attendance Decision (Binomial Choice): Each family makes an independent weekly decision to visit or not, with probability determined by:
- Demographic characteristics (income, distance, family composition)
- Cumulative ad impressions (awareness/consideration)
- Random utility component capturing unobserved preferences
Temporal Aggregation: Weekly individual decisions aggregate to simulated park attendance, which is compared against actual (pre-COVID) attendance patterns for model validation and calibration.
Implementation Strategy
Performance-Critical Core (Java): The simulation engine was implemented in Java to handle millions of agent-week computations efficiently. Each full simulation run (52 weeks × 100,000+ agents) required 30-45 minutes — prohibitive for iterative fitting without optimization.
Fitting Framework (Python): I built a Python orchestration layer using Bayesian optimization (Gaussian Process surrogate) to navigate the expensive simulation landscape efficiently:
- Each candidate parameter set required a full Java simulation run
- Bayesian optimization minimized the number of simulations needed to find high-likelihood parameter regions
- Surrogate model guided exploration toward parameter combinations that matched historical attendance patterns
Calibration Target: The model was fit to reproduce aggregate weekly attendance patterns from 2018-2019 (pre-COVID) under actual historical ad spend levels, validating that the micro-level agent rules could recover observed macro-level behavior.
Development & Transition
I completed the core architecture and fitting framework, then transitioned final parameter fitting to an offshore team. The model was fully calibrated and ready for scenario planning at the time of my departure.
Key Technical Decisions
- Agent-Based vs. Alternative Approaches:
- Why not wait for new data: Business couldn’t delay optimization decisions for 2+ years
- Why not extrapolate: Structural break (COVID) meant pre-COVID relationships were unreliable
- Why simulation: Micro-level behavioral rules can be validated against pre-COVID data while allowing counterfactual exploration of recovery scenarios
-
Java Implementation: Accepted language heterogeneity (Java core, Python fitting) to achieve necessary runtime performance. 30-45 minute runtimes were still expensive, but Python-based simulation would have been prohibitive for standard optimization techniques requiring hundreds of evaluations.
-
Bayesian Optimization for Fitting: Given the expensive objective function (full simulation per evaluation), random or grid search was infeasible. Gaussian Process surrogate modeling allowed intelligent exploration of parameter space with minimal simulation runs.
-
Census-Based Synthetic Population: Rather than using Disney’s actual customer data (unavailable or privacy-restricted), US Census distributions provided a realistic, publicly verifiable foundation for agent generation that could be validated against aggregate attendance demographics.
- Poisson/Binomial Distributions: Chose standard probability distributions with clear behavioral interpretations (Poisson for ad exposure opportunities, Binomial for yes/no attendance decisions) to maintain model interpretability for marketing stakeholders.
Technical Challenges
| Challenge | Solution |
|---|---|
| Expensive simulation runs | Java implementation + Bayesian optimization to minimize evaluations |
| No post-COVID validation data | Calibrated on pre-COVID data; model validity depends on behavioral stability assumption |
| Aggregate validation only | Validated that micro-rules recover macro-patterns; individual-level prediction impossible |
| Parameter identifiability | Used informative priors from marketing theory; accepted multiple parameter combinations producing similar aggregate behavior |
Reflections
This project represents methodological innovation under constraint. When historical data became unreliable, the solution was not more sophisticated analysis of bad data, but building a generative model from behavioral first principles. The agent-based approach traded data requirements for modeling complexity — a trade-off only viable because the simulation could be validated against pre-COVID aggregate patterns.
The hybrid implementation (Java performance, Python optimization) reflects pragmatic engineering: use the right tool for each layer of the stack. The result was a capability-preserving solution that allowed marketing planning to continue through an unprecedented disruption.