Live Cell Tracking in Oscillating Intensity Images

Rice University (PhD Thesis) • 2009-2011

C++ Computer Vision Particle Filters Optical Flow Finite Elements Simulation Qt Eigen OpenMesh

Research Problem

Biological cell tracking in microscopy images is complicated by oscillating intensity — fluorescence levels that vary over time due to photobleaching, focus drift, or staining dynamics. Traditional tracking algorithms assume consistent appearance, causing them to fail when cell brightness changes significantly during observation.

Challenge: Develop a tracking algorithm robust to intensity oscillations while maintaining sub-pixel accuracy for biological research applications.

Technical Approach

Core Algorithm: Particle Filter with Optical Flow

I designed a two-stage tracking algorithm combining sequential Monte Carlo methods with region-based optical flow:

Stage 1: Optical Flow Estimation

  • Region-based optical flow calculation between consecutive frames
  • Robust to local intensity variations through normalized correlation
  • Provides proposal distribution for particle filter

Stage 2: Particle Filter Tracking

  • Sequential Monte Carlo (Particle Filter) framework for state estimation
  • State space: cell position, velocity, and deformation parameters
  • Observation model: intensity-aware similarity metric accounting for oscillations
  • Resampling: adaptive resampling to maintain particle diversity

Validation via Finite Elements Simulation

Since ground truth for live cell tracking is difficult to obtain, I built a finite elements simulation to generate synthetic test data:

  • Physics-based cell deformation model using finite elements
  • Controlled oscillating intensity patterns
  • Known ground truth for quantitative evaluation
  • Enabled systematic testing of algorithm robustness

Implementation

Software Architecture:

  • Language: C++ (cross-platform, ~25,000 lines)
  • GUI Framework: Qt for visualization and parameter tuning
  • Numerical Computing: Eigen (matrix operations), ANN (approximate nearest neighbor), FIGTree (fast Gauss transform)
  • Mesh Processing: OpenMesh for finite elements simulation
  • Build System: Cross-platform compilation (Linux/Windows/Mac)

Key Components:

Module Purpose Lines of Code
Optical Flow Engine Intensity-robust motion estimation ~6,000
Particle Filter Sequential state estimation ~8,000
FEM Simulation Synthetic data generation ~7,000
GUI & Visualization Qt-based interface ~3,000
Testing & Metrics Evaluation framework ~1,000

Technical Challenges

Challenge Solution
Intensity oscillations Observation model with adaptive intensity normalization
Particle degeneracy Adaptive resampling based on effective sample size
Real-time performance Optimized C++ with fast Gauss transform approximations
Validation without ground truth Physics-based FEM simulation with known trajectories
Sub-pixel accuracy Bilinear interpolation in optical flow calculation

Research Outcomes

  • Novel algorithm: First particle filter-based tracker explicitly designed for oscillating intensity microscopy
  • Validation methodology: Demonstrated that synthetic FEM data effectively predicts real-world performance
  • Open source: Cross-platform implementation enabling adoption by biology researchers
  • Accuracy: Sub-pixel tracking maintained despite 50%+ intensity variation

Skills Demonstrated

  • Sequential Monte Carlo methods: Particle filters for state estimation in stochastic systems
  • Computer vision: Optical flow, image registration, feature tracking
  • Physics simulation: Finite elements for deformable object modeling
  • Software engineering: Large-scale C++ architecture, cross-platform development, numerical optimization
  • Research methodology: Problem formulation, algorithm design, validation, quantitative evaluation

Connection to Professional Work

This thesis established foundations I later applied in industry:

  • Particle filters → Agent-based simulation: Both use Monte Carlo methods for complex system modeling
  • Physics simulation → Marketing mix simulator: Finite elements and agent-based models both require simulation validation
  • C++ performance optimization: Directly informed the Java-based Disney simulator where runtime was critical
  • Research rigor: Validation methodology influenced how I approached model evaluation at Disney and Aven

Reflections

This project taught me that simulation and state estimation are universal tools. Whether tracking cells in microscopy or optimizing marketing spend during COVID, the core challenges are similar: incomplete observations, stochastic dynamics, and the need for principled validation when ground truth is unavailable. The 25,000 lines of C++ also taught me respect for software engineering — research code that can’t be maintained or reproduced isn’t science.