The Peeking Problem
[ Live ]Checking your A/B test results early feels harmless. A live simulation of what it actually costs you, in both frequentist and Bayesian terms.
- [ Statistics ]
- [ TypeScript ]
- [ Data Viz ]
Staff Data Scientist at Intuit. 10+ years in FinTech, working across full-stack analytics and experimentation.
About
I'm the person teams come to for experimentation platform expertise: building the tooling, defining the guardrails, and helping teams make sense of a messy result. Lately I've been picking up causal inference too, for the questions a randomized test can't answer.
Read the full story →Projects
Checking your A/B test results early feels harmless. A live simulation of what it actually costs you, in both frequentist and Bayesian terms.
How much should you trust surfboard volume calculators? A comparison of how thirteen recommendation systems disagree for the same surfer.