In A/B testing, while our estimator of effect size is often unbiased, adding up all the significant effects turns out to be quite biased. This is called the curse of the winner, and I’ll explore how this harms many A/B testing programs, and show how controlling error rates does nothing to ensure our effect estiamtes are appropriate.