How long should I let my test run?

rawpixel-633849-unsplashTheoretically you should generally calculate the sample size you need prior to running a test and run it for only that long. No longer. No shorter.

Here are some of my favorite tools from Evan Miller.

If you didn’t plan ahead, then run the test for at least one buyer-cycle or a reasonable timeframe to rule out seasonal effects.

If you aren’t able to collect enough samples during a test, feel free to run the test again and do a meets like meta-analysis.

As an aside, I’d avoid MVT tests as they are typically not focused on answering any specific hypothesis, thus, generating fewer learnings other than a certain combination worked.

About Rommil Santiago

A Toronto-based growth marketer looking to improve the world.
%d bloggers like this: