A/B test
A/B test (split test)
Email Marketing
An A/B test sends two or more versions of a message to small groups, measures which performs better, and sends the winner to everyone else.
What is A/B test?
An A/B test changes one thing, such as the subject line, offer or button, and lets results decide.
Pick the right measure: opens for subject lines (with care, because of Apple Mail Privacy Protection), clicks for content, and orders for offers.
In SmartyTap, email and text campaigns can test up to seven versions. You set the test size (10% to 100%), pick the winner by open rate, click rate or placed order rate (texts use placed order rate) and the test length (1 to 72 hours); the winner then sends to everyone else automatically. SmartyTap compares whole versions and doesn't run multivariate tests.
Example
A store tests "Free shipping today" against "Your weekend treat" on 20% of its list for 6 hours; the winner by clicks goes to the other 80%.
Why A/B test Matters
Each test replaces a guess with evidence, and the lessons carry into every campaign after.
A/B test FAQ
How big should an A/B test be?
Big enough that each version reaches a few hundred people. With a small list, test on a larger share or split the whole list.
Can I test more than one change at once?
You can, but you won't know which change made the difference. Change one thing per test.
Explore More Terms
Browse our complete glossary of e-commerce and subscription terms.
