Stop guessing
Compete your emails. Mumara picks the winner.
Pick two or more broadcasts. Pick a metric — open rate or click-through. Send each variant to a slice of your audience. Mumara measures, declares the winner, and ships the winning version to the rest of your list automatically. Set up the test in a minute; the test takes care of itself from there.
- Multivariate, not just A/B
- 2 winning metrics built in
- Auto-rollout to the leftover audience
- Configurable sample + duration
What you get
The split-test feature, in numbers.
Built around the constraints real testing actually has — sample sizes, time-bound experiments, multiple variables, and the moment you need the winner to ship without you clicking anything.
- Variants per test
- A · B · C · D
- Variants per test
- Not just two. Compete as many email broadcasts as you want in one test.
- Winning metric
- Either
- Winning metric
- Open rate or unique click-through rate — pick the one that defines the win for this test. More metrics added as the engine grows.
- Sample size
- 5–100%
- Sample size
- Test on a slice of your list, or split everyone equally. Sliding scale.
- Rollout to leftover
- Auto
- Rollout to leftover
- Winner declared, leftover audience receives it — no manual deploy step.
The setup
Configure once. The test runs itself.
Name the test. Pick the broadcasts you want to compete. Choose Open Rate or Click-Through Rate as the winning metric. Set the sample percentage and how long to wait. Decide whether the winner gets sent to the leftover audience automatically or you just want the data. Save. The test takes it from there.
-
Sample-size guidance built in
Mumara recommends test percentages by list size: 50-100% under 1,000 contacts, 20-30% from 1,000-10,000, 10-20% up to 100,000, 5-10% above. Default sensible; override anytime.
-
Duration in your unit of choice
Minutes, hours, days, weeks. Open-rate tests typically run 4-24 hours. Click-rate tests want 24-48 hours to stabilize. Configurable per test.
-
Determine-only or auto-rollout
Two modes for the post-test action. 'Find winner and update results' if you just want the data; 'Find winner and send leftover to it' if you want the campaign to finish itself.
What to test
Six things teams test most often.
Pick one variable per test — that's the rule. Mixing variables muddies the result and you never know what actually moved the metric.
-
Subject lines
- Test
- Short vs long, question vs statement, emoji vs no emoji.
- What you learn
- Which phrasing actually drives opens for THIS audience — not just industry averages.
-
Sender identity
- Test
- Brand name vs personal name vs role (e.g. "CEO of Mumara").
- What you learn
- Whether your audience opens for the brand or for the person behind it.
-
Call-to-action
- Test
- Button copy, button colour, button position in the email.
- What you learn
- Which exact CTA gets the click. "Shop now" vs "Learn more" can move CTR by double-digit percentages.
-
Offer presentation
- Test
- Percentage discount vs dollar amount, bundled vs single product.
- What you learn
- Which framing your audience actually finds more compelling. Surprising answers.
-
Content layout
- Test
- Image position, email length, copy density, layout grid.
- What you learn
- Whether shorter beats longer for your list — and where the hero image really belongs.
-
Send timing
- Test
- Tuesday morning vs Thursday afternoon, weekday vs weekend.
- What you learn
- When your specific audience actually engages. Industry rules don't survive contact with your data.
The mistakes that hurt
Why most teams stop testing.
The problems aren't with testing. They're with testing badly — and then concluding testing doesn't work.
-
Testing several variables at once
Different subject AND different CTA AND different send time. The winner won across the board, but you can't say WHY — and the next test has nothing to build on.
-
Samples that are too small
200 contacts over 30 minutes won't tell you anything statistically useful. The difference you see is noise. You ship the 'winner' and the result doesn't replicate.
-
Running tests once, then never again
A/B testing isn't a one-time project. The audience that opened your March test isn't the same audience opening your November send. The work is ongoing — and most teams quit after the first round.
-
Determining a winner, then forgetting to deploy it
The test reports back. You read the results. You move on. Three weeks later you realize the campaign never sent the winner to the rest of the list. Mumara's auto-rollout option exists for exactly this.
“I've managed email campaigns for years and have used almost every major platform out there. Mumara gives me the level of control that I've been missing. The segmentation capabilities are incredibly detailed, and the automation workflows are robust and easy to map out. A professional-grade tool for serious marketers.”
James
Verified Trustpilot reviewer
Common questions
What buyers usually ask.
Is this just A/B, or can I test more variants?
Multivariate by default. The setup form's Campaigns field is a multi-select — pick two, three, four, however many broadcasts you want to compete. The sample audience is split equally among the variants and the winner is determined the same way.
What metrics decide the winner?
Two — Open Rate and Unique Click-Through Rate. Pick which one defines the win for each test. Open-rate-driven tests are typically about subject lines, sender, and preheader. Click-rate-driven tests are typically about the body, CTA, and offer.
How big should the sample be?
Mumara surfaces sample-size guidance in the builder, scaled by list size: 50-100% under 1,000 contacts, 20-30% from 1,000-10,000, 10-20% up to 100,000, 5-10% above. These are the docs' recommended starting points. Smaller samples produce noisier results.
What if the test runs at 100%?
Then everyone in the audience is split equally among the variants. The winner is still determined — but there's no 'leftover' audience to roll out to. The 100% mode is for pure measurement when you want every recipient included in the experiment itself.
How long should I let a test run?
Mumara's guidance: open-rate tests typically need 4-24 hours to stabilize; click-rate tests want 24-48 hours. Configure duration in minutes, hours, days, or weeks. Too short and you measure noise; too long and you've delayed the rest of your send.
What happens after the winner is declared?
Two modes you choose at setup. 'Find winner and update the results' — Mumara records which variant won; you take the data and decide manually. 'Find winner and send leftover to it' — Mumara automatically sends the winning variant to the rest of your list as soon as the test concludes. Set it up once; the campaign finishes itself.
What if the results are inconclusive?
It happens — variants performing within margin of error. Mumara surfaces this state explicitly rather than picking arbitrarily. The recommendation is to increase the test percentage, extend the duration, or run against a larger list and retest. The product is honest about the limitation rather than pretending there's a winner when there isn't.
Can I edit a test once it's running?
No — running and completed tests are locked. This is intentional. Editing mid-run would invalidate the experiment. Build the test deliberately, save, run.
Does Mumara compute statistical significance?
Mumara picks the winner by which variant scored higher on the chosen metric in the sample. There isn't a built-in p-value / confidence-interval computation — so this is a practical pick-a-winner test, not a rigorous statistical test. For larger samples that matters less; for smaller samples treat results with appropriate skepticism. Worth knowing.
Related
Pages that pair with this one.
-
Broadcasts
The variants in a split test ARE broadcasts. Build 2-5 versions, then compete them.
Read more -
Segments
Run the test against a specific segment for tighter signal — engaged users behave differently than dormant ones.
Read more -
Triggers
Once you find a winning broadcast, set it as the action on a trigger — the test result becomes the production workflow.
Read more
Mumara Campaigns
Set up your first split test in under a minute.
Split tests ship on every Mumara Campaigns tier — Self-Hosted and Mumara Machine. No credit card to try. Compete the two emails you've been debating internally; ship the winner.