Manage Your Experiments: What to A/B Test First on Amazon

Key takeaways: Manage Your Experiments (MYE) is Amazon's free A/B testing tool for brand-registered sellers. It shows two versions of a title, main image, bullet points, product description or A+ Content to different shoppers for 4 to 10 weeks and reports which one sold more and how confident it is. Test the main image first, because it has the largest effect on clicks and sales; then the title; then A+ Content; then bullets. Change one thing per test and let it run to the end.

What Manage Your Experiments is

MYE lives in Seller Central under Brands. You pick an eligible ASIN, choose the element to test, supply version B, set a duration, and Amazon splits the ASIN's traffic between A and B. At the end it reports units, sales and conversion for each version, with a probability that one beats the other. You then publish the winner in one click.

It is the only way to test a listing change on real Amazon traffic with a real control group. Before-and-after comparisons are confounded by seasonality, competitors and your own ads; MYE runs both versions at the same time to the same population, which removes most of that noise.

Requirements

• Brand Registry. The ASIN must belong to your registered brand.
• Enough traffic. Amazon marks ASINs as eligible when they have had sufficient recent sessions to produce a result. Low-traffic ASINs do not qualify, and the threshold is not published; the tool shows eligibility per ASIN.
• A compliant version B. The variant must meet the same listing and A+ policies as the original. A test with a prohibited main image is rejected, not run.
• For A+ tests, the new A+ version must be approved before the experiment can start, so build it a few days ahead.

Tests run for a duration you choose, typically between 4 and 10 weeks. Amazon can end a test early if it reaches high confidence.

What you can test

What to test first

1. Main image. It acts before the click, so it affects every session the ASIN gets, paid and organic. A main image test frequently produces the largest sales difference of any test, in either direction. Version B ideas that stay inside Amazon's rules: a different angle, the product filling more of the frame, the pack or included items shown, product out of its box versus in it. Why the main image matters this much is in How to Improve Amazon Click-Through Rate (CTR).

2. Title. Test the order of the first 80 characters, not the whole string: lead with the attribute versus lead with the product type, digits versus words, variant early versus late. Title tests read cleanly because the change is visible in results.

3. A+ Content. The biggest conversion lever below the fold, and the one most worth testing because a build is expensive. Test a new A+ against the current one, or a Premium version against the standard one where you are eligible. Comparison chart versus no comparison chart is a common first A+ test.

4. Bullets. Test the objection-ordered rewrite against the original. Effects are smaller than image and A+ tests, so these need the full duration.

Description tests come last; the field is far down the page and rarely moves the number on its own.

Running a clean test

• One variable. A new image and a new title in the same test tells you a combination won, not which part. Run them as two tests.
• Make version B different enough to matter. Changing one word produces a result you cannot read. Test a real alternative hypothesis.
• Do not change anything else on the ASIN during the test. Price, coupons, ad spend and inventory should hold steady. A coupon added in week three ruins the read.
• Avoid event weeks. Prime Day and Q4 traffic behaves differently; start tests so they end before the event or after it.
• Run to the end. Early results swing. Amazon's confidence figure is the signal; act on it when the tool says the result is reliable, not when week two looks good.
• Log every test. Hypothesis, versions, dates, result, decision. A year of logged tests is a manual for your category that no competitor has.

Reading the result

Amazon reports units, sales, conversion rate and sample size for each version, and a probability that B beats A. A probability near 50% means no meaningful difference; publish whichever you prefer and move on. A high probability with a positive difference means publish B. A high probability with a negative difference is also a win: you have learned what not to do without paying for it across the whole ASIN.

Also read the projected annual impact Amazon shows, with caution. It extrapolates the test period; a test run over a slow month understates the effect and one over a strong month overstates it.

When you cannot use MYE

For ASINs below the traffic threshold, or for elements MYE does not cover such as secondary images, fall back to a disciplined before-and-after: change one element, hold everything else, and compare four weeks of unit session percentage or click share against the four weeks before, excluding event weeks. It is weaker evidence, and we treat it that way. The measurement approach for conversion changes is in How to Improve Amazon Conversion Rate (CVR).

How we test creative before committing to it, and the gallery process the tests feed, are below.

B10How we test creative before we commitdecisions based on Amazon's own A/B testing rather than opinion — only winners go live.

B7The 7-slot image set — what each image is fora gallery where every image has a job: the main image wins the click; slots 2–7 remove doubt and win the sale. This is also why we ask for raw photos from several angles.