Page Contents
Last updated: October 2026
CRO testing is one of the best ways to learn what makes your visitors act. But a test is only as good as its preparation. Teams that jump straight into A/B tests often end up with unclear results, fights about what the numbers mean or winners that do not hold up after launch. A few days of groundwork make every CRO testing program faster and more reliable.
This guide explains what CRO testing is, why it matters, which goals to measure, five preparations to make before you start and how to build a CRO roadmap. Where figures come from secondary sources, we say so.
What Is CRO Testing?
CRO testing is the practice of running controlled experiments on your website, app or emails to see which version gets more conversions. In its simplest form, an A/B test splits traffic between the current version (the control) and a changed version (the variation) and compares how many visitors in each group complete the goal. Other methods include multivariate tests, which change several elements at once (see multivariate testing), split URL tests for bigger redesigns and qualitative tests such as user interviews.
Testing is one part of the wider conversion rate optimization process. Research tells you what to test, and testing tells you what works.
Why Is CRO Testing So Important?
- It replaces opinions with evidence. Teams often disagree about design and copy. A test settles the question with real visitors.
- It lowers risk. You can try a change on a share of traffic before rolling it out.
- It compounds. Each result teaches you something about your audience that guides the next idea.
- It protects revenue. Some well-meant changes hurt conversion, and a test shows it before everyone sees it.
Expect many tests not to win. A 2014 VWO survey of its customers found that about one in seven A/B tests produced a winner, and other analyses report higher shares depending on how a win is defined. Learning from every result, including the losers, is the real return.
Goals to Measure While CRO Testing
Decide before the test what success looks like.
- Primary goal: the single metric that decides the winner, such as completed purchases, qualified leads or sign-ups. Keep it close to revenue.
- Secondary metrics: numbers that help you interpret results, such as revenue per visitor, average order value, bounce or engagement and form starts.
- Guardrail metrics: numbers that must not get worse, such as page speed, refunds and customer satisfaction.
- Micro conversions: smaller steps such as clicks and video plays that explain why a change worked. See micro conversions.
In Google Analytics 4, mark your main goal as a key event (the name Google now uses for conversions) and keep event names consistent across variations.
5 Preparations Before Using CRO Testing
1. Know Your Visitors Deeply
Good tests start with understanding. Review who your visitors are, where they come from, what devices they use and where they struggle. Combine analytics with qualitative research:
- Session recordings to watch real visits.
- Heatmaps to see clicks and scrolling.
- Surveys and user feedback to hear the reasons in visitors’ own words.
Use this to write specific hypotheses, such as “visitors on mobile do not see the pricing link.” For personalization ideas, see website personalization.
2. Measure Micro Conversions
Final conversions can be rare, so tests on them take longer. Smaller actions happen more often and show where people stall. Track steps such as product views, add-to-cart clicks, form starts and video plays. They give you earlier signals and help explain results, although you should not declare a winner on them alone.
3. Make Sure Your Design Reflects Your Brand
Every variation should still feel like your company. Before testing, set down your brand rules for color, type, tone and layout, so test ideas do not drift into off-brand pages that confuse returning visitors. Design consistency also makes results easier to compare. Our guide to CRAP design principles gives a quick way to review contrast, repetition, alignment and proximity.
4. Document Everything
A test log prevents repeating mistakes and helps new team members learn. For every test, record:
- The hypothesis and the research behind it.
- The pages, audience and devices included.
- The primary and secondary metrics.
- The baseline conversion rate, expected lift, sample size and planned duration.
- Start and end dates, and anything unusual that happened, such as a campaign or outage.
- The result, the decision and the lesson.
A shared spreadsheet or a simple project board is enough to start.
5. Do CRO Mapping and Check That Tracking Works
List the key journeys on your site, such as home page to product to checkout, and note the conversion rate and the main problems at each step. This map shows where tests will matter most. See funnel analysis and buyer journey mapping. Then verify tracking: test every event on every variation before launch, make sure traffic is split evenly and exclude your own team’s visits. Broken tracking is a leading cause of results that mislead.
Can Your Test Finish? Check Sample Size First
Before building anything, estimate whether you have enough traffic. A common approximation, at 95% significance and 80% power, is n per variant ≈ 15.7 × p × (1 − p) ÷ d², where p is your baseline conversion rate and d is the absolute lift you want to detect. At a 2.0% baseline, detecting a 5% relative lift needs roughly 307,700 visitors per variant, a 10% relative lift needs about 76,900 and a 20% relative lift needs about 19,200. This is a rough guide, and calculators differ slightly, but it shows why small lifts on low traffic are hard to prove. If your numbers are small, test bolder changes, run longer (low-traffic sites often need four to eight weeks) or rely on qualitative research.
Plan the duration in advance as well. Most sources recommend at least one to two full business cycles, often two weeks, so that weekday and weekend behavior are included, and warn against stopping as soon as a result looks significant. See landing page testing mistakes for more on early stopping.
What Is a CRO Roadmap?
A CRO roadmap is a plan for your next three to six months of testing. It links business goals to the metrics you will move, the hypotheses you will test and the order of work. A good roadmap is flexible, because results will change your priorities.
How to Build a CRO Roadmap
1. Determine Your Business Goals
Start with outcomes the business cares about, such as more qualified leads, higher average order value or lower cost per acquisition. Everything on the roadmap should support one of them.
2. List Metrics and Strategies for Each Goal
For every goal, name the metric that shows progress and the strategies that could move it, such as simplifying checkout or clarifying pricing. See website metrics for ideas.
3. Set Your Current State and Baseline
Record today’s numbers for each key page and journey, by device and traffic source. The baseline is what you will compare against, and it feeds your sample size estimate.
4. Create Test Hypotheses
Write each idea as “If we change X, then Y will happen, because Z,” with evidence from research. Aim for a list of ten to twenty ideas.
5. Prioritize Your Hypotheses
Rank ideas so you test the most promising ones first. Popular scoring methods include ICE (impact, confidence, ease) and PIE (potential, importance, ease), which are quick but subjective, and PXL, which uses mostly yes-or-no questions to reduce subjectivity. Add a feasibility check, because a high-scoring idea is no use if the test cannot finish on your traffic. For more, see how to build a CRO program.
Tools You Will Need
- Analytics for traffic, events and funnels, such as GA4.
- Behavior tools for recordings, heatmaps and goal tracking.
- An experimentation tool to run tests. Google Optimize shut down on September 30, 2023, and GA4 has no built-in testing, so you need a third-party platform.
- A test log for documentation.
Compare options in our list of CRO tools, and run a quick CRO audit to see where to start.
Frequently Asked Questions About CRO Testing
What is CRO testing?
CRO testing is running controlled experiments, usually A/B tests, to find which version of a page, message or flow gets more conversions.
What should I do before running a CRO test?
Research your visitors, set goals and metrics, verify tracking, document your plan and check that you have enough traffic to finish the test.
How long should a CRO test run?
Calculate the sample size first, then run for at least one to two full business cycles, often two weeks. Low-traffic sites may need four to eight weeks.
How much traffic do I need for CRO testing?
It depends on your baseline conversion rate and the lift you want to detect. Smaller lifts need far more visitors. Use a sample size calculator before you start.
What is a CRO roadmap?
It is a prioritized plan for upcoming tests, linking business goals, metrics and hypotheses over the next few months.
Conclusion
CRO testing works best when the preparation is done well. Know your visitors, track micro conversions, keep designs on brand, document every test and map your journeys, then check your sample size before you build. Turn your goals into a roadmap of prioritized hypotheses, test one change at a time and record each lesson. With that foundation, every test, win or lose, makes your next decision better.
