In today's Whiteboard Friday, we welcome Will Critchlow as he demonstrates a method of testing for both your top-of-funnel SEO changes and your conversion-focused CRO changes at once. So it's SEO testing plus CRO testing together. If you're only testing one, which is common — I mean most organizations are only testing the conversion rate side of things — it's perfectly possible to have a winning test, roll it out, and do worse. SEO testing So the way that we do an SEO test is we actually split the pages. What you do is you first of all set it up in the same way as an SEO test, because we're essentially starting with SEO at the top of the funnel. Googlebot = stateless requests So what I didn't show in this diagram is if you were running this test across a site section, you would cookie this user and make sure that they always saw the same treatment no matter where they navigated around the site. So because Googlebot is making stateless requests, in other words just independent, one-off requests for each of these of these pages with no cookie set, Google sees the split. Firstly, we can evaluate the relative performance in different directions, taking care that conversion rate applies generally across all channels, and so a relatively small drop in conversion rate can be a really big deal compared to even an uplift in organic traffic, because the conversion rate is applying to all channels, not just your organic traffic channel. We can test those different things, run sequential tests, try and take the best of the SEO tests and the best of the CRO tests and get it working together and crucially avoid those situations where you think you've got a win, because your conversion rate is up, but you actually are about to crater your organic search performance. We think this is going to just be the more data-driven we get, the more accountable SEO testing makes us, the more important it's going to be to join these dots and make sure that we're getting true uplifts on a net basis when we combine them.
Testing for only SEO or only CRO isn’t always ideal. Some changes result in higher conversions and reduced site traffic, for instance, while others may rank more highly but convert less well. In today’s Whiteboard Friday, we welcome Will Critchlow as he demonstrates a method of testing for both your top-of-funnel SEO changes and your conversion-focused CRO changes at once.
CC
Off
English
Click on the whiteboard image above to open a high-resolution version in a new tab!
Video Transcription
Hi, everyone. Welcome to another Whiteboard Friday. My name is Will Critchlow, one of the founders at Distilled. If you’ve been following what I’ve been writing and talking about around the web recently, today’s topic may not surprise you that much. I’m going to be talking about another kind of SEO testing.
Over at Distilled, we’ve been investing pretty heavily in building out our capability to do SEO tests and in particular built our optimization delivery network, which has let us do a new kind of SEO testing that hasn’t been previously available to most of our clients. Recently we’ve been working on a new enhancement to this, which is full funnel testing, and that’s what I want to talk about today.
So funnel testing is testing all the way through the funnel, from acquisition at the SEO end to conversion. So it’s SEO testing plus CRO testing together. I’m going to write a little bit more about some of the motivation for this. But, in a nutshell, it essentially boils down to the fact that it is perfectly possible, in fact we’ve seen in the wild cases of tests that win in SEO terms and lose in CRO terms or vice versa.
In other words, tests that maybe you make a change and it converts better, but you lose organic search traffic. Or the other way around, it ranks better, but it converts less well. If you’re only testing one, which is common — I mean most organizations are only testing the conversion rate side of things — it’s perfectly possible to have a winning test, roll it out, and do worse.
CRO testing
So let’s step back a little bit. A little bit of a primer. Conversion rate optimization testing works in an A/B split kind of way. You can test on a single page, if you want to, or a site section. The way it works is you split your audience. So your audience is split. Some of your audience gets one version of the page, and the rest of the audience gets a different version.
Then you can compare the conversion rate among the group who got the control and the group who got the variant. That’s very straightforward. Like I say, it can happen on a single page or across an entire site. SEO testing, a little bit newer. The way this works is you can’t split the audience, because we care very much about the search engine spiders in this case. For the purposes of this consideration, there’s essentially only one Googlebot. So you couldn’t put Google in Class A or Class B here and expect to get anything meaningful.
SEO testing
So the way that we do an SEO test is we actually split the pages. To do this, you need a substantial site section. So imagine, for example, an e-commerce website with thousands of products. You might have a hypothesis of something that will help those product pages perform better. You take your hypothesis and you only apply it to some of the pages, and you leave some of the pages unchanged as a control.
Then, crucially, search engines and users see the same experience. There’s no cloaking going on. There’s no duplication of content. You simply change some pages and not change others. Then you apply kind of advanced mathematical, statistical analysis trying to figure out do these pages get statistically more organic search traffic than we…
COMMENTS