8 Amazing Experiments That Will Change How You Approach SEO

Want more people to find and read your content? Then you need to be visible on search engines like Google. That’s why search engine optimization (SEO) is so important—because search engines remain the web’s dominant traffic driver.

Even if you know little to nothing about SEO, Google has made two things clear:

• It’s important that other websites link to yours. Since the beginning, links have helped Google figure out how to rank all the content that exists on the web.
• You need to publish relevant content. If you want people to link to you, you need great content. And it needs to match the intent of the person searching for the things you sell or know lots about.

But Google is evolving.

Welcome to the Machine Learning Era

While still incredibly important for SEO, links and great content aren’t enough to guarantee top rankings on their own. That’s because Google is increasingly using machine learning to determine how web pages rank in search results.

Since Google started relying more on machine learning to power its search results, one new thing has become clear to me: user engagement metrics are changing the way you should approach SEO.

Machine learning systems such as Google RankBrain reward high engagement with better visibility. So you should focus on two user engagement metrics if you want better rankings:

1. Increasing click-through rate (CTR): This is the percentage of people who click on your listing in the search results.
2. Increasing dwell time: This is how long visitors stay on your website after clicking through from search results.

Here’s a simple diagram to explain how machine learning works.

This explanation isn’t just based on opinion or speculation (or what Google has said publicly). It’s based on me running multiple experiments and analyzing the surprising results.

Prepare to have your mind blown! Here are eight crazy SEO experiments that will change how you think about SEO forever.

8. Higher CTR = Higher Search Rankings?

If you’re looking for signs of machine learning and RankBrain at work, look no further than our organic search click-through rate experiment.

This click curve, based on analysis of 1,000 keywords from the Google Search Console for the WordStream site, clearly illustrates the zero-sum nature of Google’s search results. In each period, the pages occupying the most prominent positions got more and more clicks, while the lower positions got less and less.

Heading forward, Google will reward pages that attract engagement with higher positions, and push down pages with low engagement.

7. Does CTR Impact Rankings?

Google says they don’t use CTR as a direct ranking factor. But that could also be interpreted as an implicit acknowledgement that CTR can indirectly influence your rankings.

We wanted to find out. For our next experiment, we tried to isolate the natural relationship between CTR and ranking by taking the difference between an observed organic search CTR minus the expected CTR.

This chart illustrates the difference between what I term “unicorns” and “donkeys.”

• Unicorns: Pages that beat the expected average organic CTR for a given position were far more likely to rank in the top four positions.
• Donkeys: Pages that failed to beat the expected organic search CTR were more likely to appear in positions 6–10.

The good news: increasing your engagement by three percent should help move you one position higher in organic position, on average.

6. What Happens When you Make a Headline More Clickable?

More people click on it. Duh!

In this experiment, we decided to change the title of one of our pages (“Guerrilla Marketing: 20+ Examples & Strategies to Stand Out”). It was ranking in eighth position and had a CTR of one percent. A total donkey!

So, guided by a proven headline formula, we changed it to “20+ Jaw-Dropping Guerrilla Marketing Examples & Strategies.” Nothing else on the page changed—we didn’t add or edit any of the text, build any links, or add any images.

What happened?

Our retitled page jumped up to the fifth position in Google and the CTR increased to 4.2 percent.

And, oh yeah, organic traffic increased by 112 percent over five months!

Boosting your CTR is super valuable for SEO. In addition to getting more clicks, a higher CTR can help improve your rankings, which results in even more clicks.

5. Do Engagement Rates Impact Rankings?

You want searchers to stick around after they click on your result and land on your website. We know this is important because Google measures dwell time.

One big problem: we can’t measure dwell time—only Google can. So we have to look at other user engagement metrics such as time on site and bounce rate as a proxy for dwell time.

Our next experiment did just that, and we found some interesting results. Look at the “kinks” in these two graphs:

These graphs show that, on average, pages with lower bounce rates and higher time on site are more likely to appear in the top six organic search result positions. Pages with higher bounce rates and lower time on site, however, were more likely to rank seventh on the page, or lower.

4. Can We See the Impact of Time on Site on SEO?

Absolutely! And so can you.

We used the WordStream site for this experiment, looking at time on site and organic traffic before and after Google rolled out RankBrain. The results are stunning.

Before:

After:

Google figured out that some of our pages were ranking well, but had below average time on site. Those pages no longer rank or bring is nearly as much traffic.

This experiment really shows Google’s machine learning algorithms at work. Your traffic won’t vanish overnight. It will gradually disappear. If you have any pages with time on site that are below average, those are your most at risk pages.

3. How Does Google Pick Featured Snippets?

This is what a featured snippet looks like on Google:

But wait—why is Google using a featured snippet from a page only ranked in Position four (WordStream), instead of a page that ranks in Position one (Moz)?

To find out, I analyzed data for the 981 featured snippets WordStream has earned. As it turns out, content ranking as far back as Page seven of Google results can become a featured snippet:

Digging deeper into Google Analytics and Search Console data, it turns out featured snippets are not chosen based on traditional ranking factors, word count, or even where you rank.

Based on my analysis, featured snippets are awarded to pages that have an unusually high CTR and unusually high time on site.

2. Social Shares and Rankings: What’s the Real Relationship?

For years many people have wrongly assumed that one of Google’s ranking factors is the total number of social shares your content gets.

But there is a definite correlation between social shares and organic rankings. We ran an experiment to figure out the link. Here’s what we discovered:

Based on our analysis, it’s not about the total number of shares. It’s about how many people engaged with your content. High engagement rates correlate with high CTR, and vice versa.

My theory is that the same emotions that make people share content on social media also make people click on that content in search results. This is especially true for headlines with ridiculously high CTRs.

1. What’s the Relationship Between Engagement and Conversion Rates?

Getting people excited enough to click on your search result increases the odds that they will also ultimately convert (whether it’s completing a purchase, subscribing, completing a form, or taking some other type of conversion action).

We know this because we gathered and analyzed lots of data. Here’s just one example of what we discovered in multiple WordStream client accounts:

As this chart illustrates, on average, increasing your CTR by two times can boost your conversion rate by 50 percent.

If you have unicorn CTRs—20 or 25 percent—then your conversion rate will be around two percent. But if you have average or below average conversion rates—around one percent or less—you’ll also have average conversion rates.

See? Increasing your CTR increases your conversion rates. Pretty amazing!

What’s it All Mean for SEO?

User engagement is the new unicorn of ranking factors. Here are three key takeaways:

1. CTR impacts organic search ranking: If you want to reach a more prominent position in search results, you need to beat the expected CTR for that given position.
2. Dwell time impacts ranking: Dwell time validates the CTR and featured snippet eligibility. It can override other “traditional” ranking signals.
3. Machine learning is changing everything: It isn’t just Google search and SEO. Other machine learning systems reward high engagement. YouTube uses machine learning to recommend videos. AdWords ad targeting is entirely based on machine learning. Facebook uses it for its news feed and ad targeting. We’re living in a machine learning revolution.

For greater insight into just how well your SEO efforts are working, download The Comprehensive Guide to Content Marketing Analytics & Metrics eBook.