What Landing Page Quality Score is Really All About

A year ago, in October 2011, Google announced a change to how Quality Score was calculated. Since then, landing page quality is supposed to play an important role the ad auction, where ad ranks and click prices are determined. It was easy to miss that this was actually a fundamental change – one that didn’t add up.

In my opinion, Google has once again given priority to marketing and neglected to tell the real purpose behind this change. In consequence, advertisers got a completely wrong picture of what landing page quality is and what they’re supposed to do. Basically, all conclusions and strategies to deal with landing page quality are utterly wrong.

With this blog post, I’d like to take you on a little tour. First, we’ll look at what changed and where things don’t add up. Then we’ll follow the money to find a better explanation for things. Finally, we’ll look at the consequences and what you can do to optimize for landing page quality.

A little Paradigm Shift?

It all started with an innocent-sounding announcement from Google. They were going to “increase” the importance of one ingredient of Quality Score:

“the weight given to relevance and landing page quality in determining Quality Score and how ads are ranked on Google.”

They also hinted that this wasn’t such a big deal. Well.

This was actually a paradigm shift. Before this change, landing page quality wasn’t considered in the ad auction at all. This used to be a major misunderstanding about Quality Score: landing page never mattered when rankings and click prices were determined. Google never made much of an effort to state this clearly, but it used to be in the “fine print” in the Quality Score documentation. Well, now landing pages play a role when ad positions and prices are determined.

And there is a second change: landing page quality can help you now. This, too, used to be different. In the past, a bad landing page could get you in trouble, but there was no reward for a good or a great one. It used to be purely a penalty factor. Or, as Andrew Goodman, who apparently also talked to Alferness, put it:

In the past, landing page quality was usually administered as if it were “policy”: that is, there was an attempt to look for what Alferness calls “negative signals”.

So now they look for positive signals, too. And now those can help you with your ad rank.

Strong and Slight Boosts

Reading further beyond Google’s official announcement, I found two more interesting sources with information about this. On Search Engine Land, Pamela Parker quoted Google’s Jonathan Alferness. Alferness confirmed that advertisers with better landing pages will rank better and will get a “strong boost” in ad auctions.

George Michie also wrote an article on the RGB blog, dealing in great detail with the changes about landing page quality. He, too, got to talk to Alferness. This time, Alferness only spoke about a “slight boost” and George pointed out that Google’s official announcement even states the expectation that “most campaigns will not see a significant change in overall performance”. As it turns out, advertisers really didn’t see any change over this – if they did, they kept it to themselves.

Making Users Happy

What I find very interesting is how Alferness stressed a familiar point in the interview with George Michie:

The intent is to improve user experience.

Alferness goes on and makes exactly the same argument as Google always has: They need quality ads, because otherwise users wouldn’t be happy. But Google wants to make users happy so that they come back. According to Alferness (and Google in general), it’s not about short term profits – it’s a long term strategy to make users happy and get them to come back.

This sounds reasonable and it’s a great marketing story. It’s no surprise that Google sticks to this, no matter what. But the thing is, this also made perfect sense back in 2005 when Google first introduced Quality Score and made the exact same argument: that it’s all about making users happy. But for six years Google was perfectly happy with leaving landing pages out of the ad auction. And they also didn’t have a problem with advertisers believing the opposite, since this technicality was somewhat hidden in the fine print.

So when the reasoning hasn’t changed since 2005, then why did Google wait until 2011 to do something about it?

Testing a Long Term Strategy

There’s another weak point in Google’s story: They tested the effects of the change in first. Remember that Google stated that it’s a long term strategy and that it had little actual impact on advertisers. So how do you test the long term effects of a little change that would sometimes result in a slightly different ad order? For Google, this apparently wasn’t much of a problem. Let’s look at the time table:

That’s odd, isn’t it? They managed to test the effects of a little tweak on long term loyalty of users – and they did it two months (one month outside of Brazil). And that included “one to two weeks” until the experiment was rolled out and everything had stabilized. And the time to evalute things. And the time a company like Google needs to decide on a major change about their primary revenue source.

(It’s also worth noting that Google had and has a market share of more than 90% in Latin America, Spain, and Portugal. Looks like the room for improvement was rather limited.)

Following the Money

So some things don’t really add up. So how can we understand this move from Google? Following the money is often a good idea. In fact, I already did this a series of articles on what Quality Score actually is. Back then, early 2011, I determined what (auction) Quality Score had to be in order for Google to maximize revenue. The conclusion was:

Quality score is Google’s estimate for click-through probability.

Sounds strange, I know. You can call it “predicted CTR” if you think that’s catchier. But labels aside: that’s what Google needs to do: make an estimate of the chance an ad gets clicked – because that’s the chance they get paid.

Putting the highest bidder in the top spot, regardless of the chance the ad actually gets clicks, is a bad strategy (unless you don’t care about revenue). In order to rank ads correctly, Google has to look at the pay (bid) and the chance they get paid.

Of course, estimating the chance of a click-through is an incredibly hard thing to do, especially if you don’t have much historical CTR data to go on. But it pays: The better Google’s estimate is, the more money they make. Improve the estimate a tiny bit might just get Google a fraction of a cent per auction – but with billions of ad auctions each year, it adds up to a big deal.

So what does this have to do with landing page quality? Well, nothing – and that’s the problem. Because in order to determine click-through probability (a.k.a. maximizing revenue), the landing page is useless. Back when I first published the article about Quality Score, this was an important point: Had Google wanted to make Quality Score about user experience, they would’ve incorporated landing pages right away. Instead they didn’t, but only mentioned this in the fine print.

Now the problem is: If you dilute Quality Score with anything else besides click-through probability, the outcome is farther away from the revenue maximizing optimum, meaning the result is less money for Google. So why include landing pages when they don’t have anything to do with whether anyone clicks an ad? After all, the user only sees the landing page after the click.

Although…

Wait a minute… let’s check something…

ip full car insurance

Oops, look at that – there’s a landing page. Visible before the click. So much for that.

Looking at the preview in the screenshot, I notice that the ad promises £30 cash back and the preview clearly picks up that promise. A potential customer who was interested in the ad before would probably click now – and Google would make some money.

So we’re back at the money. As I wrote above, Google needs to know the likelihood of getting paid (a.k.a. click-through probability a.k.a. Quality Score) as precisely as possible. The better the estimate, the more money Google can make. So if knowing about the effects of Instant Preview on click-throughs can improve the estimate, it would be leaving money on the table to ignore this.

This is why I believe that landing page quality is actually about the quality of the Instant Preview.

Is that right?

I realize that this contradicts something that most people had long accepted. The reasoning so far is based on common sense and the assumption that Google wants to maximize revenue. I’d say that’s enough, but let’s see what else we can find.

I mentioned some odd things about the official version about landing page quality. There was the issue with the strong or slight boost that no one seems to have felt. Considering how often Instant Preview is (probably) used and that it’s not available for all ads, I’d say that’s no surprise.

Then there’s the issue why landing page quality started to matter in 2011, after it was basically ignored for six years. But if you consider that Instant Preview for ads was first introduced in April 2011, it suddenly makes perfect sense.

The oddly short test also makes sense with regard to Instant Preview. Testing the long term effects of slightly different ad orders on user loyalty in a short time and without much room for improvement sounds implausible to me.

But if it’s all about improving click-through probability estimates, it’s much easier. Google could basically just have done a series of A/B tests to find out what method of considering landing page factors made them the most money. Unlike long term user loyalty, click-throughs and ad revenue can be measured easily and immediatly.

What do we do with this?

Advertisers understandably want to leverage landing page quality to improve Quality Scores. George Michie got a few general statements out of Alferness, which can help us understand what Google looks at to determine landing page quality. According to him, Google uses the following basic criteria:

  • Relevance
  • Content
  • The user’s response

Since it’s incredibly hard for a machine to analyze a landing page’s quality, advertisers have translated those factors into criteria that Google could evalute. Relevance is commonly interpreted in terms of on-page SEO factors, which in turn is interpreted as the necessity of having keywords on the landing page. Google employees have often denied that there is any benefit of keywords on a landing page, but as far as I know, there is no official statement.

With content, things are also unclear. There are some Google guidelines about landing page experience which include relevant and original content, transparency, and ease of navigation. Not all of the cited aspects and examples can be evaluated algorithmically, which suggests that at least some of those guidelines have nothing to do with Quality Score.

As far as evaluating user responses to assess landing page quality is concerned, bounce rate is usually the factor that comes to mind. As with keywords, Google employees have occasionally denied that bounce rate plays a role in evaluating landing page quality.

Okay, now let’s look at those things with Instant Preview and the problem of estimating click-through probabilities in mind. In order to estimate the probability behind Instant Preview click-throughs, the most significant and reliable thing to look at is click-through data from the past – just like general click-through data is the most reliable part of general click-through probability estimates (or, in Google’s words: CTR is the most important factor in Quality Score). So there we already have the user response to look at: It’s not bounce rate, but Instant Preview click-through rate.

It stands to reason that all other criteria (like relevance and content) are secondary when it comes to estimating click-through probability for previews. Again, it’s the same as with CTR and Quality Score in general: In the absence of enough historical click-through data, Google has to look at other factors (like relevance). But the more first-hand experience Google has with an ad’s performance, the influence of those factors fades away. For example, if Google has shown an ad a million times with abysmal click-through rates, it would be foolish to look at relevance and conclude that the ad might do great in the future. The same is true with Instant Previews.

In consequence, even if you could optimize a landing page with regards to relevance and content, the positive effect of such optimizations would fade away once Google starts to see how users actually react to your previewed landing page.

Understanding Landing Page Quality

When optimizing a landing page with regard to Quality Score and Instant Preview, there are short term and long term criteria. Short term criteria are all those things that Google might look at to algorithmically evaluate a landing page. But mid to long term it’s all about the user’s reaction to the preview.

To understand the importance of the criteria, let’s look at a few examples. I’ve searched for a ski vacation and found three interesting landing page previews:

ski-previews

The first landing page is from snow.com. It’s highly relevant, has the keywords, and a lot of text. From an SEO perspective, the page looks good. If we look at Google’s landing page guidelines about relevant and original content, transparency and ease of navigation, the page is good, too. So if Google really looks at those criteria, this page should be off to a great start. However, the preview looks kind of boring and if users don’t react well on it (by clicking the ad), the effect on Quality Score will be negative as soon as Google realizes how little users like the preview.

The second page from Ski Bundle is a little more engaging but has a lot of text, too. Google has highlighted a snippet with keywords. You can also see a promotion (the “Banff Bundle” for $780), which could be a good thing. But in this case, it’s more of a problem, since the ad was for ski vacations in general and the promoted bundle is just a random part of a rotating set of promotions. To make matters worse, the promotion is actually no longer part of the rotation and nowhere to be found on the landing page, but it’s still part of the preview.

The third landing page from SnowTrex has a lot of images and is much less text heavy in comparison. There are no promotions standing out and no keywords visible, but it’s obviously about ski vacations and it seems to offer a lot of choices without requiring a lot of reading. Of the three examples, I’d say that most people would probably click on this one, which would perfectly underscore how little keywords and textual content matters to the actual human user – because the page the preview is from is actually in German.

How to Optimize for Instant Previews

Now let’s look at how we can actually optimize a landing page with regard to Instant Preview. I believe it’s possible, although uncertain, that some on-page SEO can help a page initially, but the effect will be overridden quickly when Google starts to see how actual people react to the preview. This is why I believe that no investment in landing page SEO has a chance of ever paying off.

In order to truly improve landing page quality in the eyes of Google, you would have to improve the page so that actual users would want to click the ad after seeing the preview. Since we don’t get any data on Instant Preview click-through rates, A/B testing isn’t an option. But there are a few pointers:

  • Make sure that you can see the page is about the same thing as the ad. If you make a promise in the ad, consider if it should be visible in the preview.
  • Pictures, like product images, are easier to grasp and more appealing than textual content.
  • When you have a large number of landing pages, like an online retailer, having readable headlines is helpful. Product images might not tell everything, like that the page is only about a certain brand.
  • Some selling points or trust signals might be suitable to be visible in the preview, especially when they come in the form of logos.
  • Consider whether you have control over what shows up in the preview. Automatic slideshows and things like that can be a problem.
  • Make sure the preview is up to date. If you change something on the landing page, you can’t be sure that Google updates the preview in time. Consider using a new page or at least a new destination URL. Appending a meaningless parameter like “?lp=1” to the URL should be enough to make Google get a new preview screenshot.

These are just a few ideas and I’m sure there are a lot more. However, keep in mind that a landing page should always focus on getting users to convert. Sacrificing conversions for clicks would be a bad idea.

Still, some optimization made with regard to conversion rate can also be helpful with the Instant Preview. For example, making a page more accessible by getting rid of text and using attractive images instead could lead to improvements in both areas. And making sure the offers on the landing page match the ad’s promises is always a good idea.

Conclusion

Unless Google has designed the ad auction mechanism to leave money on the table, optimizing a landing page with regard to Quality Score means optimizing for Instant Preview so that people click the ad.

There are a few consequences that come with this conclusion. On the one hand, some commonly accepted strategies won’t work: Keywords on landing pages, or SEO in general, practically have no effect on Quality Score. And neither do other things you might consider quality, like page load time. Unless it affects click-throughs, it doesn’t count.

On the other hand, you can do some things to improve the preview itself, but if you want to optimize the layout of a landing page, better do it with regard to conversions and take improvements in Quality Score as a bonus.

So far it stands to reason that the influence of Instant Previews is fairly small, since not everyone uses them and not all ads provide the option. However, as user behavior changes, more people might use the feature, so that its weight in Quality Score increases. In any case, if Google ever pushes Instant Preview in any way, think of your Quality Scores.