What Landing Page Quality Score is Really All About

A year ago, in October 2011, Google announced a change to how Quality Score was calculated. Since then, landing page quality is supposed to play an important role the ad auction, where ad ranks and click prices are determined. It was easy to miss that this was actually a fundamental change – one that didn’t add up.

In my opinion, Google has once again given priority to marketing and neglected to tell the real purpose behind this change. In consequence, advertisers got a completely wrong picture of what landing page quality is and what they’re supposed to do. Basically, all conclusions and strategies to deal with landing page quality are utterly wrong.

With this blog post, I’d like to take you on a little tour. First, we’ll look at what changed and where things don’t add up. Then we’ll follow the money to find a better explanation for things. Finally, we’ll look at the consequences and what you can do to optimize for landing page quality.

A little Paradigm Shift?

It all started with an innocent-sounding announcement from Google. They were going to “increase” the importance of one ingredient of Quality Score:

“the weight given to relevance and landing page quality in determining Quality Score and how ads are ranked on Google.”

They also hinted that this wasn’t such a big deal. Well.

This was actually a paradigm shift. Before this change, landing page quality wasn’t considered in the ad auction at all. This used to be a major misunderstanding about Quality Score: landing page never mattered when rankings and click prices were determined. Google never made much of an effort to state this clearly, but it used to be in the “fine print” in the Quality Score documentation. Well, now landing pages play a role when ad positions and prices are determined.

And there is a second change: landing page quality can help you now. This, too, used to be different. In the past, a bad landing page could get you in trouble, but there was no reward for a good or a great one. It used to be purely a penalty factor. Or, as Andrew Goodman, who apparently also talked to Alferness, put it:

In the past, landing page quality was usually administered as if it were “policy”: that is, there was an attempt to look for what Alferness calls “negative signals”.

So now they look for positive signals, too. And now those can help you with your ad rank.

Strong and Slight Boosts

Reading further beyond Google’s official announcement, I found two more interesting sources with information about this. On Search Engine Land, Pamela Parker quoted Google’s Jonathan Alferness. Alferness confirmed that advertisers with better landing pages will rank better and will get a “strong boost” in ad auctions.

George Michie also wrote an article on the RGB blog, dealing in great detail with the changes about landing page quality. He, too, got to talk to Alferness. This time, Alferness only spoke about a “slight boost” and George pointed out that Google’s official announcement even states the expectation that “most campaigns will not see a significant change in overall performance”. As it turns out, advertisers really didn’t see any change over this – if they did, they kept it to themselves.

Making Users Happy

What I find very interesting is how Alferness stressed a familiar point in the interview with George Michie:

The intent is to improve user experience.

Alferness goes on and makes exactly the same argument as Google always has: They need quality ads, because otherwise users wouldn’t be happy. But Google wants to make users happy so that they come back. According to Alferness (and Google in general), it’s not about short term profits – it’s a long term strategy to make users happy and get them to come back.

This sounds reasonable and it’s a great marketing story. It’s no surprise that Google sticks to this, no matter what. But the thing is, this also made perfect sense back in 2005 when Google first introduced Quality Score and made the exact same argument: that it’s all about making users happy. But for six years Google was perfectly happy with leaving landing pages out of the ad auction. And they also didn’t have a problem with advertisers believing the opposite, since this technicality was somewhat hidden in the fine print.

So when the reasoning hasn’t changed since 2005, then why did Google wait until 2011 to do something about it?

Testing a Long Term Strategy

There’s another weak point in Google’s story: They tested the effects of the change in first. Remember that Google stated that it’s a long term strategy and that it had little actual impact on advertisers. So how do you test the long term effects of a little change that would sometimes result in a slightly different ad order? For Google, this apparently wasn’t much of a problem. Let’s look at the time table:

That’s odd, isn’t it? They managed to test the effects of a little tweak on long term loyalty of users – and they did it two months (one month outside of Brazil). And that included “one to two weeks” until the experiment was rolled out and everything had stabilized. And the time to evalute things. And the time a company like Google needs to decide on a major change about their primary revenue source.

(It’s also worth noting that Google had and has a market share of more than 90% in Latin America, Spain, and Portugal. Looks like the room for improvement was rather limited.)

Following the Money

So some things don’t really add up. So how can we understand this move from Google? Following the money is often a good idea. In fact, I already did this a series of articles on what Quality Score actually is. Back then, early 2011, I determined what (auction) Quality Score had to be in order for Google to maximize revenue. The conclusion was:

Quality score is Google’s estimate for click-through probability.

Sounds strange, I know. You can call it “predicted CTR” if you think that’s catchier. But labels aside: that’s what Google needs to do: make an estimate of the chance an ad gets clicked – because that’s the chance they get paid.

Putting the highest bidder in the top spot, regardless of the chance the ad actually gets clicks, is a bad strategy (unless you don’t care about revenue). In order to rank ads correctly, Google has to look at the pay (bid) and the chance they get paid.

Of course, estimating the chance of a click-through is an incredibly hard thing to do, especially if you don’t have much historical CTR data to go on. But it pays: The better Google’s estimate is, the more money they make. Improve the estimate a tiny bit might just get Google a fraction of a cent per auction – but with billions of ad auctions each year, it adds up to a big deal.

So what does this have to do with landing page quality? Well, nothing – and that’s the problem. Because in order to determine click-through probability (a.k.a. maximizing revenue), the landing page is useless. Back when I first published the article about Quality Score, this was an important point: Had Google wanted to make Quality Score about user experience, they would’ve incorporated landing pages right away. Instead they didn’t, but only mentioned this in the fine print.

Now the problem is: If you dilute Quality Score with anything else besides click-through probability, the outcome is farther away from the revenue maximizing optimum, meaning the result is less money for Google. So why include landing pages when they don’t have anything to do with whether anyone clicks an ad? After all, the user only sees the landing page after the click.

Although…

Wait a minute… let’s check something…

ip full car insurance

Oops, look at that – there’s a landing page. Visible before the click. So much for that.

Looking at the preview in the screenshot, I notice that the ad promises £30 cash back and the preview clearly picks up that promise. A potential customer who was interested in the ad before would probably click now – and Google would make some money.

So we’re back at the money. As I wrote above, Google needs to know the likelihood of getting paid (a.k.a. click-through probability a.k.a. Quality Score) as precisely as possible. The better the estimate, the more money Google can make. So if knowing about the effects of Instant Preview on click-throughs can improve the estimate, it would be leaving money on the table to ignore this.

This is why I believe that landing page quality is actually about the quality of the Instant Preview.

Is that right?

I realize that this contradicts something that most people had long accepted. The reasoning so far is based on common sense and the assumption that Google wants to maximize revenue. I’d say that’s enough, but let’s see what else we can find.

I mentioned some odd things about the official version about landing page quality. There was the issue with the strong or slight boost that no one seems to have felt. Considering how often Instant Preview is (probably) used and that it’s not available for all ads, I’d say that’s no surprise.

Then there’s the issue why landing page quality started to matter in 2011, after it was basically ignored for six years. But if you consider that Instant Preview for ads was first introduced in April 2011, it suddenly makes perfect sense.

The oddly short test also makes sense with regard to Instant Preview. Testing the long term effects of slightly different ad orders on user loyalty in a short time and without much room for improvement sounds implausible to me.

But if it’s all about improving click-through probability estimates, it’s much easier. Google could basically just have done a series of A/B tests to find out what method of considering landing page factors made them the most money. Unlike long term user loyalty, click-throughs and ad revenue can be measured easily and immediatly.

What do we do with this?

Advertisers understandably want to leverage landing page quality to improve Quality Scores. George Michie got a few general statements out of Alferness, which can help us understand what Google looks at to determine landing page quality. According to him, Google uses the following basic criteria:

  • Relevance
  • Content
  • The user’s response

Since it’s incredibly hard for a machine to analyze a landing page’s quality, advertisers have translated those factors into criteria that Google could evalute. Relevance is commonly interpreted in terms of on-page SEO factors, which in turn is interpreted as the necessity of having keywords on the landing page. Google employees have often denied that there is any benefit of keywords on a landing page, but as far as I know, there is no official statement.

With content, things are also unclear. There are some Google guidelines about landing page experience which include relevant and original content, transparency, and ease of navigation. Not all of the cited aspects and examples can be evaluated algorithmically, which suggests that at least some of those guidelines have nothing to do with Quality Score.

As far as evaluating user responses to assess landing page quality is concerned, bounce rate is usually the factor that comes to mind. As with keywords, Google employees have occasionally denied that bounce rate plays a role in evaluating landing page quality.

Okay, now let’s look at those things with Instant Preview and the problem of estimating click-through probabilities in mind. In order to estimate the probability behind Instant Preview click-throughs, the most significant and reliable thing to look at is click-through data from the past – just like general click-through data is the most reliable part of general click-through probability estimates (or, in Google’s words: CTR is the most important factor in Quality Score). So there we already have the user response to look at: It’s not bounce rate, but Instant Preview click-through rate.

It stands to reason that all other criteria (like relevance and content) are secondary when it comes to estimating click-through probability for previews. Again, it’s the same as with CTR and Quality Score in general: In the absence of enough historical click-through data, Google has to look at other factors (like relevance). But the more first-hand experience Google has with an ad’s performance, the influence of those factors fades away. For example, if Google has shown an ad a million times with abysmal click-through rates, it would be foolish to look at relevance and conclude that the ad might do great in the future. The same is true with Instant Previews.

In consequence, even if you could optimize a landing page with regards to relevance and content, the positive effect of such optimizations would fade away once Google starts to see how users actually react to your previewed landing page.

Understanding Landing Page Quality

When optimizing a landing page with regard to Quality Score and Instant Preview, there are short term and long term criteria. Short term criteria are all those things that Google might look at to algorithmically evaluate a landing page. But mid to long term it’s all about the user’s reaction to the preview.

To understand the importance of the criteria, let’s look at a few examples. I’ve searched for a ski vacation and found three interesting landing page previews:

ski-previews

The first landing page is from snow.com. It’s highly relevant, has the keywords, and a lot of text. From an SEO perspective, the page looks good. If we look at Google’s landing page guidelines about relevant and original content, transparency and ease of navigation, the page is good, too. So if Google really looks at those criteria, this page should be off to a great start. However, the preview looks kind of boring and if users don’t react well on it (by clicking the ad), the effect on Quality Score will be negative as soon as Google realizes how little users like the preview.

The second page from Ski Bundle is a little more engaging but has a lot of text, too. Google has highlighted a snippet with keywords. You can also see a promotion (the “Banff Bundle” for $780), which could be a good thing. But in this case, it’s more of a problem, since the ad was for ski vacations in general and the promoted bundle is just a random part of a rotating set of promotions. To make matters worse, the promotion is actually no longer part of the rotation and nowhere to be found on the landing page, but it’s still part of the preview.

The third landing page from SnowTrex has a lot of images and is much less text heavy in comparison. There are no promotions standing out and no keywords visible, but it’s obviously about ski vacations and it seems to offer a lot of choices without requiring a lot of reading. Of the three examples, I’d say that most people would probably click on this one, which would perfectly underscore how little keywords and textual content matters to the actual human user – because the page the preview is from is actually in German.

How to Optimize for Instant Previews

Now let’s look at how we can actually optimize a landing page with regard to Instant Preview. I believe it’s possible, although uncertain, that some on-page SEO can help a page initially, but the effect will be overridden quickly when Google starts to see how actual people react to the preview. This is why I believe that no investment in landing page SEO has a chance of ever paying off.

In order to truly improve landing page quality in the eyes of Google, you would have to improve the page so that actual users would want to click the ad after seeing the preview. Since we don’t get any data on Instant Preview click-through rates, A/B testing isn’t an option. But there are a few pointers:

  • Make sure that you can see the page is about the same thing as the ad. If you make a promise in the ad, consider if it should be visible in the preview.
  • Pictures, like product images, are easier to grasp and more appealing than textual content.
  • When you have a large number of landing pages, like an online retailer, having readable headlines is helpful. Product images might not tell everything, like that the page is only about a certain brand.
  • Some selling points or trust signals might be suitable to be visible in the preview, especially when they come in the form of logos.
  • Consider whether you have control over what shows up in the preview. Automatic slideshows and things like that can be a problem.
  • Make sure the preview is up to date. If you change something on the landing page, you can’t be sure that Google updates the preview in time. Consider using a new page or at least a new destination URL. Appending a meaningless parameter like “?lp=1” to the URL should be enough to make Google get a new preview screenshot.

These are just a few ideas and I’m sure there are a lot more. However, keep in mind that a landing page should always focus on getting users to convert. Sacrificing conversions for clicks would be a bad idea.

Still, some optimization made with regard to conversion rate can also be helpful with the Instant Preview. For example, making a page more accessible by getting rid of text and using attractive images instead could lead to improvements in both areas. And making sure the offers on the landing page match the ad’s promises is always a good idea.

Conclusion

Unless Google has designed the ad auction mechanism to leave money on the table, optimizing a landing page with regard to Quality Score means optimizing for Instant Preview so that people click the ad.

There are a few consequences that come with this conclusion. On the one hand, some commonly accepted strategies won’t work: Keywords on landing pages, or SEO in general, practically have no effect on Quality Score. And neither do other things you might consider quality, like page load time. Unless it affects click-throughs, it doesn’t count.

On the other hand, you can do some things to improve the preview itself, but if you want to optimize the layout of a landing page, better do it with regard to conversions and take improvements in Quality Score as a bonus.

So far it stands to reason that the influence of Instant Previews is fairly small, since not everyone uses them and not all ads provide the option. However, as user behavior changes, more people might use the feature, so that its weight in Quality Score increases. In any case, if Google ever pushes Instant Preview in any way, think of your Quality Scores.

About Martin Roettgerding
Martin Roettgerding is the head of SEM at SEO/SEM agency Bloofusion Germany. On Twitter he goes by the name @bloomarty, and you can find him regularly on #ppcchat.

  • Hi Marty, Thanks for taking the time to write this post.

    I had a couple of thoughts. Like you say, given the small percentage of users that use instant preview, I think landing page quality score must be concerned with the post click experience. Whilst I agree with you that “in order to determine click-through probability (a.k.a. maximizing revenue), the landing page is useless” Quality landing pages matter to the future viability of the AdWords model. Without a quality experience, users will just lose confidence and look to the organic results.
    The points you make in regards to instant previews are definitely valid, I just feel that in a monetary sense, losing revenue by penalising the ad rank of poor landing pages, will be readdressed in increased lifetime value by ‘ad clickers’ who’ve had positive experiences.
    Given how hard it is, especially in some sectors for 1st (organic) position, AdWords needs to reflect that in terms of quality and fairness. Assessing the landing page using on page SEO metrics + load speed etc I think does that. Quality Score needed to evolve that way, times are harder now. Their should be no quick wins for anyone these days and the same quality experience should befall all user clicks, be it organic or paid.
    At least this change should stop some SEOs calling PPC an easy way to game things. It would seem now that we’re not so different.

    • Martin Roettgerding

      Hey Shaun,
      Thanks for commenting! The argument of Google caring about user experience and confidence makes sense, but so far Google has always ensured landing page quality with fairly hard penalties where appropriate.

      The AdWords mechanism also takes care of quality through bids. Advertisers have a huge incentive to provide a positive user experience in order to get users to convert and make some money. If they are successful, a click becomes more valuable and they bid higher. This indirectly leads to the promotion of quality.

      In general, I find it hard to believe that Google would leave some money on the table because they’d rather consider “true quality” a tiny bit. Because basically, we’re talking about an impact that was negligible for advertisers and probably for users, too. Google would see the (negative) impact on their bottom line, but no one else would really appreciate the difference.

      Adding to what I already wrote, there are some other things which don’t seem to fit. If Google would want to consider quality in the ad auction, diluting the formula in order to really consider quality, without making a big deal of it, seems odd. At least Google should offer some pointers on how to improve landing page experience. If they really care so much and they are able to really analyze landing pages, why not point advertisers in the right direction?

      I believe a great thing to look at here is load time. It’s important because it’s undisputedly tied to user experience and it’s easily measurable. Giving advertisers some pointers on how they’re doing in that area would give them an incentive to improve. But Google doesn’t seem to care that much. And they never indicated that load time now is any more than the penalty factor it used to be.

  • Fascinating thoughts, Martin. The idea that “preview optimization” might enter the SEM lexicon never occurred to me.

    Thanks for the shout out, and for contributing another really excellent piece to the industry.

    • Martin Roettgerding

      Thanks, George!

  • Hi Marty – great post.
    I completely agree with your thoughts on how discounting cpcs for highly optimised pages (as opposed to just OK ones) doesn’t really make economic sense for Google. They’re impacting their bottom line and getting nothing in return (and anyone who believes they still focus on long term SERP quality from their dominant market position needs to take a look at some of the results out there!).
    I do however have some anecdotal evidence that including the exact search query on the landing page reduces CPC/increases position. Obviously there’s too many factors at play in the landscape for this ever to be provable, but I’ve seen it in a couple of places where I’ve added the exact search query to a page where it hasn’t existed at all before.
    As a result it’s always been my assumption that Google is really penalising for poor page experience (in this case irrelevant content – no mention of what the searcher was looking for – but also extremely slow load time, doorway pages and spam) rather than giving the fastest, most relevant and best designed page in the world cheaper clicks. I’d be interested in your thoughts on this? Have you managed to do much testing of “landing page quality” elements of campaigns?
    Cheers

    • Martin Roettgerding

      Hey Matt,
      I’ve heard similar anecdotal evidence, even long before the change in 2011. I’ve always been sceptical about this since I’ve never witnessed it myself. We have of course tested landing page elements and those included keywords, but we’ve never seen any notable changes in anything but conversion rate and order value.

      I’m also sceptical because in my experience, those things are extremely hard to measure and people often take the shortcut by only looking at keyword Quality Scores. Then if the number goes up by one, it’s considered a great success, regardless of actual performance. And even if you look at performance, it’s easy to interpret the always fluctuating numbers in a way that supports a theory. It’s just human nature to look for things that reinforce our beliefs…

      The old Quality Score documentation was clear about landing page quality not affecting the ad auction, but it could be interpreted as landing page quality influencing minimum bids. A higher minimum bid would effectively exclude you from some cheaper auctions. This wouldn’t affect CPC and position in the auctions you can take part in, but in your account you only see averages. Without the cheaper auctions contributing, the average CPC in the account would be higher. This would be one possible explanation, but it’s of course speculation…

      In any case, this would be about lifting a penalty, not getting rewarded. Having the exact match keyword on a page might help reassure Google, but rewarding someone for this wouldn’t make sense. Also, having an exact match for all potential search queries that could match your keywords would be impossible. If anything, this is about topics, not keywords.

      • Yeah it’s pretty much impossible to measure. there’s too many other factors in play, which is why I was interested in your experience, so thanks for that!
        You’re right to reiterate that visible QS is meaningless – changing some of these things will definitely help to shift the “Landing Page Quality” needle on the Diagnostic Tool (or whatever the hell it’s called now!) to “average” or “above average”, but like vQS, this in itself has zero impact on CPC, or Position.

  • Here’s some related info from Brad Geddes from his talk at the Hero Conf 2012:
    http://www.ppchero.com/demystifying-quality-score-with-brad-geddes/

    In the first quote below, Brad supports this article’s assertion that landing page quality seems loosely correlated (at best) with performance, and back to the key importance of CTR.

    —– QUOTE 1 —–
    “Landing page, they’re a little bit tougher these days. Up until October of
    last year, a landing page was good or bad. There was no goodness or
    badness in landing pages. It was an absolute good or bad. And in October
    last year, Google made a change that a landing page affected quality score
    more. More is a really undefined term. And I’ve only, still to this point in
    time, seen one account ever that landing page was deemed relevant, but
    working on the landing page helped quality score. I have seen one case
    where that wasn’t true, where making landing page changes did affect
    quality score. So this is a big change last October that Google went
    through.

    “But more is one of those terms that it’s hard to say is it 1% or is it 60%?
    I’m guessing it’s more in the 5%-10% range if you try to quantify it, but it’s
    a tough one. So your landing page really matters, but if you have a landing
    page, that’s okay, you’re probably fine. And it really gets into CTR.”

    —–QUOTE 2——
    Here Brad refers to landing page relevancy. “Reforcing” is manually initiating a landing page relevancy review.

    “They get their relevancy algorithms about once a week. They do landing
    pages like every 6 weeks. So if you have a bad landing page, I hope you
    have a rep, because a rep can force a recall and force certain things to
    happen. If you don’t have a rep, call them up, beg and plead a little bit and
    see if you can get them to reforce it. Sometimes it will work, sometimes it
    doesn’t work. But that’s something to sort of look for on the landing page.
    That’s your biggest bane is bad landing pages.”

  • roohani ilaj kay wazaij

    Nice Post frndzZz

    get more islami wazaif post

    islami wazaif