AdWords Keyword Quality Score Shakeup

For the last few weeks, AdWords advertisers have enjoyed seeing higher than usual keyword Quality Scores. Since we’ve been tracking those hourly for the last couple of months, we’ve been able to closely monitor this development. The following chart sums it up nicely:

Average Keyword Quality Scores, November 2012

Included are the average visible Quality Scores of about 1,600 keywords from three of our clients. As you can see, the increases in keyword Quality Scores started in the morning of November 6th. In one account there was a sudden jump between seven and eight in the morning. The other two accounts saw gradual increases over 23 hours.

All times in the chart are local German time, so you’d have to subtract nine hours to get to pacific time – looks like the increases happend outside of Google’s office hours.

However nice it is to see high Quality Scores, nothing has actually changed. CPC’s didn’t budge, same, impression counts remained unchanged, and average positions didn’t change either.

In my opinion, this is just one more example of how little keyword Quality Scores actually matter. But even though almost everyone knows that this metric is completely different from what Google uses in the actual ad auction, the myth of the little number from 1-10 playing a role with CPC’s and ad positions remains strong.

So far, the only significant improvement after the keyword Quality Score increases was with advertisers’ spirits. My impression is that the higher visible Quality Scores did a better job of spreading joy among advertisers than many real performance improvements ever did. This might be a good time to rethink KPI’s…

(Cross posted from our German Blog)

About Martin Roettgerding
Martin Roettgerding is the head of SEM at SEO/SEM agency Bloofusion Germany. On Twitter he goes by the name @bloomarty, and you can find him regularly on #ppcchat.

  • Thomas

    Hi Martin,

    thanks for analyzing! I didn’t recognize better QFs in my accounts.
    But I can see your point. It’s time for more convincing KPIs.



  • I noticed the same thing in several of my accounts and heard from other advertisers that visible quality score jumped without any affect on CPCs, position, etc.

    I agree that quality score should matter very little. Same as SEO, the less you focus on QS (or links, PR, etc. for SEO) and more you focus on improving the quality of your account (ads, landing pages, keywords, etc) the better your account will do.

  • I too noticed higher QS of 10 almost across the board in a few accounts, although I can’t remember the day. But I did not make anything of it until someone mentioned it in a forum. QS of 10 is not unusual in my case – as it shouldn’t be in yours either – as that’s what we get paid for.

    Since you track QS every hour (why, I don’t know but good that you do in this case), you are likely the only one in the world able to determine when this increase happened. I track once a week.

    I therefore compiled data from my biggest client a few weeks before until today. I didn’t think I would see a change in CPC. I have noticed an increased lately but attributed this to competitors and I do make bid changes to gauge many different things.

    However, and this surprised me, the increased happened on November 6. To be fair, CPC is not consistent day to day. Six days earlier, the CPC was higher and there were five consecutive days later in November where it was lower and more inline with figures of previous months.

    I’ve noticed out of the ordinary changes in average positions too. On November 6, the position was lower but only very slightly. Then on the 18th, it went up, higher than the norm but only to go back down on the 23rd, lower than the norm. It has now, just as suddenly in the last five days, went back to levels seen from November 17-22. The CTR has increased and the CPC decreased too.

    I’m aware that Google tests different things and upgrades the service all the time. And sometimes there has been glitches when doing so. What they are doing, I can only speculate.

    But to say that QS doesn’t matter is wrong. It does matter.

    True, there was something a bit weird going on lately. As a database programmer, I can attest to rolling out new versions of software before its time and some issues found under actual work load. Maybe that’s what happened in this case and I’d say the issues are likely resolved judging by my data.

    The evidence of QS is also all over the stats. That’s why what you call a myth is still strong. There’s evidence that it exists. And, if QS was a made up number, I have no idea how they would rank ads and determine CPC. The QS system, when you have a good idea of how it’s calculated, is actually beautiful simplicity that is a win for the advertiser, a win for the searchers and a win for Google.

    I also find it strange to hear someone who does this for a living saying that QS is a myth. Maybe you’d like to share your thoughts on what KPIs (Key Performance Indicators for those not sure what it means) you would use in Adwords.

    Lucid Web Marketing

  • Martin Roettgerding

    Hey Pierre,
    Interesting to hear that there were changes in one of your accounts. To be clear,
    we’ve had fluctuations in our accounts as well, just nothing out of the ordinary. CPC’s and other stats always fluctuate from day to day, and they did this in November just like in any other month.

    Anyway, I wanted to clarify that I never said or wrote that Quality Score doesn’t matter. The thing that doesn’t matter is the keyword attribute you can see in the AdWords interface (a.k.a. keyword Quality Score or visible Quality Score).

    Google has taken the liberty to use the term Quality Score for two things: For a score from 1-10, attributed to keywords and visible in an AdWords account, and for a value used in the ad auction, where ad positions and click prices are determined.

    Auction QS matters a lot, but it’s unknown to advertisers. Keyword QS on the other hand is just a number. It’s perceived as important because its assumed to be somehow connected to auction QS – mainly because few people who talk or write about this make the distinction between keyword QS and auction QS, which contributes to the ongoing confusion and keeps certain myths alive.

    This phenomenon, where keyword Quality Scores suddenly increase three points without having any significant impact on anything is one good example of how little impact this number has. It very much proves that, as a KPI, the number is useless.

  • Ed

    Totally backing up your theory.

    I knew that my QS had increased recently but I wasn’t sure about the date. At that time I wasn’t using scrip to track QS but I know that I had post something in our team skype to mention that to my colleague.
    I post the message the 06/12… crazy

    Anyway, great analysis again.


  • >> Auction QS matters a lot, but it’s unknown to advertisers.

    I’ve never heard of this and I’ve been doing Adwords for 11 years and professionally managing client accounts for 7 years. Not saying it doesn’t exist, I learn new things all the time.

    I think QS is QS and there’s only one used for everything. It’s shown at the keyword level where maybe it should be shown at the group level like Bing does. I’m sure both calculate it for each kw-ad combination.

    Some people qualify the QS but I think that’s just semantics. I try for instance to avoid saying keyword QS because it’s really not the keyword that’s the driving factor in the QS calculation (ads are what people click on, not keywords). So ad QS is more precise although like I said, it really is kw-ad. Does it confuse people who see keyword QS, ad QS and whatever other QS? Surely. But as long as we define QS as one specific and unique thing, there shouldn’t be. It’s like those who say that their site doesn’t rank well organically. You and I both know that search engines rank pages, not sites and we know what they mean if they say site ranking, although technically wrong. Nobody suggests there’s a page ranking AND a site ranking.

    Seeing keyword QS as you call it go up or down by three points, while curious, does not prove anything, certainly not that there are different QS or that keyword QS, if there is such as thing as auction QS, is useless.