Expanded Broad Match: Taming the Beast

There's a great thread going on in WMW wherein advertisers are discussing particular cases of Google's Expanded Broad Match (EBM) matching ads to searches that have nothing to do with the product/service they sell, and in cases of brand keywords are matching to other brands - resulting in legal cease & desists and pissed off partners.

For those not well-versed in Google's EBM, it is a match type option whereby the advertiser leaves it to Google find the maximum amount of related searches against which to match the advertiser's ad. It's the broadest match type and the one many, many advertisers use to get the maximum volume of [hopefully well-targeted] traffic.

It would appear that Google is permitting itself more and more liberties recently to support advertisers' quest for ever more volume. You can't fault Google for this, least of all given that their broad matching - pimples, hiccups and all - is the best broad match in the market, but in cases where advertisers are feeling pain from EBM wackiness it certainly begs the question:

How can Google on the one hand be turning the Quality Screws more and more while riding the white horse of The Sacred User Experience, and at the same time encourage all new and existing advertisers to use EBM when it has such profound flaws built into it and which C - L - E - A - R - L - Y provide both a horrible user and advertiser experience?

The answer harks back to one of my SEM predictions for 2007:

3. George Reyes' fingerprints found on Quality Score knob at the 'Plex.

Likewise, David of Blogation fame wrote a great summary of how Google's auction system has changed in terms of how ad rank is calculated and how that change has led to an EBM-dominated world. He says:

These days, the value of the tail has diminished, if not disappeared entirely. There are three primary reasons for this:

1. The Search Engines are Smarter: All of the search engines now have "broad matching" or "advanced matching" features. This means that a big competitor that bought the keyword "mortgage rates" will still likely show up when a user types in "best mortgage rates" or "discount online mortgage rates." The rationale behind this - from the search engine perspective - is fairly obvious; by populating obscure long-tail searches with results from very competitive keywords, the bids increase dramatically (no more $.10 clicks on the long-tail).

It used to be that the search engines tried to balance their drive for additional revenue with user experience concerns by looking at "token matching." Think of a token as a word. A phrase like "bad credit mortgage loans" has four tokens. The old rule was as follows: if a generic phrase matches at least 50% of the tokens in a long-tail phrase, the search engine would consider that relevant enough to show the broad match results on the long-tail phrase.

So, in the "bad credit mortgage loans" example above, "mortgage loans" and "bad credit loans" would be broad-matched, but "mortgage" and "loans" would not (because they only have 25% of the tokens).

Based on what I see in the market today, this is no longer the case. The search engines have basically moved away from the "token-matching" system and are getting closer and closer to a "category-based matching" approach. In this model, a broad phrase like "mortgage rates" could be matched not only on "bad credit mortgage rates" but on "home loan rates", "find a mortgage", and "san mateo county loans for new home buyers."

One other related point worth noting. In the olden days, even if a broad match had a 50%+ token match and did show up alongside long-tail searches, it had to compete in the same auction for top ranking.

Here's what I mean: let's say that you paid $5 CPC on Google for the keyword "mortgage rates" and you elected to be broad-matched across all long-tail keywords Google thinks are relevant to you. Let's say that I bought the keyword "alabama low mortgage rates" and I was willing to pay $2 CPC for this keyword.

When a user did a search for Alabama low mortgage rates, Google looked at two factors to determine ranking - maximum CPC and click-through rate (CTR). The key point here, however, is that Google looked at CTR on a keyword-specific basis. So, if your broad matched keyword had an overall CTR of 10%, but only a .5% CTR on the specific keyword "Alabama low mortgage rates", Google would use the .5% CTR to determine ranking. So, if my $2 bid had a CTR of 10% (due to the fact that I would likely have highly-specific ad-text, and I might be an Alabama-specific mortgage lender),
my effective "cost per thousand" (CTRxCPC=eCPM) basis would be $200 and yours would only be $25. So, if we went head-to-head against each other, I would clearly show up more highly than you.

That is not the way things work today. As far as I can tell, Google has moved away from the keyword-specific auction model to more of an "overall CTR" auction model. In this new scenario, if your generic keyword has an overall CTR of 5% (but remember, only .5% on the specific keyword) and my specific keyword has a 10% CTR, the eCPMs would be $250 for you and $200 for me - you would show up first. The long and the short of it is that this new system decreases the relevance of the tail by enabling high-CPC generic keywords to outposition targeted keywords.

By changing "token matching" to "category matching" and by changing the concept of a "single auction" to an "overall auction", the results may be slightly less specific-ads for the user, but virtually eliminate market inefficiencies (read: no more low CPCs) for the search engine.

Of course, the flip-side of that is that as user queries are consolidated into a few categories, the bulk of ad-generated traffic is consolidated into those advertisers who can pay top position for the most popular searches. Or, to put it another way, instead of entering into 10,000 auctions for 10,000 different keywords, advertisers are now entering into just a few auctions for those 10,000 keywords. If you want to get traffic, you have to win in one of these few auctions.

to which EF's own David White responds thoughtfully:

Interesting post! Although the calculations are correct, I'd question the bid assumptions - looking at the words "mortgage rates" and "alabama low mortgage rates", I think that the second term could justify a higher price. The traffic could be driven to an Alabama-specific landing page with relevant offers that convert better than the generic mortgage rate offers.

Therefore, it may be more realistic to use a $5 CPC for mortgage rates and a $10 CPC for alabama low mortgage rates. If you run the calculation for this scenario, you'll see that the new eCPM for mortgage rates is still $250 but it would be $1000 for alabama low mortgage rates. Even if you assume that the bids are equal ($5 for alabama), the alabama term would still win with an eCPM of $500.

Tail terms may no longer be a gold mine, but they still make sense because they allow you to outbid the broad-match advertisers on the terms where you convert best.

This front line of advertiser need and search engine supply is what will keep optimal SEM solutions in demand for a long time to come...