Thursday, June 21, 2007

Expanded Broad Match: Taming the Beast

There's a great thread going on in WMW wherein advertisers are discussing particular cases of Google's Expanded Broad Match (EBM) matching ads to searches that have nothing to do with the product/service they sell, and in cases of brand keywords are matching to other brands - resulting in legal cease & desists and pissed off partners.

For those not well-versed in Google's EBM, it is a match type option whereby the advertiser leaves it to Google find the maximum amount of related searches against which to match the advertiser's ad. It's the broadest match type and the one many, many advertisers use to get the maximum volume of [hopefully well-targeted] traffic.

It would appear that Google is permitting itself more and more liberties recently to support advertisers' quest for ever more volume. You can't fault Google for this, least of all given that their broad matching - pimples, hiccups and all - is the best broad match in the market, but in cases where advertisers are feeling pain from EBM wackiness it certainly begs the question:

How can Google on the one hand be turning the Quality Screws more and more while riding the white horse of The Sacred User Experience, and at the same time encourage all new and existing advertisers to use EBM when it has such profound flaws built into it and which C - L - E - A - R - L - Y provide both a horrible user and advertiser experience?

The answer harks back to one of my SEM predictions for 2007:

3. George Reyes' fingerprints found on Quality Score knob at the 'Plex.

Likewise, David of Blogation fame wrote a great summary of how Google's auction system has changed in terms of how ad rank is calculated and how that change has led to an EBM-dominated world. He says:

These days, the value of the tail has diminished, if not disappeared entirely. There are three primary reasons for this:

1. The Search Engines are Smarter: All of the search engines now have "broad matching" or "advanced matching" features. This means that a big competitor that bought the keyword "mortgage rates" will still likely show up when a user types in "best mortgage rates" or "discount online mortgage rates." The rationale behind this - from the search engine perspective - is fairly obvious; by populating obscure long-tail searches with results from very competitive keywords, the bids increase dramatically (no more $.10 clicks on the long-tail).

It used to be that the search engines tried to balance their drive for additional revenue with user experience concerns by looking at "token matching." Think of a token as a word. A phrase like "bad credit mortgage loans" has four tokens. The old rule was as follows: if a generic phrase matches at least 50% of the tokens in a long-tail phrase, the search engine would consider that relevant enough to show the broad match results on the long-tail phrase.

So, in the "bad credit mortgage loans" example above, "mortgage loans" and "bad credit loans" would be broad-matched, but "mortgage" and "loans" would not (because they only have 25% of the tokens).

Based on what I see in the market today, this is no longer the case. The search engines have basically moved away from the "token-matching" system and are getting closer and closer to a "category-based matching" approach. In this model, a broad phrase like "mortgage rates" could be matched not only on "bad credit mortgage rates" but on "home loan rates", "find a mortgage", and "san mateo county loans for new home buyers."

One other related point worth noting. In the olden days, even if a broad match had a 50%+ token match and did show up alongside long-tail searches, it had to compete in the same auction for top ranking.

Here's what I mean: let's say that you paid $5 CPC on Google for the keyword "mortgage rates" and you elected to be broad-matched across all long-tail keywords Google thinks are relevant to you. Let's say that I bought the keyword "alabama low mortgage rates" and I was willing to pay $2 CPC for this keyword.

When a user did a search for Alabama low mortgage rates, Google looked at two factors to determine ranking - maximum CPC and click-through rate (CTR). The key point here, however, is that Google looked at CTR on a keyword-specific basis. So, if your broad matched keyword had an overall CTR of 10%, but only a .5% CTR on the specific keyword "Alabama low mortgage rates", Google would use the .5% CTR to determine ranking. So, if my $2 bid had a CTR of 10% (due to the fact that I would likely have highly-specific ad-text, and I might be an Alabama-specific mortgage lender),
my effective "cost per thousand" (CTRxCPC=eCPM) basis would be $200 and yours would only be $25. So, if we went head-to-head against each other, I would clearly show up more highly than you.

That is not the way things work today. As far as I can tell, Google has moved away from the keyword-specific auction model to more of an "overall CTR" auction model. In this new scenario, if your generic keyword has an overall CTR of 5% (but remember, only .5% on the specific keyword) and my specific keyword has a 10% CTR, the eCPMs would be $250 for you and $200 for me - you would show up first. The long and the short of it is that this new system decreases the relevance of the tail by enabling high-CPC generic keywords to outposition targeted keywords.

By changing "token matching" to "category matching" and by changing the concept of a "single auction" to an "overall auction", the results may be slightly less specific-ads for the user, but virtually eliminate market inefficiencies (read: no more low CPCs) for the search engine.

Of course, the flip-side of that is that as user queries are consolidated into a few categories, the bulk of ad-generated traffic is consolidated into those advertisers who can pay top position for the most popular searches. Or, to put it another way, instead of entering into 10,000 auctions for 10,000 different keywords, advertisers are now entering into just a few auctions for those 10,000 keywords. If you want to get traffic, you have to win in one of these few auctions.

to which EF's own David White responds thoughtfully:

Interesting post! Although the calculations are correct, I'd question the bid assumptions - looking at the words "mortgage rates" and "alabama low mortgage rates", I think that the second term could justify a higher price. The traffic could be driven to an Alabama-specific landing page with relevant offers that convert better than the generic mortgage rate offers.

Therefore, it may be more realistic to use a $5 CPC for mortgage rates and a $10 CPC for alabama low mortgage rates. If you run the calculation for this scenario, you'll see that the new eCPM for mortgage rates is still $250 but it would be $1000 for alabama low mortgage rates. Even if you assume that the bids are equal ($5 for alabama), the alabama term would still win with an eCPM of $500.

Tail terms may no longer be a gold mine, but they still make sense because they allow you to outbid the broad-match advertisers on the terms where you convert best.

This front line of advertiser need and search engine supply is what will keep optimal SEM solutions in demand for a long time to come...

Tuesday, June 19, 2007

The Real Shoe Money

When you hear the name ShoeMoney in the world of search it's synonymous with Jeremy Schoemaker, the SEO/AdSense guy who prints money due to his insane SEO and AdSense skills.

But here on Searchquant I'm writing to literally ask you for... shoe money.

I've become a Soldier's Angel to an entire U.S. platoon stationed in Afghanistan, my role being to help them out in any way they can while they hunt for bad guys in the mountains and generally put themselves in harms way.

So far myself and a group of like-minded people have sent them 20+ pairs of the ballistic-grade sunglasses they needed along with a ton of snacks. Now, though, I want to try and get them each a pair of the boots they need for the 2-3 day patrols they're conducting in the mountains of Afghanistan in order to engage an enemy funded by countries whose gas we buy.

The shoes will cost ~US$2000 (includes shipping). If anyone's interested in playing angel to soldiers overseas and in harm's way, email me at chris dot zaharias @ gmail dot com letting me know how much you're in for, or PayPal me at the same email address; I'm not supposed to divulge the soldiers' names or exact location, but can give you more details in private.

No matter what we all think about the wars in Afghanistan and Iraq, they're our soldiers and are fighting our wars, right?

Thanks to everyone who helps out!

Monday, June 04, 2007

Searching For A Way To Help Our Soldiers

You can be for or against American foreign policy, but our soldiers are our soldiers and they're doing our elected officials' bidding in Iraq, Afghanistan and a number of other places. Helping them deal with the reality of dangerous fighting, extended tours and a generally anti-U.S. world falls to us U.S. citizens, then, but what to do?

One great way to help them is to join Soldier's Angels, an organization that links up people who care about our soldiers with the soldiers themselves. Join the group and they'll match you up with someone who's likely stationed in Iraq or Afghanistan; from there they give you lots of tips for how to help your soldier, from writing letters, to sending snacks they crave while overseas, to helping them deal with their family's daily life in absentia.

If you're searching for a way to help our soldiers, believe me, this is the way to go. Cindy Sheehan recently complained that 1% of America is fighting this war, so if you don't want her words to be true, sign up here.

My soldier and his platoon are about to receive more snacks than they've ever seen in their lives, along with ballistic-grade Oakley sunglasses for the looooong patrols they go out on regularly. Thanks to everyone who contributed!!!

Interview with Efficient Frontier Founder Anil Kamath

Back in mid-2003 Chris Moore at RedPoint Ventures asked me to meet Anil Kamath and talk about the keyword management technology he had built. I went into the meeting thinking I was going to tell Anil that he had no hope to build a business. Bid management systems already existed (GoToast and others started in 2002) and the SE's themselves provided bidding-related functionality as well.

Then Anil showed me a simple example of how Efficient Frontier (then Everest) algorithms worked better than anything on the market, subsequent to which I got down on my hands and knees and begged him to let me start Sales for the company.

Since then Anil's been busy building and enhancing EF's algorithms and technologies, so it's rare for him to see the light of day. However, he was interviewed recently by Eric Enge of Stone Temple Consulting and the transcript is here. The discussion is on one of my favorite topics: portfolio based bid management.


Google Analytics Alternative