Simulcra FTW!

How the “link economy” reminds me of post-modern theory i (tried to) read in college.

Read the rest of this entry »

an innovator’s dilemma in pay-per-click search?

What happens to advertising revenues when search gets better?

Last spring, I wrote a post about the conflict of interest with regard to algorithmic search results and paid search listings. This is a follow-up post, with a different spin on the same idea: that there is an “innovators dilemma“ with regards to search innovation.  Better search results may in fact cannibalize ad clicks, and thus reduce expected revenues for a given query.

Here’s an example case to consider – a query for: “Student Loan”

In theory, a search engine dedicated to organizing the world’s information should very much be able to provide valuable structured results for this term. So suppose a hotshot whiz-bang engineer comes up with an awesome way to amass and store up-to-the minute results on lenders, rates, reviews, etc. laid out in a clean, organized manner, and all based on what is most relevant to the searching party. What would be the response from the “business” side of the house that pays that engineer’s salary?

Here are some accumulated statistics from Google’s AdWords tools for “Student Loan” and 50 recommended related terms:

So – suppose this awesome new search innovation was so good that more queries were now satisfied by algorithmically generated data, and the percentage of clicks on ads falls in half to 3%. This search innovation just cost over $3 M in monthly losses. Take this example to its logical end, and the theoretically perfect search engine’s ad-click rate will be 0, as the algorithm will always answer my query, and 0 x any $CPC will always be 0.

There are two obvious rebuttals here:

1) Increase Price: the CPC that advertisers are willing to pay will go up. Because that price is determined by a second price auction bid stack, presumably destination sites are bidding based on the value of the arriving traffic.  It’s debatable whether or not this is exogenous to organic results yielded by the search engine. But perhaps if users are still unsatisfied with the awesome improvements described above, their clicks will prove more valuable to the advertisers, and thus advertisers will be willing to pay more, and the $14.31 will go up.

2) Increase Volume: If more total people search on the site, then total number of clicks may be the same, even if a smaller percentage of people click on ads. Perhaps the search engine can grab more market share because of its (newly improved!) ability to answer queries such as “student loan.” Or perhaps the entire search market will continue to grow. But, while this rationale may have been sensible over the past ten years, you can only replace margin with volume for so long, as you either reach market saturation (market won’t grow) or market dominance (you own the market, and there’s no more share to grab from competitors).

There’s more to say on this subject – in the meantime, I am hopeful, though pessimistic, that the technology press will start to think more critically about search business models — as they like to say in mutual fund prospectuses: “past performance is not a good indicator of future success.”

Update, 2/13/2011: since I first wrote this post, there have been many articles written about how search all of a sudden feels broken. Most of the criticism has to do with “content farms” such as Demand Media, that produce low cost (and quality) content, do aggressive SEO, and then make money from clicks on ads when searchers arrive and find crappy content. This is arbitrage, essentially. IMHO, this is essentially the same “Innovator’s Dilemma” dynamic as I describe above. Google makes money on clicks that occur at these content farm sites. By improving search relevance and not having those links listed in the organic results, they will cannibalize advertising revenue.

kindle book prices are plenty high…despite what publishers say…

I’m too cheap to buy a real deal Kindle, but I did grab the (free!) iPhone app as soon as it came out, and have to say – I’m impressed. I’ve read Dan Ariely’s “Predictably Irrational” (a fantastic read worth a post in and of itself…) and am currently reading “Dreams of My Father” by some dude with a funny name.

I gotta say – it’s a perfectly serviceable reading experience: fonts are crisp, bookmarking is easy, and as someone w/ a short-attention span, I appreciate being able to “turn the page” every few seconds.

But I think the $9.99 price point is too high, despite the whining from Publishers. Here’s why.

Obviously, there are savings in the marginal costs of producing each book, since there is no need for paper, printing, binding, shipping, etc. So I’d expect these cost savings to be reflected in the price.

But I’m also bothered by the lack of lendability in the current Kindle model, and think this restriction should be reflected in a deeper discount. I very much enjoy sharing books with friends and then discussing them. Always have. I am also a big believer in the concept of a public library. Both of these “scenarios” are challenged by the current Kindle model in which an ebook is tied to a single device. I can’t even lend a book to my wife. Lame. When we can’t read books that we get on loan (i.e. for free), we pay an implicit premium over a physical (lendable) book, as we need to buy everything we want to read on the Kindle.

The following is a very, very, back of the envelope look at relative per unit costs & revenues of physical and (unlendable) digital books, that I think supports the case that Kindle content should be cheaper. Or lendable.

This embedded spreadsheet contains a quick collection of sample data comparing “real” books to Kindle versions on Amazon:

Let’s first talk about the paperbacks:

a) Average price is $10.25 for “real”, $9.57 for Kindle…

b) BUT, marginal costs of physical production are 12.5% less for Kindle books, based on NYTimes article referenced above)and this is self-reported in that article…I frankly imagine it to be more.

c) AND, assume that 25% of the books that I read I get as loaners from friends, wives, enemies, co-workers, libraries, etc. With the Kindle, I need to buy every book that I want to read. So I buy more books. (this 25% number comes from 1 data point here = me…did I mention back of the envelope?)

Based on these assumptions, spreadsheeted out below, we see there is ~50% more profit per Kindle sale than from “real” sales, at least with paperbacks.

And all this, not to mention that the same NY Times article states that Amazon is actually paying more than $9.99 back to the publishers – as much as $13 per book to ease their concerns about the business model.  (I do not know how this compares to Amazon’s physical book subsidy, or not. Amazon’s payments to publishers is an interesting topic, but a bit beyond the scope of this little post…)

(Disclaimer #1: Yes, I know there are additional costs, royalties, marketing, etc, that go into a bottom line profit calc, but these are essentially equal between two forms, hence “relative baseline” qualifier)

(Disclaimer #2: I’m not really addressing the relative merits (“willingness to pay”) for Kindle vs. non-Kindle content. It’s all very debatable. My feeling is that there are pros and cons for each form, but ultimately, I am paying for the content more than the form…and that if anything, the physical form would command more of a premium than the Kindle form. Think: bookshelf-show-off-effect). But the goal here is to show that Kindle model is plenty profitable and prices could/should be lower. So this question, while interesting, is a bit tangential.

(Disclaimer #3: Would I really buy all the books that I am getting via loan from friends, etc?  Perhaps not…And of course I can still get physical loaners…for now…but even if I would only otherwise buy 2 out 3 books that I get on loan, the numbers work out in the Kindle’s favor)

Hard covers are a murkier picture, as the prices vary more widely, in both Kindle and “real” forms. I’m not totally clear why (though it does seem clear that Dean Koontz is a greedy bastard). But following the same approach, Kindle dollars still come out on top.

So: I say let’s lower the prices…or figure out a reasonable way to lend content…part 2 of this post will include a proposal for the later…

how the search business reminds me of wall street

Is there a conflict of interest between quality organic search results and pay-per-click sponsored links?

This starts with a simple story. My brother runs a small translation company in Berkeley called ION Translations. Like many small business owners, he has explored using Google’s uber-popular AdWords service to help promote his business.

“Berkeley Translation” is one the search phrases for which he currently bids, and indeed, if you search on Google, as of 2/14/09, you will see his ad listed on the right-hand side sponsored links list. You do not, however, see a listing for “Ion Translations” in the “organic” search results. So perhaps my brother needs to do some work on optimizing his site for search engines?

But…here’s where it starts to get a little fishy…if you search for “Berkeley Translation Service“, a phrase that my brother is not bidding for – lo’ and behold, there is Ion Translations, listed in the “organic” search results local link section. Note as well that otherwise the organic list result is very similar.

Curious. Is it indeed a quirk in the Google algorithm that by including the term “service” in the query that all of a sudden my brother’s company shows up in the organic search listings? Really? Or…(cue the organ music…) is there a more nefarious story about how Google is (not) rendering “organic” links based what sponsored links are associated with the query …?

Indulge me the conspiracy for a few, if you will, and let’s try and put aside the quirky lovable do-no-evil brand that Google has admirably built over the past 8 years. Let’s think plain and simple about incentives.

First, recall how the AdWords model works: advertisers bid for keywords, Google lists them based on expected revenue, and only when a user clicks on an ad does Google get paid. (see appendix for more info on how the model works…)

If Google aims to maximize the revenue of this single instance query, then of course they would not want links to the same page (iontranslations.com in this instance) in the “paid” ads section to also appear in the “organic” list. After all, if the same link appears in both lists, Google would risk losing revenue if the searcher clicked on the “organic” link. And from Joe Searcher’s point of view, he just wants to find what he’s looking for, and probably doesn’t really know or care how Google’s business model works and will just click away on the most relevant seeming link. The only person who would prefer the organic click to the paid link click is the advertiser who would pay in that case (in the longer term the customer would also prefer that the advertiser not be unnecessarily charged, as higher costs would be passed along…)

…But of course, life ain’t so simple…and in Google’s (and all other search engines’) defense, let’s kick a little lay-man’s game theory. Searching is not a “one shot game.” Google has been able to build and maintain its formidable lead as the top search engine because users trust its results.. Trust is built over time, by delivering stellar service time and again. Any kind of public disclosure of the type of tinkering to drive revenue would really be damaging (I think, right?) But it would also be exceptionally difficult to prove, as Google’s algorithm for ranking results is a proprietary black box.

Given that Google’s search mechanics are, unlike many of the other offerings from the company, not “open”, I can’t help but be suspicious with the structure of the paid search listings business model. For these more fringey searches, like “Berkeley Translation”, for which we know there is a very long tail, might a search engine be able to get away with it?

In fact, I think there is a darn uncanny parallel here to the conflict of interests in investment banking that have ebbed and flowed throughout the years.

In the banking world, there are analysts and traders. Analysts are supposed to be objective and report on the fundamental strength or weakness of securities. Traders buy and sell and make money off of commissions. But as we saw in the dot-com boom, analysts have the power to manipulate information flow, in such a way that unsuspecting sheep (the rest of us) buy or sell stocks and allow the traders (and analysts – see Blodget, Henry) to make money because of market swings that they can anticipate. Sarbanes-Oxley, in fact, was largely a response to this condition. And banks also had the same reputation interests to protect, as banking is not a one-shot game either.

So – let’s transpose: in the search world, “organic” results are the analysts (an objective, scientific, heuristically driven list of links sorted by relevance), and paid listings are the “traders” (commissions are made based on trades/clicks.). And there they are – sitting on that sparse page together, with the tenuous white column serving as the firewall. If the analysts happen to have a moral slip-up and manipulate the objective information…they can drive business to the traders…

OK OK OK, I’ll stop channeling Michael Moore. To be clear – I am not accusing Google of any wrong-doing. I do actually think this is a strange quirk of circumstance. But – and this is really my point: how can I be sure?

The bottom line is that this model deserves more scrutiny than it’s received. And not just Google, but Yahoo, Microsoft, etc…who all have essentially the same approach. There is something quite troubling about these incentive structures, especially given search has become the way in which we gather and consume information. All we can do is trust that companies are doing the ethical thing…which we know all-too-well doesn’t always work out…

Appendix:
The paid keywords business model, used by Google and others, is that advertisers pay if and only if a searcher clicks on the advertisement. The advertisers have stated a willingness to pay for a click, and the search engine calculates a likelihood of a click, and then ranks the list based on the product of those two numbers: (Probability of a Click) x (Willingness to Pay). The special sauce is really in the first number – the probability of a click, which is based on lotsa data from previous queries, which is why Google has such a strong advantage in this space.

(Yahoo used to charge for listing, not for click, which had its own issues…but I’ll leave a comparison on the sidelines for now…).

Full disclosure: I work for Microsoft, not on anything related to search. I am writing this on a Mac. I use a bunch of Google products, which are for the most part very very very good. But I try not to click on their ads. When I can actually tell that they are ads.

Fighting “Thong” Adsense Ads on FutureMe

Google does a lot of neat things. But AdSense is just not good yet. At least for small sites with not very much text like FutureMe.

For about the past four months, virtually *all* ads running on the footer of the front page of futureme.org have been something along the lines of “Buy Thongs” “Discount Thong Underwear”, and “Visit i-thong.net.” (I have since been waging a battle to screen out these ads — see postscript — so you may not have the pleasure of seeing these ads, but trust me. Thongs.  Everywhere.)

Huh?

What’s especially curious, is that the actual destination of several of these, is not actually a thong superstore, but info.com (a search engine aggregator) which could have many, many different link tags, but somehow “Find Thongs” was the text chosen for the ad.

So, Mr. Google, just what about FutureMe – a site where people write letters to themselves in the future – would lead you to show ads for buying thongs? And not one ad. Three ads. Repeatedly. Again, and again, and again. Thongs. This is not randomness. This is systematic displaying of ads about thong underwear.

Perhaps I am just getting reactionary in my old age, but I have a hard time believing that thongs are really the best thing to advertise on FutureMe. I would think that attractive ads would be more along the lines of: financial services, weight loss, continuing education, job sites, etc — things related to self-improvement.

But perhaps Google seems to feel, apparently with utter and resounding conviction, that a primary means of self-improvement is to acquire lots and lots of thongs.

Or, maybe this is indeed the work of one of the Google’s taxonomists trying to create a self-fulfilling prophecy: people go to a site where the think about the future -> people see thong ads -> people associate thongs with their future selves -> everyone wears thongs -> weirdo taxonomist is happy!

I’m confused. And disturbed.

postscript: using Google’s “competitive ad filter” I was able to screen three of these sites.

but then another thong site showed up!  so i screened that one too.

the battle may be won, but i fear not yet the war. 

<rant> but i just don’t understand why Google has not introduced a more sophisticated means of having site owners *help* to target ads. the reality is that FutureMe is sparsely (but effectively) designed (nice job Jay!) and there is not much text to parse. so I want to be able to add terms in my AdSense account about *how people use the site* which is *different that what the site says*, and I would argue, much more relevant to what kind of ads to serve…</rant>