Setting up a new digital ad campaign or hiring an agency/someone to do it? Read this Harvard Business Review article on some techniques of which to be aware and that can often overstate performance for a given tactic.

Imagine you run a retail store and hire a leafleteer to distribute handbills to attract new customers. You might assess her effectiveness by counting the number of customers who arrived carrying her handbill and, perhaps, presenting it for a discount. But suppose you realized the leafleteer was standing just outside your store’s front door, giving handbills to everyone on their way in. The measured “effectiveness” would be a ruse, merely counting customers who would have come in anyway. You’d be furious and would fire her in an instant. Fortunately, that wouldn’t actually be needed: anticipating being found out, few leafleteers would attempt such a scheme.

In online advertising, a variety of equally brazen ruses drain advertisers’ budgets — but usually it’s more difficult for advertisers to notice them. I’ve been writing about this problem since 2004, and doing my best to help advertisers avoid it.

Overstating the Effectiveness of Sponsored Search Campaigns
A first manifestation of the problem arises in sponsored search. Suppose a user goes to Google and searches for eBay. Historically, the top-most link to eBay would be a paid advertisement, requiring eBay to pay Google each time the ad was clicked. These eBay ads had excellent measured performance in that many users clicked such an ad, then went on to bid or buy with high probability. But step back a bit. A user has already searched for “eBay.” That user is likely to buy from eBay whether or not eBay advertises with Google. In a remarkable experiment, economist Steve Tadelis and coauthors turned off eBay’s trademark-triggered advertising in about half the cities in the U.S. They found that sales in those regions stayed the same even as eBay’s advertising expenditure dropped. eBay’s measure of ad effectiveness was totally off-base and had led to millions of dollars of overspending.

Now, eBay is unusual in its dominance of U.S. consumer auctions. Your company is probably less fortunate in the markets it serves, and if you don’t buy your trademark as a keyword to show your search ads, Google will try to sell your trademark to your competitors, a tactic which some courts have allowed. But if a user searches for Dell, an ad for a competitor like Lenovo tends to underperform. Some users may be willing to consider an alternative at Google’s suggestion, and others may be tricked or not realize the difference, but at least a portion will recognize that Lenovo is something else entirely.

A recent study by researchers at the University of Chicago generalizes these methods and shows that buying your own trademark tends not to be as good an investment as standard measurement tools suggest. Tempting as it may be to increase spending on these (supposedly) “top-performing” keywords, I’d advise the opposite: Cut them, perhaps all the way to zero.

Overtargeting Display Ads
Another problem arises with “retargeting,” which recognizes consumers who didn’t make purchases. The logic: if you went to Expedia and looked at a hotel but didn’t make a reservation, Expedia will arrange for its ad to be shown as you browse the web in the coming days. The banners can be eerily precise, often promoting the specific properties you considered. This approach makes it easy to click back to where you were and complete the purchase.

Here too, standard metrics indicate that the campaign works. No doubt the folks who browsed at Expedia are good candidates for buying from Expedia. Showing banners may remind them to do so. But how many of them would have made a purchase anyway? Certainly not zero. (Consider the traveler who was waiting to finalize his itinerary, perhaps awaiting confirmation from a friend or a business associate.) Yet most measures of ad effectiveness will give full credit to the retargeting vendor — asserting, falsely, that had it not been for the retargeting banner, the user would not have purchased. This hasty analysis leads advertisers to run retargeting campaigns that appear to yield profitable purchases more than sufficient to cover retargeting costs. But if an advertiser considers that some of the sales would have happened anyway, the appeal of retargeting campaigns necessarily diminishes.

It turns out that even demographic targeting of banner ads (without retargeting) is also at risk. Suppose your company is fortunate enough to enjoy popularity with a given demographic group — say, 40% market share among men aged 18 to 25. You might target banner ads to that same group, hoping to reach the 60% of customers in this group that you don’t yet serve. But remember that you’ll also be addressing the many customers that already use your offering. You might falsely attribute “success” to a campaign that prompted purchases from the customers who were going to buy from you regardless.

My advice: try a randomized experiment. Take a portion of the users who would have seen a retargeting campaign or a demographically-targeted campaign. Rather than showing them your ad, decline to advertise to them, and track how many of them buy anyway. If 20% of them still make a purchase, your ads are actually 20% less effective than basic measurements would suggest.

Paying for Affiliate Sales That Would Have Happened Anyway
Affiliate marketing is supposed to align incentives perfectly, paying only for success — like a 10% commission if a user actually buys a given product, but zero for impressions and clicks. Is fraud “impossible,” as some have claimed? Not at all.

Consider a sneaky affiliate who “stuffs” a cookie on every user’s computer as the user browses an unrelated web site. With a moderately popular site (or a banner or widget on someone else’s site), this “cookie-stuffer” might claim to have referred millions of users to a given merchant. Some of those users are bound to make purchases, and the merchant will pay the affiliate a commission as if it truly caused the user’s sale. Worse, the merchant is unlikely to suspect the problem; with real sales, merchants are often slow to realize that some customers’ decisions are uninfluenced by any affiliate marketing activity.
Mere speculation, you worry? Not so. In 2008 indictments in San Francisco, Shawn Hogan and Brian Dunning were charged with wire fraud for using these methods to claim more than $20 million from eBay. They were, for a time, eBay’s largest two affiliates, and they report that eBay wooed them with dedicated account managers, chartered jets, and more. Only years later did eBay realize it was being swindled. (Disclosure: I advise eBay on certain aspects of affiliate marketing fraud, and litigation records indicate that I uncovered Hogan and Dunning’s activities.)

The take-away: if you run an affiliate marketing program, you shouldn’t assume it’s fraud-free. A good start is to know your affiliates — browse their sites to examine their offers and approach. Some affiliates try to keep their sites secret, claiming that merchants might copy their proprietary methods. I can understand their worry, but if they want to get paid, they should expect reasonable oversight. If an affiliate’s site doesn’t look quite right — too ragged for the volume it reports, too hasty, or otherwise not quite right — you should push for specifics and check third-party sources like affiliate network staff, server logs, and online discussion forums to try to confirm your suspicions.

Measuring Success When Adware Intervenes
When users’ computers are infected with advertising software like adware and malware, advertisers are at still greater risk of being separated from their money with little to show for it. I’ve tracked adware that covers advertisers’ sites with their own pay-per-click ads, so that a user browsing (say) rcn.com sees paid links for RCN rather than (or on top of) the genuine RCN site, prompting unnecessary clicks. I’ve found banner injectors that insert ads into other companies’ web pages without permission from those pages, and certainly without paying those pages’ publishers. Remarkably, some injectors insert an advertiser’s banner ad into its own web site — a particularly outrageous scam against the advertiser, which then pays to retain a user it already serves. (An example is Revizer adware and ad network Criteo charging Zappos for users already on Zappos.com.) Adware can also monitor a user’s browsing, then invoke the affiliate link to the site a user is about to buy from.

Adware tends to be particularly tricky to uncover since testing is so difficult. Advertisers are rarely willing to set up testing labs to see adware in action first-hand. As a stopgap, consider insisting on higher standards from responsible networks. Revise contracts to allow for clawback of any payments later shown to result from adware. While you’re at it, you might look for one-sided terms throughout an ad network’s contract; ad network defaults tend to protect their interests only, disclaiming every possible warranty or guarantee to leave advertisers vulnerable if anything goes wrong.

The Way Forward: Aligning Incentives for Marketing Managers
It’s probably no surprise that advertising networks offer services that aren’t in advertisers’ best interests. An ad network is an advertiser’s vendor — fundamentally, not a genuine partner or ally, but a supplier whose direct interest is charging more for doing less. Savvy advertisers should view the relationship accordingly.

How about ad agencies and advertising buyers? Advertisers often pressure agencies and buyers to deliver near-impossible results. They often face tough demands — “20% more customers for 10% less money” and so forth — and cutting corners can feel almost unavoidable. I understand advertisers’ insistence on results, and I share it. But when measurement is imperfect, an excessive focus on measured results invites vendors to game the system with tactics that advance measured indicators without genuine results.

I’ve even seen instances in which a company’s in-house staff become complicit, knowing that vendors are up to no good, but afraid to call them out on it. Companies almost invite this behavior through bonuses and performance objectives — “$10k extra if you increase ROI by 10%” — yielding temptations too enticing for some to resist.

Advertising is hard work. Short of the rare product that practically sells itself, advertisers and their vendors should expect to hustle to find scalable and cost-effective methods. When you see a new tactic delivering outsized results, you might ask yourself whether it’s too good to be true. Sadly, often it is.

Original POST

Author: Benjamin Edelman