PPC - Still Misunderstood.

Filed Under (the PPC Book) by Jeff Hudson on 24-12-2008

A client emailed yesterday and asked for a report that showed their conversion rate for 2008 and how that compared to 2007.

sigh…

A question like that is my fault. Education is just as important as management, and at the end of the day the education part is the first to go when you’re busy.

How about a report with:

Spend
Revenue Generated
Top Selling SKU
Revenue by SKU

(try getting those numbers from your yellow page rep)

John over at ppchero highlights more of this education gap.

The reality is, as long as some of us feel we’ve been in this industry, it’s still new new new. And it’s still a black box in the eyes of the average SMB. At the end of the day what your client needs to realize is it’s not a black box when it comes to performance, tracking, and measurability. I’ll take this black box over a year long YP contract or Superbowl ad any day.

After Christmas break, take your clients back to class when you’re planning 2009. It will help us all in the long run.

Die Experimenting?

Filed Under (AB Testing, the PPC Book) by jennifer on 05-12-2008

Tagged Under : , ,

Or Experiment or Die?

My name is Jennifer Webster, PPC amateur and aspiring statistician. If there is both an art and a science to PPC management, my role here is to expand on how science - or at least statistics - can help you make the most of your campaigns.

I’ve spent most of the last week playing around with the Google Website Optimizer, and I’d like to start out my contributions here with some thoughts on website optimization and experimental design. First off, the Website Optimizer is yet another well-designed Google product with enough features to keep math geeks like myself up many nights playing and exploring.

Over on the Google Website Optimizer blog, the taglines include “Experiment or Die!” and “Always Be Testing.” Clearly, Google is pushing for more campaign optimization, although it is notable that most of the blog entries are written by Google Authorized Consultants, who make their living running website optimization experiments, so perhaps they have some motivation to get more people optimizing.

The most important part of any experiment is the design. A well designed experiment is an extremely powerful tool that may give you (or your client) a huge advantage in the marketplace. A poorly designed experiment is a meaningless waste of time. This “Experiment or Die!” mentality worries me. Experiment for experiment’s sake leads to what we statisticians call “analysis paralysis,” where you have so much data and so many, often contradictory, conclusions that you can’t make any useful decisions.

A key element of experimental design is power. Does the experiment you’re conducting have enough power to actually prove any thing? Power comes from two elements: the number of items that you’re sampling (unique website visitors) and the size of the effect. Its much easier to detect a large effect (like a 50% improvement in conversion rates) than it is to detect a very small effect (like a 1% improvement). In PPC website optimization, the limiting factor is your number of unique visitors. If you get millions of unique visitors, your experiments will have higher power than a site with a few hundred or thousand visitors per month, and you can experiment to your hearts content. With fewer visitors, you have to be a little more thoughtful about how you design your experiments

I started my analysis with a simple A/B split test, the most basic website optimization experiment. Assume you have two versions of your landing page. You drive traffic half your traffic to one page and half to the other and track conversion rates over a set number of unique visitors. Below is a graph of the power of that experiment assuming four different effect sizes: conversion rate improvements of 5% (green), 10% (blue), 25% (red) and 50% (black).

A finding that surprised me: if you’re only expecting a 5% improvement in conversion rates, 100,000 unique visitors only give you 25% power to detect the effect. That means two things: if one landing page really does convert better, you only have a 25% chance of detecting that, and if you see a difference between the two pages in your experiment, there’s only a 1 in 4 chance that the difference is real. So if you already have a well optimized sites and you’re constantly testing to improve 1% here and 2% there, it is likely that you’re wasting a lot of time and energy chasing after trends that may or may not actually be real. However, for a site that’s never been optimized or is being significantly overhauled and is expecting improvements on the order of 25-50%, those tests are generally well powered with only 10000-25000 visitors. Predicting effect size is really just an educated guess based on what you already know about your site and your campaign.

Moving from an A/B split to a multivariate design only compounds the problem. In a multivariate design we take two or more elements of the campaign and test two or more versions of each. As a test case, let’s take the two landing pages from the previous example and add different versions of the ad copy. Now, rather two variables (Landing Page 1 vs. Landing Page 2), we have four:

• Landing Page 1, Ad Copy 1
• Landing Page 1, Ad Copy 2
• Landing Page 2, Ad Copy 1
• Landing Page 2, Ad Copy 2

And the power curves look like this (50% - black, 25% - red, 10% - blue, 5% -green). Note that the scale for number of visitors now runs from 0 to 1,000,000.

The practical question that comes out of all of this is how many visitors do I need before it’s worth my time to do an optimization experiment? In genetics, we consider an experiment to be well powered if we have 80% power to detect the given effect. In the table below are some rough estimates for the number of unique visitors necessary to provide 80% power for a variety of different optimization tests and effect sizes. Keep these numbers in mind as you’re designing your optimization experiments. Carefully select the variables you’d like to change, and let the experiment run long enough to give you meaningful results.

Next week, more on the fundamentals of split testing and multivariate testing.

Barack’in the Blog

Filed Under (the PPC Book) by Jeff Hudson on 03-12-2008

Couple quick updates here in PPC Bookland.

1. Taking my cue from Barack Obama, the best thing to come out of Chicago since the 85 Bears, I’m bringing in some voices with alternate viewpoints on paid search. Why? Because I’m always focused on the art of paid search, and less on the science. So I went hunting for some scientists, and found one, and we’ll be publishing a few guest posts here as an ‘experiment’. I will look for your feedback as this develops.

2. Take the time to listen to Perry Marshall’s interview on PPC Rockstars. Perry’s into the art of paid search, and his writing is always creative. There are a few nuggets in the interview that are worth your time.

3. From my very unscientific analysis, Thanksgiving did not bring a significant drop to my overall November traffic in the larger campaigns I manage. I was extremely surprised and it messed up a few of my budgets. Other than seeing a dip on Thanksgiving day, traffic was almost back to normal Friday and over the weekend. What this tells me:

Nobody takes time off anymore

Nobody likes their relatives

The Detroit Lions being on television is actually good for online commerce.

Anyhow, no complaints from me.

I do have a question for you though:

What are you planning for budget, as a percentage of normal, for the last 2 weeks of the month?

Lies, Damn Lies, and Newspapers

Filed Under (the PPC Book) by Jeff Hudson on 26-11-2008

Tagged Under :

Title of an article in the PRINT VERSION business section of the Star Tribune this morning:

INTERNET AD REVENUE PROJECTED TO DROP

“eMarketer estimates US advertisers will spend $25.7 billion on the Internet next year - about $2.7 billion, or 10 percent, less than a forecast from just 3 months ago.”

Here is the
online version of same article.

C’mon. Is that just a wee bit misleading? Sounds to me like the PROJECTION is dropping. Not the actual revenue. But hey, who’s paying attention anyway?

Just to clarify, Ad Revenue is not going to drop, it’s actually going to grow to the tune of an 8.9% increase over 2008

Nice try Mr. Newspaper Publisher. Enjoy your gold watch and pending retirement. Maybe I’ll throw you a bone on the content network every once in a while.

Hear me now, believe me later

Filed Under (the PPC Book) by Jeff Hudson on 25-11-2008

A tip for you advertisers out there.

When your PPC Analyst:

Gently suggests you current landing page is holding you back…

Offers to build you a better landing page…

Says they have a good hunch it will work out better for you…

LISTEN TO THEM

You’ll thank me later…

Busy?

Filed Under (the PPC Book) by Jeff Hudson on 20-11-2008

Multi-tasking and wide skill sets are at a premium in this crazy industry.

For example, here’s what I’ve had my hands on in the past day. IN THE PAST 24 HOURS:

adwords
YSM
MSN Adcenter
facebook
twitter
basecamp
xcart
google analytics
freshbooks
wordpress
magneto
elance
odesk
guru.com
excel
word
outlook
iphone
iphone apps
adwords editor
permutator
IM
Authorize.net
cpanel

I could probably come up with 20 more, as could you, but you get the idea.

A quote that was just sent to me by my minneapolis jiu jitsu instructor struck me as particularly timely.

I do not fear your ten-thousand techniques.
I fear the one technique you have practiced ten-thousand times.

- shaolin proverb

Last week - Crush it.

Filed Under (the PPC Book) by Jeff Hudson on 17-11-2008

Tagged Under :

Just a friendly reminder. If you’re managing a PPC campaign that is budget driven - this is basically the last week of November for all intents and purposes. Better press to be ahead of schedule going into next week!

Good luck!

The New New Shopper

Filed Under (the PPC Book) by Jeff Hudson on 14-11-2008

Tagged Under : ,

SO retailers are desperate and sales are down and no one is buying flat screens at Best Buy. Gloom and doom persists. How can you find those new shoppers no one else is targeting? Google has an idea for you…

In the realm of ‘fundamental’, but important, Amanda Kelly over at the Adwords blog reminds us of the importance of mining for new queries. I’ve seen numerous pie in the sky estimations about % of queries that new, and Amanda dishes an interesting number:

Did you know that 20% of the queries Google receives each day are ones we haven’t seen in at least 90 days, if at all?

**finger on chin…thoughtful pose….knowing nod of head**

Imagine. You own a shop and 20% of the people walking outside your store didn’t even exist yesterday. They appeared out of thin air. And they are window shopping. They’re interested in what you sell but don’t know who you are.

Here’s how to strike up a conversation and lure them in…

Adwords recommends that you run the Search Query report to find these NEW NEW keywords.

That’s a good start, and certainly part of the solution, but you need to dig deeper than that. There are 2 other options that you can use in order to get the real data:

1. Google Analytics Keyword Filtering. Hands down the best data you’ll ever get for free. If you are running broad match keywords (admit it, you are), this step is absolutely necessary. If you’re not, I should take away your logins ;)

2. Tag and Bag (it’s hunting season up here, forgive me) - If you are using a 3rd party analytics platform, this is for you. In your Adwords settings, tag your urls. Now login to your analytics platform and pay attention to what keywords you’re seeing. Log files might even be more accurate, but are harder to work with.

3. Bonus extra - Revenue by product and keyword

In Google analytics - visit:

>Ecommerce Overview
>Product Overview
>Sort by Revenue ***Now you are looking at your biggest selling product
>Click on the product name link
>Select: Dimension - change to Keyword

Now you are looking at the keywords that drove sales on this product and can segment the best revenue producers.

I used this last night to find 12 new keyword variations that were not in a campaign and were driving revenue for the website.

Try it, you’ll be surprised at what you find. Go find some NEW NEW shoppers!

Quality Score tweaks

Filed Under (the PPC Book) by Jeff Hudson on 31-10-2008

Yesterday the Adwords blog detailed some improvements being made to the quality score algorithm. I found them to be very interesting.

1. More precise Quality Score calculation

The most telling comment:

Clickthrough rate (CTR) is the most significant component of Quality Score because it directly indicates which ads are most relevant to our searchers.

I disagree with this totally, but that’s beside the point. High CTR will make Google the most money, but does not mean the ad is ‘most relevant’.

Anyway, we’ve always been told that CTR was normalized for position, and apparently it wasn’t, really.

In the coming days, we’ll update the portion of the Quality Score algorithm that accounts for ad position

To me that means it was taken into account, but not all that strongly.

2. Higher quality ads above the search results

This is an interesting revelation - the concept of a ceiling - or threshold that would keep all the ads down.

To appear above the search results, ads must meet a certain quality threshold. In the past, if the ad with the highest Ad Rank did not meet the quality threshold, we may not have shown any ads above the search results.

Secondly…

With this update, we’ll allow an ad that meets the quality threshold to appear above the search results even if it has to jump over other ads to do so.

Good, so we don’t all get punished for poor ad quality by our competitors. I believe this new approach will benefit skilled PPC managers the most.

I like!

More thoughts from our PPC friends on this subject:

Ad Position, CTR & Quality Score - We’re Going to do What we Already Did

Google Updates AdWords Quality Score To Be “Fairer”

All I really want for Christmas is a separate Search Partner Campaign.

Filed Under (the PPC Book) by Jeff Hudson on 20-10-2008

Google Adwords continues to slowly move the transparency meter when they don’t really have to (commendable because they have no competition). Last Thursday it was announced that Adwords will now show search partner data separately from google search.

Now, we show one set of statistics for Google and another set aggregating search partner performance. Search partners include AOL, Ask.com, and many other search sites around the web. You can view ad group or campaign performance at a summary level, or broken down by different combinations of Google, our search partners, and our content network. Additionally, separate Google and aggregate search partner statistics will soon be available in the Report Center.

PPC managers have been calling for this development for some time. The problem lies in the fact that Google will allow you to run Google + Search Partner, or Google only, but not Google Search Partner only. The ideal solution would be to create a separate Google search and search partner campaign, both identical, but with separate distribution. Similar to the best practice of separating Search campaigns from Content (you’d better be doing this by now), we need the ability to do the same with Google and the Search Partners.

Why don’t they offer this? They’re afraid to, basically. If everyone starts opting out of large parts of their networks they’ll lose a not insignificant portion of their revenue, of course. I disagree with this notion, however. Has revenue dropped with better transparency in the BIG BAD content network? No, because advanced ppc analysts know that this is where the growth lies in every campaign. Search can take you from 0-60mph, but the content network (well controlled), can take you to F1 type speeds.

C’mon Adwords. This is a nice tease, but give us what we want.

All I really want for Christmas is a separate Search Partner Campaign.