Rand Fishkin recently discussed the accuracy of visitor data for SEOMoz site traffic and the sometimes egregiously incorrect data provided by these tools. He rallied a call-to-arms for Webmasters to share the level of deviation on their own sites and form an industry consensus. We took Rand’s spirited dissertation to heart and analyzed a large (anonymous) site, comparing actual data with the public estimates.

Alexa.com – Lots of Data (Most of It Wrong)

Alexa gets credit for offering more data than other free public services, including search keyword traffic, clickstream data and audience demographics. The only issue being the extremely inaccurate nature of this information.

The top queries from search traffic were not indicative of actual organic search activity, with a few queries not appearing at all in the real data. Perhaps most inexcusable was the “% of traffic from US”, comparing Alexa’s 74% with the actual 96%. This is an important metric to advertisers and the discrepancy here could very much impact our business opportunities.

The demographic of site visitors has been shown to be Males, 35-44, (confirmed through a number of methods) but Alexa interpreted our visitors instead as Females, 45-54, which is the exact opposite from reality.

Compete.com – Under-reporting Visitors & Proud of It

Compete reported Unique Visitors at 57.92% lower than actual data, though some penance is given with the option to export CSV data. They might be forgiven a margin of error for sudden swing months, though this large discrepancy is averaged over 7 months time.

Actual Data: 4,596,476 unique visitors 

Compete Data: 1,934,332 unique visitors

DoubleClick – Google Gets It Wrong

You would expect the master of data to have accurate numbers, especially considering that the crux of their revenue stream depends on adoption of AdWords and AdSense. The margin of error is not much different than Compete and Alexa at 63% under-reported Unique Visitors and 24% under-reported Pageviews.

Unique Visitors:

• DoubleClick: 68,000
• Actual: 185,733
• Difference: 63.39%

Pageviews:

• DoubleClick: 830,000
• Actual: 1,086,206
• Difference: 23.59%

Quantcast – Great if Quantified (Otherwise Not)

Electing to place a Quantcast tag on your site (hereafter known as being Quantified!) allows you to publically share real data. The tag does not estimate, nor does it guess, but instead provides accurate and reliable data. However, not everyone is comfortable with this level of transparency and instead relies on Quantcast to estimate their traffic. A quick review of 3 large sites shows unique estimates under-reported by 85% – 90%. Not even close to reality. Similar to Alexa’s missteps on audience demographic, Quantcast viewed our Males, 35-44, site instead as Females, 50+.

ComScore  – You Pay for the Best of the Worst

The only paid service mentioned here is noticeably more accurate than the alternatives, showing only a 12% pageview under-reporting discrepancy compared with real data. Being the service most widely respected, it was refreshing to see ComScore actually coming close to providing true analytics.

Actual Data: 10,161,527 pageviews 

ComScore Data: 13,519,743 pageviews 

The goal of these traffic tools is to offer a public face for your site without giving away all that valuable data. So in that context estimations are perfectly useful, though a margin of error between 10%-15% is not an unreasonable request for general accuracy. All tools error on the side of conservative under-reporting, which is an understandable tendency, though the degree of discrepancy should still fall within moderate boundaries. It is interesting, however, that many tools show a similar magnitude of error around 60% under-reported, perhaps indicate that the same (inaccurate) data source is referenced by multiple tools.

Without getting into the technical aspects of the change, the biggest potential impact would be on those of us who use keyword-level data to optimize site content, layout, user experience, etc.

Despite Google being very clear that this change would only impact less than 10% of queries, our initial reaction included questions such as, “How will we be able to optimize our site for branded vs. non-branded searches?”, “How can we connect user search intent to on-site behavior (i.e., conversions)?”

Before we get into the data we have seen over the past 10 days, here is an example of how we use keyword-level data to optimize a site:
Let’s say we have 20,000 users arriving on a site from organic search. We can divide those users into a few different groups to learn what type of content various users are looking for, what browsers and devices they are using, etc. Perhaps we want to compare people who search for brand terms (e.g., Jeep, new Jeep Wagoneer, etc.) to those who are searching for industry/category terms (e.g, SUV Reviews, new SUVs 2012, etc.). Using advanced segments in Google analytics, we can compare these two groups relatively easily and find that (as an example) those searching for brand terms are 3Xs more likely to download a brochure while those searching industry/category terms are 5Xs more likely to be on a mobile device and 12Xs more likely to search for a nearby dealer. Using this data we can change a site’s experience to better serve our various organic search audiences.

We’ve been keeping a pretty close eye on the data since Google implemented this change and, in general, it has had very little (if any) impact on our ability to measure organic search traffic, track users’ paths through the site, see where users are leaving, what devices users are searching from, etc.

Looking at data from a few different sites that we work on across several industries, here is the percent of organic search data that has been affected:

News/Magazine Site: 1.6%
Insurance Company: 0.9%
Home Improvement Company: 0.7%
Recipe Site: 1.5%
Food Company: 1.2%
Food Company: 1.5%
Pet Products Company: 1.2%
Luxury Home Goods Company: 1.3%

Just to make sure the data above is clear, using the News/Magazine site as an example, of all of the organic search traffic going to that site (over 15,000 visits over the past 10 days) only 1.6% or 250 visits were ‘blocked’. In other words, instead of seeing what queries brought those 250 search visits to the site, we see “(not provided)” as shown in the screen shot below.

So, at the end of the day, it’s not that concerning. Until that number starts to approach the 10% – 15% point, we can still do all of the keyword-level analysis we did before (including the example above) and feel comfortable that we are making decisions based on statistically sound data.

To review…

• This change only affects organic search data for users who are signed into their Google account.
• It does not impact your ability to appear in organic search results.
• This is not a change to Google’s search algorithm. Nothing is different in terms of what it takes to appear in organic search results.
• This does not impact paid search data. Those users’ queries will still appear in the data.
• The change will be seen in all analytics platforms, whether you are using Google Analytics, Webtrends, etc.
• This only applies to Google searches. Yahoo/Bing search queries will still be displayed.
• So far, the amount of data being blocked is so small that it is not worth worrying about.

If anything, it might even provide new data that we did not have access to before. For example, if we start to see that the (not provided) traffic really likes subscribing to our RSS feed, we definitely need to make sure that our feed appears correctly in Google Reader (since we know those users are signed into a Google account and are more likely to use Google Reader to manager their RSS subscriptions).

We’ll continue to watch the data (as we tend to do), but don’t expect any massive changes to your ongoing organic optimization efforts, measurement ability or traffic patterns.

We are often asked to justify the amount of money and resources spent on paid search, organic search, social media, user experience optimization and other online marketing tactics when a site doesn’t have a clear ecommerce conversion (i.e., it doesn’t generate direct revenue through the sale of products, fee-based memberships, etc.). Despite the lack of an ecommerce conversion, most sites still have goals – or some desired action that users should take.

These actions can range from downloading a file or watching a video to calling a phone number or signing up for a newsletter. All have value, and that value can be attributed to keywords (paid and organic), display ads, Facebook posts, referral links, etc.

The Value of Micro Conversions

Downloading/Printing a Coupon

If your site offers coupons, chances are you are paying to get those coupons out through other channels. What are you paying to mail your coupons? If you spent $10,000 sending coupons to 10,000 people, using a 10% open rate, you spent $10 per open. Apply that $10 to every digital coupon download. The argument could even be made that online coupon downloads deserve a higher value than direct mail because the user asked for it as opposed to receiving it unsolicited.

Email Sign-Ups

If you have an active email newsletter, there are several ways to arrive at the per email value of that tactic. Are you already paying a company like Response Media to acquire email addresses for you? If so, you already know what each email address is worth (since you are paying $X for each address anyway). Do you have ads in your emails? If so, divide the ad revenue by the number of subscribers to find the ad revenue per email address.

Watching a Video

If you are running TV ads or pre-roll advertising, you have all the data you need to put a value on each video view on your website. Let’s say you spent $10,000 to run a TV ad that was viewed by an estimated 50,000 people. Each online video view, in this case, should be worth at least $.20. Again, the argument could be made that an online view deserves a much higher value than a broadcast view because it is 100% accurate and user-activated.

Calling a Phone Number

Your business offers a professional service of some sort (e.g., law firms). You know that the average lifetime value of any given client is $10,000 and 1 out of every 50 callers turns into a client. Using this data, you can assume that every phone call is worth $200.

How to Use the Data

Now that you have a value defined for each desired action on your site, you can start applying it to traffic sources, visitor types, days of the week, etc.

Which traffic source has the best ROI?

Once you have all of the data for a given time period, apply the value of each conversion to arrive at the total value for each traffic source. Then, using the costs associated with those traffic sources incurred during the same time period, you can estimate the ROI for each source.

What time of day should I publish Facebook posts?

Analyzing user behavior at various times throughout the day can reveal some interesting trends. As you can see, Facebook posts are most successful at driving traffic to the site early in the morning, mid-afternoon and again in the early evening. We can also see that users are most prone to watch a video during lunch and at 7PM. With this data we not only know when to post to Facebook, but the type of content to promote at certain times of the day to earn the highest return possible.

This example is particularly useful because Facebook posts do not differ significantly in time/effort from one post to the next and they are almost always presented to the same audience. Those two variables being constant, the comparative value of these actions becomes even more accurate and compelling.

This type of valuation can get really fun when we start to apply values to Facebook likes, comments and other metrics associated with social engagement, but that’s another post

At the end of the day, this type of analysis is based on our belief that simply understanding how many visitors your site has, or the overall CTR of a paid search campaign is not sufficient data to truly understand whether specific tactics are effective and worth the time, energy and money.

We have organized these moments into the following categories:

• Organic Web Search
• Paid Text Ads
• Organic Image Search
• Organic Video Search
• Organic Maps Search
• Social Networking Sites
• Mobile Search
• Website Landing Pages
• Analytics and Measurement

Download the PDF to learn 50 Ways Search Marketing Can Change Consumer Behavior

Despite the bad economy, search engine marketing (SEM) spending continues to grow in the US. According to the “Search Engine Marketing in 2009” report by [x+1], 65.4% of senior-level SEM executives plan to spend at least as much on SEM in 2009 as they did in 2008. In fact, 13.1% percent want to increase spending by more than 20%.

The article goes on to explain that 60% of companies who were using search as a lead generation tool, were not satisfied with the results. Only 1 in 5 expecting their search efforts to perform “very well”.

I would argue that this reveals not that search engine optimization and marketing efforts do not provide a valuable return, but instead that 4 in 5 search marketing agencies/consultants were not properly planning for, optimizing, measuring, and reporting the results of their clients’ search campaigns accurately.

While often a formidable task (particularly for non-ecommerce-based campaigns), assigning an ROI and showing the inherent value in paid and organic search efforts is pivotal to the effectiveness of the campaign. The idea that 60% of companies are not satisfied with search results, tells me that they are simply measuring the wrong things. Metrics such as ‘impressions’ or ‘clicks’ are all-to-often given a disproportionate amount of attention. And while these can be useful indices of search volume and intent, they do not represent the true value of paid and organic search campaigns.

Agencies and clients alike must move beyond impressions and clicks and instead focus on goal conversions and the potential synergies your campaigns gain from your search efforts.

For example, it has been found that combining paid and organic search efforts can result in significant lifts in brand affinity, brand recall, and purchase intent. However, if these are not being tracked or reported, these indirect benefits of search marketing will never be recognized.

In another example, users exposed to display advertising were 22% more likely to produce a sale than those who were not exposed (even if they did not click on the ad), according to a September 2008 study by Atlas Solutions. Nonetheless, if only the direct performance of the display ads (potentially quite dismal) is reported, it is easy to see how companies could think they are not worth the investment.

Properly establishing conversions, assigning monetary values to those conversions, and accurately measuring the direct and indirect results of paid and organic search engine marketing campaigns must be an inherent aspect of any company’s search marketing analytics and measurement strategy.