Showing posts with label outlier analysis. Show all posts
Showing posts with label outlier analysis. Show all posts

Why Are Long Tail Keywords Important for SEO? A Practical Example from the Trenches of Keyword Research

One of the most important aspects of modern keyword research that I teach in Niche Blogger Content Blueprint is the use of long tail keyword phrases to both attract traffic (SEO) and determine market trends.

The hands-down best description of the long tail philosophy can be found in the Harvard Business Review article "Should You Invest in the Long Tail?", from 2008, written by Anita Elberse. In the article, whose conclusions I don't necessarily agree with, there is an excellent discussion of Chris Anderson's 2006 theories presented in his book "The Long Tail: Why the Future of Business is Selling Less of More".

The thrust of Anderson's argument is that, from a 2006 perspective, the long tail will get both longer and fatter. In other words, there will be a higher volume of long tail keyword phrases, and they will attract more visitors.

Elberse disagrees, stating that her data show that, at least in the DVD and music industry that she looked at, the long tail is getting longer and flatter: "Thus the tail represents a rapidly increasing number of titles that sell very rarely or never."

My take is that while she may be correct (at least in 2008, in one industry) as regards sales, when it comes to SEO, it really doesn't matter if the tail is flatter or fatter.

This is supported by a paper from the 2016 paper "How Relevant is the Long Tail?" which presents research across a million sites conducted by researchers at universities in Hamburg and Cologne, in Germany. Their key conclusion is worded as follows:
"Therefore we argue that the long tail contains valuable information and is a rich source for the diversification of web search engine result lists."
The paper points out that even if the tail is getting longer, the results are still vital to the search engine user, and therefore, by extension, I would argue important for the keyword researcher.

While I don't have a million clients (yet), I can provide one example from a model rail site in the UK, whom I encouraged to examine their own long tail through the lens provided by the combination of StatCounter and Google's Search Console (formerly Webmaster Tools).

What Are Long Tail Keywords?

Essentially, long tail keywords are just the phrases that attract fewer clicks but which represent much more precise search queries and hence a higher chance of action than shorter, more vague keyword phrases.

Deciding, for your niche, what represents a long tail phrase is often part of the challenge; and one that requires intimate market knowledge. Consider the following screen shot, which represents the first few hours of a specific day:


Looking through the list, several things are apparent. The first is that a lot of the phrases (ordered by Search Engine Impressions) are long-tail in nature. For example, "hornby trakmat" while a short phrase, is effectively long tail due to it's incredibly niche nature.

What we can also say is that the site does a better than average job (on this evidence alone) of attracting visitors interested in "model railway buildings free download", a worse than average job regarding "hornby trakmat" and is relatively poor at converting impressions into clicks across the board.

Questions that need to be answered include why 67% of visitors get converted for "model railway buildings free download" while 0% are converted for the phrase "scratch built model railway buildings" which are ostensibly covering similar topics.

Finally, a quick bit of maths leads us to conclude, that for the restricted time frame that this snapshot refers to, the site is missing out on an additional 30% of available traffic, following a simple calculation that I encourage all site owners to do:
  • Sum all relevant Impressions with a Pos < 11 (1st page)
  • Calculate the average CTR
  • Subtract current traffic generated (Clicks)
In this case, the calculation is:
  • 78 Impressions for model rail related queries on the 1st page...
  • From two results, a 40% CTR is the average...
  • We already see 4 Clicks...
  • ... so the addressable traffic is around 27 hits (78 * 40% - 4)
According to the application of my favourite PDCA (Plan Do Check Act) process improvement theory, we now have all we need to take action!

How to Use Long Tail Keywords in SEO?

We want more traffic. That's what SEO is all about. Forget for a moment that I usually preach getting the right traffic (as readers of "Niche Blogger Content Blueprint" will know) and concentrate on what you would do to just get more traffic.

My first piece of advice, in this case, was to tackle the top of the list: "scratch built model railway buildings".

The site had no page for this, so I had them put one up, linked to an already working long tail keyword phrase for which content was available: "model railway buildings free download".

To decide on what content should go into the page, I used keyword research to uncover, rank, and target the top questions asked on Google relating to the primary keyword phrases and, importantly, sub-phrases.

Those were: "scratch built", "model railway buildings" and, of course, "scratch built model railway buildings".

Finally, with content plans in hand, it was back to the client for the actual word-work, and then back to me for SEO tweaking.

Only time will tell if the result is a significant increase in traffic, as it will take a moment for the pages to get picked up by the search engines, but the chances are good that if the Pos improves, then the CTR will follow.

If you want a sneak peek into the methods used to evaluate the long tail phrases used to research this piece, then sign up for the free eBook "The Outlier Method"!

Friday, 19 August 2016

What if All Your Keyword Research Tools Disappeared Overnight?

How To Avoid This Often Terminal SEO Mistake

Recently I had a client whose favourite -- indeed only -- keyword research tool had been put beyond use.

Behind the initial panic was the growing realisation that they didn’t actually know how the tool did what it did. It was selecting excellent keywords, and had served her well in the past, but she had no idea how it worked under the hood.

To be clear on this point: “put beyond use” can mean any one of the following:

  • A temporary glitch;
  • The provider went out of business;
  • The fees went up beyond your capacity to pay them;
  • The source of the data that the provider was using was put beyond use.

The first is easy: it’s a waiting game. Just make sure that you are always a few days (or weeks) ahead of the data point. In other words, don’t leave it until the last minute to have a basket of keywords to work with: use seasonal and outlier analysis to keep ahead of the game!

The rest of the items in the above list are catastrophic if you have no idea how to replicate the service manually.

So, here’s the first lesson: don’t use metrics provided by third parties that you can’t replicate manually. Any services you use should merely be time-saving conveniences; if they have special sauce, one day you’ll be left with only a dry bowl of pasta.

You’ll still eat, but it won’t be pleasant...

Make Sure You Have An SEO Plan B


Even if the absolute worst case scenario is manual calculation using the most basic tools, it’s better than nothing. With the right preparation, it can be outsourced, releasing you to perform business functions that add value to your proposition.

For every SEO action -- from keyword research to social posting and link building -- make sure you have an alternative to the time-saving tools you use on a daily basis. There are some that you can’t do without (Google Search, AdWords, and the actual social networks themselves, for example) but make sure that these are:

  • In the minority;
  • Unlikely to be put beyond use.

In the keyword research field, we often deploy third party tools that gather up data and then present it in a way that lets us draw conclusions. For example, the AdWords Keyword Planner, or SEMRush analysis tools pull together a number of data points -- some proprietary, some not -- as a way to save time.

Decision tools such as Market Samurai, and automation tools like KeywordTool.io and Power Suggest Pro*, also fall into the category of must-haves for people embarking on serious strategic keyword management projects.

As long as these are used as labour saving devices, and not crutches, that’s okay. In fact, although I used -- and have paid -- for some of the above, I never use them for anything except baseline research: just the numbers.

For decision-making, I prefer to rely on my own metrics, which sometimes includes charting the data and just using my eyes. However, I do have three so-called ‘first cut’ metrics that I’ve rolled together to help make sure I weed out the non-starters.

These are automated, but could be done manually, if my toys get taken away from me!

Create Your Own Keyword Research (and SEO) Metrics


Probably the most important metrics are those I call ‘first cut’. The idea is to cut away a first set of keywords that aren’t likely to be useful. The metric that you use tends to be one of the following:

  • Pure volume - search volume vs. results returned;
  • Market value - amount of free advertising capital in the niche;
  • PPC Campaign - reducing the cost of a PPC campaign for maximum exposure.

Obviously, to calculate these, you will need tools. For example, my own go-to metric for a pure volume project is to emphasis the addressable market over the sheer number of searches: sometimes referred to as the KEI.

To calculate the KEI, you will at least need two numbers:

  • Search Volume
  • Number of Results

The second of these is easy enough to scrape from a Google Search (either manually, or with a simple script) but the Search Volume is only really available via the AdWords Keyword Planner.

The same is true for the market value metric, where the anticipated cost per click is used to test a keyword for available advertising spend against search performance, and picking outliers for inclusion.

Again, Keyword Planner is the only real source; luckily it is unlikely to be put beyond use for the foreseeable future, as it supports most of the Google keyword economy.

That’s good, because the last first cut metric, specifically used to remove expensive keywords, is almost the opposite of the market value metric, and specifically uses data from Keyword Planner.

Tools can help apply all of these metrics: for example, Market Samurai can be set up to filter for commerciality, or cheap keywords, and remove everything else. However, if you don’t know how to do the work manually, and those tools are taken away, you’ll be unable to replicate the results and you’ll have to start from scratch.

Learning how to calculate first cut metrics isn’t a good thing to have to do in a hurry. The best advice is to start now, and roll your own set of metrics: not only will you have a safety net, but you'll also learn a lot in the process. I guarantee it!

-----------------------------------------------------------------------------
* Disclosure: I participate in the Power Suggest Pro Affiliate Scheme, and the article link is an affiliate link. If you prefer not to click it, use www.powersuggestpro.com instead.

Saturday, 30 July 2016

Stop Swimming With The Sharks and Bask In The Blue Ocean of Keyword Research

In strategic management, one of the concepts we come across quite regularly is the difference between the Red and Blue Oceans. The Red Ocean is the place that we envisage most competition to take place, where conditions are tough:

  • the marketplace, and rules of the game are well known;
  • there is high competition for demand;
  • the trade-off is low margins;
  • differentiation is through price. 

(source: Blue Ocean Strategy web site)

The Blue Ocean, on the other hand, is the polar opposite of all of the above - as the authors of the Blue Ocean Strategy book (W. Chan Kim & Renee Mauborgne) put it: the aim is to "Create uncontested market space" in which to do business.

Most people in keyword research and SEO are swimming with the sharks in the Red Ocean.

It stands to reason: they're using the same techniques, the same data, and leveraging the same tools. And there's nothing intrinsically wrong with any of that, except that two people in the same niche, doing pretty much the same analysis are going to naturally put themselves in direct competition.

So, the first message is an easy one: develop your own unique way of deriving value from the data.

This is the point of an area of study known as Data Science. I recently read an excellent free eBook on the topic by Jerry Overton (a professional Data Scientist) -- Going Pro in Data Science -- in which the author explained how the key to success in data science was in picking signals out of the noise.

Looking for patterns, in other words.

Traditional SEO practitioners spend a lot of time telling is we need to look for patterns in order to chase the market. It makes sense on the face of it -- go to where the market is, and satisfy its needs -- but there's a slight problem.

Online marketing is getting so saturated that the low hanging fruit in your niche may well have already been picked. Or worse still, it's become a red ocean of marketers all competing for the same space.

Retailers suffer a similar consequence; they have to compete with each other, and online retailers. Even those in retail service -- restaurants, beauty salons and so forth -- will find that they need to adopt their own data science strategies to pick their market's minds, and develop strategies to exploit their competitive advantage.

In fact, strategic management also tells us that there are resources that contribute to our competitive advantage that must be valuable, rare, difficult to imitate and organised in such a way as to make exploitation easy (VRIO analysis).

Keyword research falls into such a category, but only if you develop your own data science to reveal the answers that you are looking for. After all, there's nothing rare or difficult to imitate about looking simply for the keyword phrase with the highest CTR, or the most search volume, or even with the least competition for space in the search engines indexes.

Everyone is looking for those keywords, so even if the space is uncontested now, it's unlikely to remain that way for long!

Luckily, most modern tools allow you to download your research data, so that different data points can be combined in a spreadsheet, and algorithms developed that let you organise, sort, and prioritise keywords according to your needs.

For example, you might download data from Google's Search Console (such as CTR and SERP data per query), and merge that in a spreadsheet with search volume data from the Keyword Planner, to enable you to perform analysis to reveal likely candidates for a new campaign centred around a keyword with certain distinct properties.

Consider the following chart, for example.
It shows something I call the 'outlier technique', where we look not for a pattern, but a signal from the noise of an existing pattern. In this case, the pattern is represented by a cluster of keyword phrases that fall into a clearly defined area, and a selection of keyword phrases that don't.

These outliers show phrases for which something out of the ordinary is happening.

The graph maps the SERP for each phrase against the CTR. The phrases in the bottom left, then, are those where competition is pretty stiff. This data is taken from the Search Console account for a client blog, so one of the first messages is that there are many instances where, despite appearing high in the SERPs, the CTR is low - not many people are choosing to click on the links, despite the entry appearing (on average) within the first three pages.

There are many reasons for this, but in some cases its that the page ranks for a keyword which is not represented by the content (heading & summary) to a sufficient extent that the searcher is convinced that a visit is worth their while.

At the other end of the spectrum, top-right of the graph, we have a phrase which, although it appears way down the SERPs, has a very high CTR.

Either the content is very good, or the market is very hungry. Both of these possibilities are worth exploring further: the next graph is a natural extension, and combines Keyword Planner data with the Search Console data, for the same set of keywords.

With this iteration, we have more signals. This time, we have captured the available traffic, a simple multiplication of the volume by our CTR. It's a simple metric, but already we see that there are three phrases that show important characteristics that make them worthy of engagement for future products, publications or traffic attracting content.

So, create your own metrics, and chase your own set of outliers, and you will be well on your way to making keyword research part of your own competitive advantage.