Keyword Research Competitor Analysis Tips and Tricks

In this article we will look at a few tips and tricks related to keyword research competitor analysis.

I'm going to assume that your target page isn't as high up in the SERPs as you would like, and that you have done some basic strategic keyword research to figure out which keyword phrases you should be targeting (depending on whether you have picked a Value or Volume keyword selection strategy.)

The reason the selection strategy matters is because the first technique is entirely keyword driven.

Search Competitor Analysis

Before anything else, you need to know who is in the Top 10 for your target keyword phrase.

The easiest way to do this is to go to your search engine of choice, and type in your search phrase as a query. Here are some important key points:

• make sure you conduct the search in an "anonymous" browser session (also called InPrivate);
• set the right geographic context by using the correct location-based engine URL (i.e. .co.uk/.com/.fr etc.);
• start with a narrow match (i.e. put the keywords "in quotes").

Once you have your list of direct search engine result page competitors, open each target link in a new window, and copy and paste the URL to a notepad or spreadsheet. Do as many as you see fit, across a wide spectrum of URLs (if a lot of them come from so-called "content farms" be wary), but make sure you have at least 10 unique domains / sub-domains.

Keyword Density

Once you've figured out who the competition is, use a tool like SEO Tools Keyword Density Checker to make a list of the one, two and three word keyword phrases, and rank them according to their relative density.

Why does this matter?

Keyword density is one of those measures that used to have recommended hard and fast rules about what the number should be. In the early days, keyword stuffing, a technique that consisted of repeating the target keyword phrase as often as possible in an article got results.

These days, however, search engines "recognise keyword stuffing as a disingenuous tactic" at best (source: Enge et al (2010) The Art of SEO, US:O'Reilly, p. 211) and "can actually get your pages devalued via search engine penalties" at worst (ibid).

Using the tool is easy: just plug in the page that you have identified as being in competition with your own, and then pick out the keywords and their relative densities from the resulting list. Now you know two things:

• the best keywords;
• the densities that are currently working.

Of course, the more pages you analyse, the better your keyword density research will be, and since the results are always relative to your own performance, you should also conduct the same exercise on one of your own pages.

Strategic Keyword Competitor Analysis

Another tactic you can use to expose keyword phrases that your competitors are targeting is to copy and paste the URL into the AdWords Keyword Planner tool.

In the same way that the SEO Tools utility above picks out keyword phrases, the Keyword Planner will analyse the content of the page, and pick up keywords from the Google and AdWords' keyword databases.

The result is a list that can be used to create content to rival that which is being put out by the competition, lifted from their own text. Incidentally, the technique can also be used as a cross check for your own content (to make sure Google is picking up the right keyword phrases) or s a way to pick out keywords to target with an AdWords campaign.

The Keyword Coach Keyword Research Tutorials are a good place to start if you need some tips on how to analyse the resulting lists of keyword and search data, or just search the blog for "long tail keywords".

Processing lists of keywords to find those that match your strategy is a useful skill to learn, and the Keyword Planner results are a great (free) data source to start with.

Finally, the SEMRush tool is great for giving you an overview of your competition, and the keywords that they are targeting. What's great is that it works well for sub-domains as well as domains. However the tool performs but less well for pages on a domain.

Maintaining a list of keywords that are preferred by the competition is a good way to add keyword research and keyword marketing to your ongoing competitive advantage, just assign some time in your keyword research strategy and process to apply these techniques and stay ahead of the pack!

Keyword Research for Email Marketing

Many businesses find that email marketing remains one of their channels with the highest ROI.

This is mainly due to the fact that they have responsive email lists, built up from customers who have already put their hands up in one of the following ways:

• downloaded and taken action upon a free report;
• purchased a relatively cheap item;
• engaged with the business.

The most responsive of the above are, clearly, existing customers. Indeed, many marketers go one step further and claim that customers' appetite for products will usually exceed your capacity to create new ones.

Subsequently, considering building a list is free, and as long as the list is correctly segmented and the appropriate message sent to each one, the conversion rates can lead to very impressive results.

What is Email Marketing?

Email marketing is often merely described as an email message sent to a prospect with commercial intent.

But that's like saying that marketing is equal to display advertising, which is patently untrue. To get an idea of just how untrue that is, I usually refer clients to Figure 1.1 in "The Marketing Book" by Michael Baker and Susan Hart which depicts the whole of marketing as an iceberg.

Advertising is at the top -- it's the bits you see -- whereas the 90% that you don't see, and which is often neglected, is the true nature of marketing, whose purpose is not only to reach a market and convince them to buy but also to understand that market, related markets, the customer's needs and products that can be presented to them.

So, if you're using email marketing just to send a coupon out (Advertising, Promotion & Public Relations in The Marketing Iceberg) then you're doing it wrong.

Email marketing should also be viewed variously as:

• a tool for Market Testing;
• a platform for Product Development;
• a way to conduct Market & Customer Needs Research.

Each type of list (customer, prospect, etc.) has its use: there is no point, for example, trying to elicit feedback about a commercial product that you know the list recipients could not have bought because they are on a prospects and not a buyers list.

So, where does keyword research come into the equation?

Using Keyword Research for Email Marketing

The first thing to remember is that different list segments will react to different messages.

A buyer probably doesn't need a lot of encouragement to buy again; they just need you to offer them a credible product that is related to something they have already bought. And if you're doing your strategic keyword management properly, you'll already know where their pain/passion points are.

Prospects, on the other hand, need to make that first decision to buy with you: and that's usually a decision that they make with their gut and then justify with logic.

It needs to feel right: so the emotive trigger words come into play.

Not only do you need to participate in a conversation that they are already having outside of your relationship (again, keyword research will throw up these conversations) but you need to push their buttons to drive them to take action.

Of course, you can also use email marketing to elicit reaction (market research), test new product ideas (free downloads in return for market research) and develop a relationship with new markets (referral and affiliate marketing).

However, each interaction needs to come with a healthy dollop of keyword research, as this is the only real way you can be sure that you are communicating with the target market.

Benefits of Keyword Research for Email Marketing

That communication underpins the key benefit: contact.

Every email is contact with the target market, existing market, or to use a more current term, your tribe.

Each time you make contact with your tribe, you build confidence. As confidence in you and your leadership of the tribe grows, so does the trust between you, and your influence within the tribe.

Using keyword research effectively shows that you understand your tribe: you speak to them using words they have either used themselves, or that make perfect sense to them. When combined with well-known trigger words (Richard Bayan's Words that Sell has a good list), your message will become very difficult to refuse.

To get started, check out the Keyword Research Tutorials or take The Keyword Coach for a test drive with one of the low cost keyword research services. Remember: there's gold in those hills, and you can learn how to mine it!

How To Use Keyword Research to Get More Sales

Most (if not all) business owners would like to get more sales: online, or offline, sales are what drives a business forward.

Sometimes it doesn't take many sales -- high ticket items that command a high price point can generate a healthy income with one or two sales a month -- but all businesses rely on having paying customers.

There are three keys to making more sales:

• finding more potential customers;
• converting more potential customers;
• selling more to existing customers.

Keyword research can help each of the above; it's all about knowing where to find your market, how to communicate with them, and then how to keep them interested.

Using Keywords to find Potential Customers

Potential customers (those who are willing to buy) use keywords differently to those who are just looking for information.

For example, they will be using terms like "how much" or "where to buy", as well as geographic trigger words such as city names. All of these indicate a person ready to buy, as opposed to someone merely researching.

Brand and model names are also useful indicators, especially when combined with words such as "versus" or "reviews".

Keyword research helps to isolate phrases that can be used to create content designed to pull in potential customers via the SERPs (search engine result pages) rather than merely appealing to everyone.

There will be fewer hits, but conversion rates should be higher.

Converting Visitors into Customers using Keywords

One of the highest converting complementary keywords is "free".

It is an emotional trigger word, as well as a way to convey the idea that the prospect is getting something for nothing.

However, in some circles it can also devalue your product offering; and there are even people who wouldn't even count a person who has downloaded a free gift as a customer at all.

So, although the word "free" crops up repeatedly in keyword research, it's unlikely to be a great indicator of a potential paying customer, unless you can find a way to convert them down the line.

But, it's an important first step; since even the act of downloading something and reading it requires some effort: it might have been zero cost in monetary terms, but it has cost effort and as such these people are to be considered customers.

Keyword research can also be used to reveal what non-free keywords your potential customers are looking for. Using those keywords in your text ought to help convert them into customers because you are participating in a conversation that they are already having with themselves.

Leverage Your Customer Base with Integrated Keyword Research

Every contact with existing customers -- those who have put their hands up by buying from you or downloading something from your site -- should be geared towards a Call to Action (CTA).

The more that your customers can be encouraged to engage with you, the more they will spend.

Provide a search function on your site; and integrate the keywords searched for with your regular keyword research activity. This way you will leverage their engagement to yield new avenues through which you can provide them even more value.

After all, that's all that you are trying to do with your search engine and keyword marketing: find out what the market needs, how they are trying to locate products to satisfy those needs, and providing them with the best solution on the market.

Anything less will not provide a basis for long term competitive advantage, and making keyword research part of that competitive advantage should be one of a business' strategic plan. 

For a more detailed look at how keyword research can help to get more sales, check out how answering these three questions that can triple your bottom line!

Defining a Basic Keyword Research Process

A basic keyword research process must start with an understanding of:

• What the goals are;
• Where the keywords are to be used;
• How a successful campaign will be measured.

If you can answer thee questions, then you stand a chance of being able to develop your own repeatable keyword research process. It is important that the process is both repeatable and measurable, as well as being easy to perform.

Unless you are outsourcing your keyword research tasks, you should aim to keep the process simple, and constrained. It is easy to waste a lot of effort trying to uncover the next big thing in your niche, but if you keep the focus narrow, you stand a greater chance of success.

What Are Your Keyword Research Goals?

The first thing to ascertain is what you want to use the keyword research for, as this will influence the tools and processes that you use. For example, you could want to get more traffic, in which case your evaluation of keywords will be skewed towards Search Volume.

On the other hand, if you want to increase conversion rates (get more sales), then a slightly more sophisticated approach needs to be taken, where you identify trigger words as well as identifying phrases associated with higher Cost Per Click figures.

A higher CPC usually means that there is money in the niche available for advertising, but the other side of the coin is that you might be looking for ways to reduce advertising expenses. Therefore, looking for High Volume, Low CPC keywords will also play a part in the research process.

Where Are The Keywords to be Used?

Having decided what the outcome should be, you then need to consider where the keyword phrases will be used. Clearly the two questions are linked, most obviously through the use of keyword phrases in a PPC campaign.

This will lead to different selections than, for example, keywords that are to be used for on-page SEO (search engine optimisation).

And, off-page SEO (inbound marketing) use will also influence the choice of keyword phrases, especially in relation to trigger words, and words which enhance brand image.

How Will You Measure Success?

Finally, in order to know if the keyword research process has worked, you need to be able to measure the impact of using the keywords. Common measures include:

• Engagement - how much traffic, how many repeat visitors, ratio of repeat visitors to sales, etc.
• Stickiness - when visitors land on the site, how long do they spend engaging with it?
• Hits - the raw number of visits / per page.
• Conversions - how many visitors become customers?

There are other important measures, but if you at least concentrate on one or two of the above in the first instance, you will begin to see the benefits of the keyword research process. It is an investment in your underlying business, rather than the bolt-on that many people seem to see it as.

However, to enjoy those benefits, a proper process must be followed.

Define an Appropriate Keyword Research Process

The process itself will have four phases:

• Research - defining the root keyword phrases;
• Expand - taking each root keyword phrase and finding long tail keyword phrases;
• Analyse - checking the past performance of each phrase;
• Deploy - put the keywords to use, and check the results!

To help in the Research phase, I've listed my favourite free keyword generator tools, with a brief how-to for each. In order to help you Expand into a complete basket of keywords, the long tail keyword generator process explained on the blog is an excellent starting point.

The techniques from the Research ad Expand phases also contain elements that enable you to Analyse keywords: for example, extensive use is made of tools such as the Keyword Planner (AdWords) to help determine search volume and anticipated cost per click.

Finally, the Deploy phase is all about using the keywords and seeing what effect they have. It doesn't need to be complex, but it is important that you stick with a defined process in order to generate repeatable results.

My advice is to start with The Keyword Coach PDCA / Smart goal setting process tailored to keyword research, and extend it according to your own needs. The best way to get started is by actually running a small project and seeing where your niche and business will benefit most from an integrated basic keyword research process.

What is Keyword Research: A Definition for Modern SEO

If you type "what is keyword research" into a search engine, you get a flood of results. Here are a few quotes:

"Keyword research is a practice search engine optimization (SEO) professionals use to find and research actual search terms that people enter into search engines." Wikipedia page on Keyword Research
"Keyword research is one of the most important, valuable, and high return activities in the search marketing field" from the MOZ Guide to SEO

"keyword research tools can help you find the right keywords to optimize a website for search engine users." from Wordstream.com.

All of these are good quotes, but they only really scratch the surface. Keyword  research is much more than just finding the right keywords...

Keyword Research is Market Research

Your future customers use keywords to communicate with the world: when they go to a search engine and type in a query, they are looking for a solution to a problem. They're looking for something that may or may not already exist; if it exists you can sell it to them, if it doesn't, you can create it.

The combination of words is also important, because each word can carry both meaning and intent.

For example, consider the difference between the following:

• "free eBooks about online investing"
• "where to buy books about online investing"

The sales pitch for an online investing course will be very different, depending on which of those phrases delivered the visitor. Learning how to gauge intent is a valuable part of the keyword research process.

The proportion of "buyer intent" keywords versus "buyer research" ones will help to give you an idea about the size of the market as well as its needs.

Keyword Research is Product Research

From the above example, we can also surmise that people want both eBooks and real books. There are people willing to pay, and those who want to sample a free eBook first.

However, there will be a number of phrases that don't come loaded with any intent. These so-called "buyer research" phrases provide valuable insights into what products exist, and what skews might be considered.

By combining brand and product names with your keyword phrase, you can begin to work out the demand for various product skews, as well as look at those already on the market. Once you find a product that satisfies the market, there are also techniques that you can use to extract the keywords from the page to see what the maker is using to attract customers, as well as find out about the competition.

Keyword Research is Competition Research

Using a tool such as SEMRush can help with both the Market and Product Research, but is really good at finding out where the competition is, and what they are using to create opportunities.

Again, intent can be inferred from some of the keyword research, as words like "review", "problems" and "good" (or, indeed "bad") can help to create a picture of how the competition is viewed, and how it might be succeeding (or failing) to satisfy the market.

Keyword Research is also for SEO

Finally, the purpose of keyword research for many people is not to find a new market, or design new products, but to attract people willing to spend money.

The keywords you use in your web content are just another form of communication. You are communicating what your page is about to the search engines, as well as proving its value.

Picking the right keywords helps search engines match the intent of search users with your intent as a content producer. Get it wrong, and your site may never be viewed by a single potential customer; get it right and you may well tap into a market larger than you thought possible.

Learn more about keyword research in the Niche Blogger Content Blueprint; a guide not just for bloggers but anyone who wants to create an online presence.

A Simple Free Long Tail Keyword Generator Process

There are a number of keyword generator services available on the web these days -- from KeywordTool.io to AnswerTheAudience.com -- but all that these do is produce what SearchEngineLand's Nate Dame recently called "long lists of words and phrases ... that can dramatically increase nothing about an SEO strategy." (Source: Why You Should Never Do Keyword Research Again).

Using a keyword analysis tool like Keyword Planner helps, but even it cannot weed out the irrelevant as it is only really designed to tell you what you already suspect: some phrases have more volume than others.

In fact, keyword definition strategy needs to take account of many factors, including working out what keywords people actually use, and then evaluating them from there. Traditional methods like brainstorming just don't cut it any more, and generic one and two word phrases are too competitive to even think about targeting.

However, follow this five step keyword generator process, and you will at least start out on a reasonably solid base!

Step 1: Determine Your Root Keyword Phrase

This step is quite important, but needs to be fairly instinctive. The root keyword phrase has to sum up something that is a bit more narrow than just a niche word (diet, health, money, etc.) and yet not too narrow that you start to preclude possibly useful variations.

Step 2: Extract Search Suggestions

Having decided what it is you want to know about, you can then visit Google and type in your root keyword phrase. Put your blinkers on, and ignore all the results, and scroll down to the bottom of the page, where you'll find a list of suggestions.

Harvest these for the next step (highlight, hit Ctrl-C, open up Notepad, hit Ctrl-V, and you're done -- Mac, Chrome and Linux users will have to use their own equivalents!)

Step 3: Use a Keyword Analysis Tool

Open up the Google AdWords Keyword Planner tool, and select the "Get ideas" option. Copy and paste your list of suggested keyword phrases into the box, and set up your geographic, language, timescale, and most importantly, set the match to "Narrow" (and not "Broad").

Step 4: Select By Volume or Value

In the resulting list, click a column heading to either order the results by "Avg. Monthly Searches" or "Suggested Bid". More advanced users can download the results, and use a formula to created a weighted version of these numbers in order to add a little finesse to the filtering process.

Step 5: Maintain Your Keyword List!

Take the top 10 results, which ought to be fairly long tail (i.e. three to four words long) and specific, and use these either as-is, or as root keyword phrases. My own preference is to first use them, note them in a spreadsheet, and generate new phrases (which also go into the spreadsheet).

Once I have a list to maintain, I can see what keyword phrases attract traffic, and which ones don't. Then, it's back through the process from time to time to generate some more.

Refinements include: seasonality tests, analysis of landing pages (to check the right keywords are being picked up) and near constant observation of behaviour using Analytics and Search Console. But those processes are for another day.

(And yes, before you ask, this article was entirely created around its own process; no prizes for picking out the long tail keyword phrases that were generated along the way!)

Why Are Long Tail Keywords Important for SEO? A Practical Example from the Trenches of Keyword Research

One of the most important aspects of modern keyword research that I teach in Niche Blogger Content Blueprint is the use of long tail keyword phrases to both attract traffic (SEO) and determine market trends.

The hands-down best description of the long tail philosophy can be found in the Harvard Business Review article "Should You Invest in the Long Tail?", from 2008, written by Anita Elberse. In the article, whose conclusions I don't necessarily agree with, there is an excellent discussion of Chris Anderson's 2006 theories presented in his book "The Long Tail: Why the Future of Business is Selling Less of More".

The thrust of Anderson's argument is that, from a 2006 perspective, the long tail will get both longer and fatter. In other words, there will be a higher volume of long tail keyword phrases, and they will attract more visitors.

Elberse disagrees, stating that her data show that, at least in the DVD and music industry that she looked at, the long tail is getting longer and flatter: "Thus the tail represents a rapidly increasing number of titles that sell very rarely or never."

My take is that while she may be correct (at least in 2008, in one industry) as regards sales, when it comes to SEO, it really doesn't matter if the tail is flatter or fatter.

This is supported by a paper from the 2016 paper "How Relevant is the Long Tail?" which presents research across a million sites conducted by researchers at universities in Hamburg and Cologne, in Germany. Their key conclusion is worded as follows:

"Therefore we argue that the long tail contains valuable information and is a rich source for the diversification of web search engine result lists."

The paper points out that even if the tail is getting longer, the results are still vital to the search engine user, and therefore, by extension, I would argue important for the keyword researcher.

While I don't have a million clients (yet), I can provide one example from a model rail site in the UK, whom I encouraged to examine their own long tail through the lens provided by the combination of StatCounter and Google's Search Console (formerly Webmaster Tools).

What Are Long Tail Keywords?

Essentially, long tail keywords are just the phrases that attract fewer clicks but which represent much more precise search queries and hence a higher chance of action than shorter, more vague keyword phrases.

Deciding, for your niche, what represents a long tail phrase is often part of the challenge; and one that requires intimate market knowledge. Consider the following screen shot, which represents the first few hours of a specific day:

Looking through the list, several things are apparent. The first is that a lot of the phrases (ordered by Search Engine Impressions) are long-tail in nature. For example, "hornby trakmat" while a short phrase, is effectively long tail due to it's incredibly niche nature.

What we can also say is that the site does a better than average job (on this evidence alone) of attracting visitors interested in "model railway buildings free download", a worse than average job regarding "hornby trakmat" and is relatively poor at converting impressions into clicks across the board.

Questions that need to be answered include why 67% of visitors get converted for "model railway buildings free download" while 0% are converted for the phrase "scratch built model railway buildings" which are ostensibly covering similar topics.

Finally, a quick bit of maths leads us to conclude, that for the restricted time frame that this snapshot refers to, the site is missing out on an additional 30% of available traffic, following a simple calculation that I encourage all site owners to do:

• Sum all relevant Impressions with a Pos < 11 (1st page)
• Calculate the average CTR
• Subtract current traffic generated (Clicks)

In this case, the calculation is:

• 78 Impressions for model rail related queries on the 1st page...
• From two results, a 40% CTR is the average...
• We already see 4 Clicks...
• ... so the addressable traffic is around 27 hits (78 * 40% - 4)

According to the application of my favourite PDCA (Plan Do Check Act) process improvement theory, we now have all we need to take action!

How to Use Long Tail Keywords in SEO?

We want more traffic. That's what SEO is all about. Forget for a moment that I usually preach getting the right traffic (as readers of "Niche Blogger Content Blueprint" will know) and concentrate on what you would do to just get more traffic.

My first piece of advice, in this case, was to tackle the top of the list: "scratch built model railway buildings".

The site had no page for this, so I had them put one up, linked to an already working long tail keyword phrase for which content was available: "model railway buildings free download".

To decide on what content should go into the page, I used keyword research to uncover, rank, and target the top questions asked on Google relating to the primary keyword phrases and, importantly, sub-phrases.

Those were: "scratch built", "model railway buildings" and, of course, "scratch built model railway buildings".

Finally, with content plans in hand, it was back to the client for the actual word-work, and then back to me for SEO tweaking.

Only time will tell if the result is a significant increase in traffic, as it will take a moment for the pages to get picked up by the search engines, but the chances are good that if the Pos improves, then the CTR will follow.

If you want a sneak peek into the methods used to evaluate the long tail phrases used to research this piece, then sign up for the free eBook "The Outlier Method"!