Basic Keyword Harvesting Techniques to Boost Traffic

Many content managers, bloggers, and internet marketers struggle with SEO and keyword research.

Part of the issue is that they are not using keyword harvesting techniques that actually reflect keywords that are (a) used in the wild, and (b) valuable. The tendency is to brainstorm keywords and then look for those that seem to have high traffic.

This really isn't the best approach.

The best approach to keyword research uses both generating and scraping together. You can harvest keywords from any page, using one of several tools.

Keyword Harvesting

The idea behind harvesting keywords is really very easy: you want to find a way to use content sown by yourself or others into the world wide web to generate root keywords that are both relevant and valuable.

There are two basic parts to the process:

• Scrape: use a tool to extract keywords from a target web page;
• Generate: use the root keywords to generate long tail phrases.

The scraping part should use either:

• Other people's content that comes up in a search for the keyword phrase most closely associated with your niche product or service;
• Your own content (see the end of this article for an example) created specifically to allow you to harvest phrases related to your niche product or service.

One of the best reasons for using keyword harvesting is that it can be done for free!

Keyword Scraper

There are several tools available to help you scrape keywords from content. The best non-free option is probably something like SEMRush, but there is a Free Generator Tool list here.

However, if you want a tool that is easy to use, complete, free and accurate, then use the AdWords Keyword Planner. The drawback is that you need to do a fair amount of post-processing, but just remember that most of the other tools use the API to this service, so you are essentially using the same information.

(For a lesson on relying too much on third party tools, read "What if all your Keyword Research Tools Disappeared Overnight?".)

The process is easy: just use the "Search for new keywords option", and plug in the URL of the page you want to scrape from in the "Your landing page" box.

Make sure you initially set "Show broadly related ideas" to "No", and set the geographic targeting and language according to your target market. The result is a list that you can export, and then sort according to value. The notion of "value" is a tricky one: for me it's usually a simple multiplication of the volume and expected bid price.

If you want to roll your own selection process, then read "Keyword Selection Strategies for Your Keyword Research and SEO Projects".

The resulting keyword list can then be used to attract visitors to your sales page, site, or blog.

Keyword Generator

Based on content produced by yourself or others, the above process yields a list of root keywords. These can be used as-is but beyond root keywords, many people like to explore content created around long tail keyword phrases.

(For a very thorough exploration of the Long Tail, I suggest reading Chris Anderson's Longer Long Tail book.)

Essentially, long tail phrases combine several words together to represent a narrow niche, and allow you to benefit from the root keyword visibility, combined with the qualifiers that make the phrase unique and targeted.

The combination is usually a much better placement in the SERPs than the root by itself, and by judiciously planning a content web around several related long tail keyword phrases, you can usually corner a bigger segment of the market than if you tried to compete head on with everyone else.

To do this, I tend to use a scraper like KeywordTool.io which scrapes auto-suggest queries from Google, YouTube, Bing, and Amazon. Starting with the root phrase, make sure you set the correct geo-targeting and linguistic attributes, and then let the tool do the rest.

Once you have the list, you can export it and then import it into the Google AdWords Keyword Tool, and analyse it in the usual way.

Conclusion: Step by Step Harvesting Process

So, here's a process that will yield a steady stream of keywords for you to harvest, but be warned: it actually takes some effort.

1. Set up a page that will contain an ever-expanding glossary of terms related to your product or service (could also be an FAQ), for example: Robotic Lawn Mower Brands and Models Guide;
2. Allow page/blog comments, but set them to be moderated (encourage feedback; valuable source of keywords);
3. Each day/week/month (pick the frequency that works for you), use the Google AdWords Keyword Planner as a keyword scraper to harvest the best (i.e. most valuable) keywords for your niche;
4. Add any new ones on a Broad Match that are relevant to your master keyword list;
5. Use a generator like KeywordTools.io to generate long tail keyword lists to work into blog content, and feed them back into the system as glossary/FAQ items, or make another page to sow new ideas and harvest the most current keywords from.

Regularly applying the above keyword harvesting process will serve as a way to take the pulse of your target market, as well as provide a rich stream of long tail keyword phrases with which to populate your content.

Really useful inspiration!

Defining a Basic Keyword Research Process

A basic keyword research process must start with an understanding of:

• What the goals are;
• Where the keywords are to be used;
• How a successful campaign will be measured.

If you can answer thee questions, then you stand a chance of being able to develop your own repeatable keyword research process. It is important that the process is both repeatable and measurable, as well as being easy to perform.

Unless you are outsourcing your keyword research tasks, you should aim to keep the process simple, and constrained. It is easy to waste a lot of effort trying to uncover the next big thing in your niche, but if you keep the focus narrow, you stand a greater chance of success.

What Are Your Keyword Research Goals?

The first thing to ascertain is what you want to use the keyword research for, as this will influence the tools and processes that you use. For example, you could want to get more traffic, in which case your evaluation of keywords will be skewed towards Search Volume.

On the other hand, if you want to increase conversion rates (get more sales), then a slightly more sophisticated approach needs to be taken, where you identify trigger words as well as identifying phrases associated with higher Cost Per Click figures.

A higher CPC usually means that there is money in the niche available for advertising, but the other side of the coin is that you might be looking for ways to reduce advertising expenses. Therefore, looking for High Volume, Low CPC keywords will also play a part in the research process.

Where Are The Keywords to be Used?

Having decided what the outcome should be, you then need to consider where the keyword phrases will be used. Clearly the two questions are linked, most obviously through the use of keyword phrases in a PPC campaign.

This will lead to different selections than, for example, keywords that are to be used for on-page SEO (search engine optimisation).

And, off-page SEO (inbound marketing) use will also influence the choice of keyword phrases, especially in relation to trigger words, and words which enhance brand image.

How Will You Measure Success?

Finally, in order to know if the keyword research process has worked, you need to be able to measure the impact of using the keywords. Common measures include:

• Engagement - how much traffic, how many repeat visitors, ratio of repeat visitors to sales, etc.
• Stickiness - when visitors land on the site, how long do they spend engaging with it?
• Hits - the raw number of visits / per page.
• Conversions - how many visitors become customers?

There are other important measures, but if you at least concentrate on one or two of the above in the first instance, you will begin to see the benefits of the keyword research process. It is an investment in your underlying business, rather than the bolt-on that many people seem to see it as.

However, to enjoy those benefits, a proper process must be followed.

Define an Appropriate Keyword Research Process

The process itself will have four phases:

• Research - defining the root keyword phrases;
• Expand - taking each root keyword phrase and finding long tail keyword phrases;
• Analyse - checking the past performance of each phrase;
• Deploy - put the keywords to use, and check the results!

To help in the Research phase, I've listed my favourite free keyword generator tools, with a brief how-to for each. In order to help you Expand into a complete basket of keywords, the long tail keyword generator process explained on the blog is an excellent starting point.

The techniques from the Research ad Expand phases also contain elements that enable you to Analyse keywords: for example, extensive use is made of tools such as the Keyword Planner (AdWords) to help determine search volume and anticipated cost per click.

Finally, the Deploy phase is all about using the keywords and seeing what effect they have. It doesn't need to be complex, but it is important that you stick with a defined process in order to generate repeatable results.

My advice is to start with The Keyword Coach PDCA / Smart goal setting process tailored to keyword research, and extend it according to your own needs. The best way to get started is by actually running a small project and seeing where your niche and business will benefit most from an integrated basic keyword research process.

What is Keyword Research: A Definition for Modern SEO

If you type "what is keyword research" into a search engine, you get a flood of results. Here are a few quotes:

"Keyword research is a practice search engine optimization (SEO) professionals use to find and research actual search terms that people enter into search engines." Wikipedia page on Keyword Research
"Keyword research is one of the most important, valuable, and high return activities in the search marketing field" from the MOZ Guide to SEO

"keyword research tools can help you find the right keywords to optimize a website for search engine users." from Wordstream.com.

All of these are good quotes, but they only really scratch the surface. Keyword  research is much more than just finding the right keywords...

Keyword Research is Market Research

Your future customers use keywords to communicate with the world: when they go to a search engine and type in a query, they are looking for a solution to a problem. They're looking for something that may or may not already exist; if it exists you can sell it to them, if it doesn't, you can create it.

The combination of words is also important, because each word can carry both meaning and intent.

For example, consider the difference between the following:

• "free eBooks about online investing"
• "where to buy books about online investing"

The sales pitch for an online investing course will be very different, depending on which of those phrases delivered the visitor. Learning how to gauge intent is a valuable part of the keyword research process.

The proportion of "buyer intent" keywords versus "buyer research" ones will help to give you an idea about the size of the market as well as its needs.

Keyword Research is Product Research

From the above example, we can also surmise that people want both eBooks and real books. There are people willing to pay, and those who want to sample a free eBook first.

However, there will be a number of phrases that don't come loaded with any intent. These so-called "buyer research" phrases provide valuable insights into what products exist, and what skews might be considered.

By combining brand and product names with your keyword phrase, you can begin to work out the demand for various product skews, as well as look at those already on the market. Once you find a product that satisfies the market, there are also techniques that you can use to extract the keywords from the page to see what the maker is using to attract customers, as well as find out about the competition.

Keyword Research is Competition Research

Using a tool such as SEMRush can help with both the Market and Product Research, but is really good at finding out where the competition is, and what they are using to create opportunities.

Again, intent can be inferred from some of the keyword research, as words like "review", "problems" and "good" (or, indeed "bad") can help to create a picture of how the competition is viewed, and how it might be succeeding (or failing) to satisfy the market.

Keyword Research is also for SEO

Finally, the purpose of keyword research for many people is not to find a new market, or design new products, but to attract people willing to spend money.

The keywords you use in your web content are just another form of communication. You are communicating what your page is about to the search engines, as well as proving its value.

Picking the right keywords helps search engines match the intent of search users with your intent as a content producer. Get it wrong, and your site may never be viewed by a single potential customer; get it right and you may well tap into a market larger than you thought possible.

Learn more about keyword research in the Niche Blogger Content Blueprint; a guide not just for bloggers but anyone who wants to create an online presence.

A Simple Free Long Tail Keyword Generator Process

There are a number of keyword generator services available on the web these days -- from KeywordTool.io to AnswerTheAudience.com -- but all that these do is produce what SearchEngineLand's Nate Dame recently called "long lists of words and phrases ... that can dramatically increase nothing about an SEO strategy." (Source: Why You Should Never Do Keyword Research Again).

Using a keyword analysis tool like Keyword Planner helps, but even it cannot weed out the irrelevant as it is only really designed to tell you what you already suspect: some phrases have more volume than others.

In fact, keyword definition strategy needs to take account of many factors, including working out what keywords people actually use, and then evaluating them from there. Traditional methods like brainstorming just don't cut it any more, and generic one and two word phrases are too competitive to even think about targeting.

However, follow this five step keyword generator process, and you will at least start out on a reasonably solid base!

Step 1: Determine Your Root Keyword Phrase

This step is quite important, but needs to be fairly instinctive. The root keyword phrase has to sum up something that is a bit more narrow than just a niche word (diet, health, money, etc.) and yet not too narrow that you start to preclude possibly useful variations.

Step 2: Extract Search Suggestions

Having decided what it is you want to know about, you can then visit Google and type in your root keyword phrase. Put your blinkers on, and ignore all the results, and scroll down to the bottom of the page, where you'll find a list of suggestions.

Harvest these for the next step (highlight, hit Ctrl-C, open up Notepad, hit Ctrl-V, and you're done -- Mac, Chrome and Linux users will have to use their own equivalents!)

Step 3: Use a Keyword Analysis Tool

Open up the Google AdWords Keyword Planner tool, and select the "Get ideas" option. Copy and paste your list of suggested keyword phrases into the box, and set up your geographic, language, timescale, and most importantly, set the match to "Narrow" (and not "Broad").

Step 4: Select By Volume or Value

In the resulting list, click a column heading to either order the results by "Avg. Monthly Searches" or "Suggested Bid". More advanced users can download the results, and use a formula to created a weighted version of these numbers in order to add a little finesse to the filtering process.

Step 5: Maintain Your Keyword List!

Take the top 10 results, which ought to be fairly long tail (i.e. three to four words long) and specific, and use these either as-is, or as root keyword phrases. My own preference is to first use them, note them in a spreadsheet, and generate new phrases (which also go into the spreadsheet).

Once I have a list to maintain, I can see what keyword phrases attract traffic, and which ones don't. Then, it's back through the process from time to time to generate some more.

Refinements include: seasonality tests, analysis of landing pages (to check the right keywords are being picked up) and near constant observation of behaviour using Analytics and Search Console. But those processes are for another day.

(And yes, before you ask, this article was entirely created around its own process; no prizes for picking out the long tail keyword phrases that were generated along the way!)

Free Keyword Tool / Generator List

Top 5 Free Keyword Tool / Generator List

Anyone who is conducting keyword research on a regular basis needs to have access to tools that save time. Those who follow me will know that I have a specific view of tools -- they should only automate something you understand fully how to to manually -- and my reasons why.

(Hint: it's to do with tools being put beyond use.)

The following tools, by and large, are just time-savers. You can replicate their core functionality given a browser and some time, but they will make your life a lot easier!

KeywordTool.io

This was among the first general-purpose auto-suggestion scrapers. It has retained its simplicity, and in the paid version, has added some search volume and CPC stats. Since these are available through other channels for free, I'm not sure how much value access to these numbers adds, but it certainly streamlines the keyword research process further.

KeywordTool.io also works across sites -- Google, Yahoo, Amazon, etc. -- and has a useful 'question seeker' mode which often uncovers valuable problems, needs, wants and desires in niches.

Ubersuggest.io

Somehow, I've always found Ubersuggest's interface to be somewhat cumbersome, but you can't fault the results. 

It does the same thing that KeywordTool.io does, but returns more results and lets you see them as text, alphabetically, or as a word cloud. Whether these are useful to you will depend on your point of view.

The ability to expand keyword phrases from within the interface, as well as explore each one on Google, Trends, and so forth also add to the power (but also make it a bit distracting). Nonetheless, it stays on my bookmark list because for the few times I've wanted to fool around with a phrase for myself or a client, it has worked wonders!

(Just remember to stay focused, or you can get easily drawn into spending hours mixing up phrases!)

SEO Book Keyword List Generator

SEO Book's free keyword phrase generator is a bit daunting at first. But, ultimately, it is rewarding once you get what it's trying to achieve.

Firstly, it only works if you know what keywords you want to combine.

Given up to five lists of words, it will then combine them in each and every way, allowing you to then input the list into other tools to measure the anticipated effectiveness of the keywords.

It's one of those tools that you need to play around with before deciding whether it's right for you, but it's likely to find a place on your bookmark list because it's a quick and effective brainstorming tool.

SEMRush.com

SemRush works in one of two modes: you can give it an URL and it will extract the keywords and measure the SEO effectiveness of the site, but equally, you can give it a keyword phrase for a report on what the keyword landscape looks like for that phrase.

It can make for very interesting reading: from the number of searches, available results, and anticipated CPC figures through to lists of related keywords and phrase matches. Of course, the free version limits the quantity of these keywords returned, but it is great for a first look at a niche.

Personally, I recommend it as a tool that can be of great use early on, before a site is created and optimised, and then later on to track the performance of a site in search engine marketing terms. Agencies and keyword research professionals will probably find themselves able to justify buying a license for the expanded results, too.

Individuals, not so much, as a lot of mileage can be got out of the free version as long as you only have one project to evaluate.

Google Keyword Planner

There's always one that breaks the rules, and this is it.

The Keyword Planner does something that cannot be done elsewhere. It matches advertising needs with Google's search database. The result is that you can use it to generate alternative keyword phrases and get search volume and PPC stats by keyword phrase.

Not to mention, that if you have a collection of keywords generated by, say KeywordTool.io or the SEO Book generator, you can test them using the Keyword Planner.

The Google Keyword Planner is available, for free, to those with AdWords accounts (not necessarily  funded accounts at present.)

(Note: the reason it breaks the rules is that, if you base your entire keyword research philosophy on it's output, then if it is ever taken away, you will not be able to replicate your own results manually.)

Which is the Best Free Keyword Tool / Generator Combination?

Of the five three free keyword generators listed here, my preference is for simplicity; and that's why I regularly turn to KeywordTool.io. However, if I'm working with a client who has a specific handle on their niche and keywords, then the SEO Book generator enables a list to be created based on combinations of sets of keywords.

It's a great time-saver if you already have the words, just want to know the best way to put them together!

Of the two evaluation tools, I personally use the Keyword Planner a lot. Probably more than I should, but then I'm also an AdWords and AdSense user.

What's great about SEM Rush is that it gives a great dashboard-style overview which can be replicated manually, and with more detail, using a combination of third party tools -- including the Keyword planner -- but is both prettier and more practical.

If you're a beginner, I'd use KeywordTool.io in conjunction with the Keyword Planner. But, the decision is yours!