Plan Do Check Act, SMART Goals and A/B Split Testing

Split testing is a valuable part of strategic keyword management. For a start, it allows you to throw away those keywords that aren't bringing in the results -- but at the same time, it gives you an opportunity to improve on so many areas of your retail marketing effort that learning it is a transferable skill in itself.

Of course, this article looks at split testing from the point of view of keyword management, but it can be, and has been, applied to all kinds of result-dependent activity.

By result-dependent we mean an activity that has a clearly defined and measurable goal; one which produces results that further our guiding strategy.

In strategic management terms, these are results that keep us in line with our mission and contribute to a sustainable competitive advantage.

At the core is something that I may well have alluded to before: PDCA.

Plan Do Check Act

The PDCA cycle (also known as Deming's Cycle) is something that I first came across in the context of process improvement. For a great overview, try the US Navy's Handbook for Basic Process Improvement, which is part of their Strategic Planning collection of documents.

(It's pretty lengthy, so don't get sidetracked, but there's a lot of useful flowcharts in there that can be easily applied to outsourced projects i.e. manual keyword expansion from a root keyword phrase, and checking of vital keyword effectiveness metrics against baseline measurements, and so on.)

The point of PDCA is that you first make a plan. Then, you carry out a process that has been created to achieve the goal set out in the plan.

Next comes a period of checking -- were the expected results achieved? -- and adjusting the process (the Do) part so that there is a higher chance of achieving the goals set out in the Plan.

In a nutshell, that's the Deming cycle, and it can be applied to a lot of different activities. Such as: A/B split testing of keyword rich headlines that form part of an advert campaign (on AdSense, for example).

However, before we look at A/B split testing (the process, or 'Do' part), we need to make sure that we understand what it is supposed to achieve. For that, I like to use SMART goals.

SMART Goals

There's a great discussion of the various meanings behind the SMART moniker on the Project Smart web site. While you're free to choose a set that make sense to you, here's what I recommend:

• S : specific (i.e. not at all vague)
• M : measurable (i.e. numerically)
• A : achievable (i.e. not unrealistic)
• R : results-oriented (i.e. has to take you to where you want to go)
• T : time-bound (i.e. set a time and/or cost limit)

In this case, your goal might be to increase click-through rates by 1% within 6 months by varying keyword placement in an advert heading.

The goal is set, but what is the process that will deliver the desired result?

A/B Split Testing

The driving principle behind A/B split testing is that you have two test cases, and you want to see which one performs the best. When you find that out, you devise another case and pitch it against the current best performer.

It sounds simple... and it is.

Within our PDCA process, it's the 'Do' part:

• Plan : "increase click-through rates by 1% within 6 months by varying keyword placement in an advert heading";
• Do : execute split testing for up to 6 months (or when CTRs increase by 1%)
• Check : did we achieve the goal?
• Act : if not, how can we improve/change the process?

The point is that Google AdWords will present you with hundreds of keyword phrases, along with data to help you pick the best one. But, it's the best one from Google's point of view, or at a push the best one that's identified using anonymised data.

It might not be the best one for you.

By split testing across candidate keywords, you can easily figure out which gives you the best return on your investment. Here, we've only put the goal in terms of CTR, but it could equally have been oriented around the eventual action of the visitor.

I've also talked about varying keyword phrases, but what about varying the words around those phrases? A great source of variations can be found in a free eBook "87 Marketing Secrets of the Written Word" by the legendary Ted Nicholas (link goes to his main site, and isn't an affiliate link.)

By picking carefully and pitching different combinations of words against each other, you will quickly build up a reliable list of variations for your chosen niche, as long as you follow PDCA, SMART and A/B Split Testing principles.

Stop Swimming With The Sharks and Bask In The Blue Ocean of Keyword Research

In strategic management, one of the concepts we come across quite regularly is the difference between the Red and Blue Oceans. The Red Ocean is the place that we envisage most competition to take place, where conditions are tough:

• the marketplace, and rules of the game are well known;
• there is high competition for demand;
• the trade-off is low margins;
• differentiation is through price. 

(source: Blue Ocean Strategy web site)

The Blue Ocean, on the other hand, is the polar opposite of all of the above - as the authors of the Blue Ocean Strategy book (W. Chan Kim & Renee Mauborgne) put it: the aim is to "Create uncontested market space" in which to do business.

Most people in keyword research and SEO are swimming with the sharks in the Red Ocean.

It stands to reason: they're using the same techniques, the same data, and leveraging the same tools. And there's nothing intrinsically wrong with any of that, except that two people in the same niche, doing pretty much the same analysis are going to naturally put themselves in direct competition.

So, the first message is an easy one: develop your own unique way of deriving value from the data.

This is the point of an area of study known as Data Science. I recently read an excellent free eBook on the topic by Jerry Overton (a professional Data Scientist) -- Going Pro in Data Science -- in which the author explained how the key to success in data science was in picking signals out of the noise.

Looking for patterns, in other words.

Traditional SEO practitioners spend a lot of time telling is we need to look for patterns in order to chase the market. It makes sense on the face of it -- go to where the market is, and satisfy its needs -- but there's a slight problem.

Online marketing is getting so saturated that the low hanging fruit in your niche may well have already been picked. Or worse still, it's become a red ocean of marketers all competing for the same space.

Retailers suffer a similar consequence; they have to compete with each other, and online retailers. Even those in retail service -- restaurants, beauty salons and so forth -- will find that they need to adopt their own data science strategies to pick their market's minds, and develop strategies to exploit their competitive advantage.

In fact, strategic management also tells us that there are resources that contribute to our competitive advantage that must be valuable, rare, difficult to imitate and organised in such a way as to make exploitation easy (VRIO analysis).

Keyword research falls into such a category, but only if you develop your own data science to reveal the answers that you are looking for. After all, there's nothing rare or difficult to imitate about looking simply for the keyword phrase with the highest CTR, or the most search volume, or even with the least competition for space in the search engines indexes.

Everyone is looking for those keywords, so even if the space is uncontested now, it's unlikely to remain that way for long!

Luckily, most modern tools allow you to download your research data, so that different data points can be combined in a spreadsheet, and algorithms developed that let you organise, sort, and prioritise keywords according to your needs.

For example, you might download data from Google's Search Console (such as CTR and SERP data per query), and merge that in a spreadsheet with search volume data from the Keyword Planner, to enable you to perform analysis to reveal likely candidates for a new campaign centred around a keyword with certain distinct properties.

Consider the following chart, for example.

It shows something I call the 'outlier technique', where we look not for a pattern, but a signal from the noise of an existing pattern. In this case, the pattern is represented by a cluster of keyword phrases that fall into a clearly defined area, and a selection of keyword phrases that don't.

These outliers show phrases for which something out of the ordinary is happening.

The graph maps the SERP for each phrase against the CTR. The phrases in the bottom left, then, are those where competition is pretty stiff. This data is taken from the Search Console account for a client blog, so one of the first messages is that there are many instances where, despite appearing high in the SERPs, the CTR is low - not many people are choosing to click on the links, despite the entry appearing (on average) within the first three pages.

There are many reasons for this, but in some cases its that the page ranks for a keyword which is not represented by the content (heading & summary) to a sufficient extent that the searcher is convinced that a visit is worth their while.

At the other end of the spectrum, top-right of the graph, we have a phrase which, although it appears way down the SERPs, has a very high CTR.

Either the content is very good, or the market is very hungry. Both of these possibilities are worth exploring further: the next graph is a natural extension, and combines Keyword Planner data with the Search Console data, for the same set of keywords.

With this iteration, we have more signals. This time, we have captured the available traffic, a simple multiplication of the volume by our CTR. It's a simple metric, but already we see that there are three phrases that show important characteristics that make them worthy of engagement for future products, publications or traffic attracting content.

So, create your own metrics, and chase your own set of outliers, and you will be well on your way to making keyword research part of your own competitive advantage.

Strategic Management of Keyword Research for Competitive Advantage

I'll apologise at the outset for the slightly highfalutin title of this post; but expressing these concepts any other way would have required a great deal more text than a blog title may have.

That, after all, is the point of these labels that we attach to larger concepts: they're shorthand.

They are, in fact, keywords.

One of the points I try to get across when coaching people about the use of keyword research in business is that all they really are is a two-way communication tool. The market tells us what it wants, and we tell the market what we've got, and why they should get it from us.

If you don't believe me, get a copy of one of Ted Nicholas' excellent books -- I recommend Magic Words that Bring You Riches or Billion Dollar Marketing Secrets -- and take a look at the tests that he has done using specific trigger words in online and offline advertising.

However, dipping into these secret pools of words and pulling some out is only half of the story: no amount of magic sauce is going to help if the basic keyword research is not aligned with your ongoing strategy.

That's where the examination of competitive advantage, and the strategic management of keyword research comes in.

The Quest for Competitive Advantage

Your competitive advantage is based on what makes your business unique, and there are competing and interesting theories on how it is best expressed. My own view tallies somewhat with what is known as the resource based view: that is, your advantage is made up of the resources at your disposal.

These resources can be tangible -- machinery, buildings and so on -- or they can be intangible -- intellectual property, processes, etc. -- but in order to contribute to your competitive advantage, they must also be valuable, not easy to replace, or imitate, and rare (the so called VRIO and VRIN frameworks).

So, for keyword research to contribute to competitive advantage, it must fulfil the VRIO/VRIN criteria by being tightly coupled to your business, forming part of the intellectual property and processes that give you an edge over the competition.

This is both time-consuming and intellectually challenging, but luckily we have a framework that can help both manage the ongoing evolution of keyword research as a competitive advantage, and stcuture the keyword research process itself: strategic management.

Strategic Management

The goal of strategic management in this context is to identify, create, and leverage a competitive advantage, in this case keyword research.

In the 1980s, Michael Porter presented a wealth of research into the emerging fields of sustainable competitive advantage and strategic management. If you want to dig a little deeper, Porter’s own YouTube channel presents his ideas much more eloquently than I have space for here.

There are, however, some basic steps in the process that are useful to examine:

• Examination of internal and external business environments (SWOT);
• Long and short range strategy formulation (mission);
• Strategy implementation (including budget control);
• Performance evaluation (measure, test, compare);
• Action through feedback (Plan-Do-Check-Act).

(Adapted from Kuratko & Audretsch, 2009)

This framework gives the business owner an excellent general-purpose tool for guiding their business through good and bad times, but also, crucially, can be aligned with the processes of keyword research and subsequent use of keywords identified as being in line with the stated strategy.

Strategic Management of Keyword Research

Here's a step-by-step process based on the above:

1. First, look at the specific keywords that define your products, as well as those being used by your competitors;
2. Decide what the strategy is to deliver as a mission statement -- more traffic, higher conversion rates, higher customer lifetime value, and so forth;
3. Use keyword research and analysis to measure the effectiveness of keyword phrases being used by the target market, and calculate the cost of traffic acquisition (online or offline, via flyers for example);
4. Set up appropriate means to measure performance;
5. Establish a regular feedback loop to take corrective action where necessary.

These five core steps are the basis of establishing keyword research as part of your competitive advantage, and, when tailored to your specific business, make it a valuable, rare, and hard to imitate resource that cannot be easily substituted to achieve the same effect in another organisation.

References

• Kuratko, D. & Audretsch, D. (2009) “Strategic Entrepreneurship: Exploring Different Perspectives of an Emerging Concept” Entrepreneurship Theory and Practice, Jan 2009, Baylor University