Automating and Scaling Keyword Research through SEMRush

 

Today i would like to introduce you to a keyword tool which is truly a gem for any SEO and keyword junkie. Meet SEMRush. The kind of tasks it can perform in keyword research and competitive analysis is truly outstanding and i am just plain lucky to find this tool. Within few minutes of reading my post you will have the same opinion. Here is how you can scale your keyword research.

 

Determine keywords which are driving majority of organic traffic to competitor(s) website and target them

 Lets say i want to know, which keywords are driving majority of organic traffic to the website say cotswoldoutdoor.com. This website sells outdoor clothing, footwear and equipment. So i go to semrush.com, type the URL of the website, select the country specific version of Google (in my case Google UK) and click on ‘search’ button. Boom. I get the keyword report which i can’t get from anywhere else (at least to my knowledge):

 

 

 The cool thing about this report is that not only you get a list of keywords which are driving traffic to your competitor’s site but you can also find out:

  • Where your competitor is currently ranking on Google (see the ‘Pos’ column in the chart above) for each traffic keyword
  • Local search volume of each keyword (see the ‘volume’ column)
  • Average CPC of each keyword in US$ (check the CPC column). I hope the cost is available in pounds & other currencies too.
  • Web page of the website which is ranking for the keyword (check the URL column). Just click on the page URL to find out other keywords for which the web page is ranking in top 20 on Google and getting traffic.
  • What percentage of page total traffic is coming from the keyword (check the traffic % metric) . This is a really cool metrics and can help you in your keyword selection. You get an idea of how much traffic you can expect from a keyword if you rank at a particular position on Google SERP. Isn’t that cool???
  • Number of Google search results and seasonality of each keyword (check the results and trend columns). You would not first like to target a highly competitive keyword which is very seasonal for a non-seasonal SEO campaign. This is where keywords trends can help you in your keyword selection.
  • Cost% and competition columns help PPC guys more. Cost% is the ratio of the estimated cost of buying the same number of visitors for a keyword to the estimated cost of purchasing the same number of targeted visitors coming to this site from Google search results. Competition metrics talk about the number of Google adwords advertiser bidding for the keyword.  Higher is the competition, more valuable is the keyword. This is another key factor to keep in mind while selecting keywords.  Advertisers won’t heavily compete for keywords which are not driving conversions.

 

Even more cool thing about these reports is that you can download them into excel, sort the columns, separate branded keywords from non-branded keywords and do all those cool things which an excel ninja can do.

 Related Post: Keyword Research for e-commerce Websites – Tutorial

 

Determine your top SEO Competitors in a scalable way

Before i discovered SEMrush, i used to follow a pretty tedious process of determining top competitors in organic search. Here is what i used to do. I created a list of top 5 or top 10 primary keywords. Run SEOmoz keyword difficulty tool for each keyword. Download the top 10 websites ranking for each keyword in excel. Manually highlight those websites which rank for majority of my targeted keywords to determine my top SEO competitors. This process is pretty time consuming, not very effective and is not scalable at all. If i want to determine top SEO competitors for say 100 keywords, then it would take me a day or two. If i want to determine top SEO competitors for say 10 websites then it would take me weeks. More the clients, harder is the work. Here is the solution i have found.

 

 

To determine top SEO competitors i simply clicked on the ‘competitors in Google’ link on the left hand side menu. And then just like Avinash style….. BOOM….Here we get a list of our top SEO competitors within seconds. The website amazon is competing with cotswoldoutdoor for 2933 keywords (check the ‘common keywords’ column). Therefore it is the number one SEO competitor. The website gooutdoors.co.uk is competing with cotswoldoutdoor for 2397 keywords and that’s why it is number 2 SEO competitor.

If you click on a common keywords link, you can determine the keywords for which website like amazon is competing with cotswoldoutdoor.

 

Form the chart above i can see that cotswoldoutdoor is ranking pretty poorly (rank 16) for the keyword ‘rab microlight jacket’. This keyword has good search volume and i need to work on it. You can download this list into excel, sort the list in decreasing order of SERP rankings and find out the keywords for which your competitor is outranking you. Similarly you can determine many hidden opportunities. Creativity is the key.

 

Keywords mining through competitor’s PPC campaigns

You know, i believe in why every SEO should know PPC theory. The reason is simple. By running PPC campaigns you can test your keywords and get a fairly accurate estimates of a keyword’s search volume and its potential to convert even before you target them via SEO.

It can take several months to year to rank for your target keywords. So if you ended up optimizing your website for keywords which didn’t generate the traffic and conversions as you expected then all of your hard work (along with the client’s money) goes down the drain .

 SEO mistakes are so costly that it is better to make some relatively cheap PPC mistakes by testing your targeted keywords first.

For small clients running PPC campaigns just to test keywords can be pretty expensive and not all SEO consultants /agencies are comfortable with the idea of suggesting PPC to their clients in any way or form. Here is the solution. Determine your competitors who run PPC campaigns and then determine the keywords they are bidding on and traffic they are getting through these keywords.  For e.g. outdoorkit.co.uk is one of the top SEO competitor of the website cotswoldoutdoor.com and they also run PPC campaigns on Google.

I clicked on the ‘competitors in adwords’ link on the left hand side menu to see this report. As you can see from the report, the website outdoorkit.co.uk is bidding on 524 keywords (check the ‘Adw Keywords’ column) which are driving traffic to the website. Lets see what these keywords are.

 

Boom. I can see the keywords on which outdoorkit is bidding. The site is bidding on ‘water proof trousers’. This keyword is relevant to my site and i should investigate it further. So i just click on the ‘waterproof trousers’ link.

 

Now i can see all the keyword data: avg. CPC, advertising competition, local monthly search volume etc. Through phrase match report and related keywords report i can get tons of keyword ideas.

 

Other features in SEMRush

 1. Ability to scrap competitors’ Google adwords ad copies. You can dowload the ad copies your competitor is using for adwords ads into excel and then mine keywords or just get ad copy ideas.

2. You can find potential ad buyer and ad sellers for any domain.

3. One of the biggest advantage of using SEMrush is that this tool is available in different languages, you can access its API and you can use country specific version of Google to mine keywords. SEMRush uses different database for different country specific versions of Google. So set of keywords, avg. CPC, local search volume etc vary from version to version.

The only downside of this tool is that it is not free. But then best things in life are not free. You can use the basic features for free but to get the full functionality (which you should) you need to upgrade to the pro account.  If you think there are other keyword tools which can do the above tasks much better then please share in the comments below.

Other Posts you may find interesting:

 

Join over 4000 subscribers!
Receive an update straight to your inbox every time I publish a new article.

 

About the Author:



My business thrives on referrals, so I really appreciate recommendations to people who would benefit from my help. Please feel free to endorse/forward my LinkedIn Profile to your clients, colleagues, friends and others you feel would benefit from SEO, PPC or Web Analytics.

 

 

  • http://seaworldparks.com Mike Gracen

    Excellent post (as usual) Himanshu. You are an endless source of useful SEO info. Keep it up!

    • http://seohimanshu.com/about-himanshu/ seo himanshu

      Thanks Mike.

      • Albert

        Any tool that covers more countries??

        • seohimanshu

          Is there any particular country you want to target? SEMRush targets many country.

          • Hiren G

            Hi Himanshu,

            Nice article – just one question though, how’s SEMRush for tracking my Indian site and its competitors?

            • seohimanshu

              SEMRush is useful for UK and US. I am not sure about other countries.

  • http://www.instacashkeywords.com/freekeywordsd Free Keywords

    I too love this one of a kind tool. The way it spy on competitors sites and SEO campaigns is exceptional.

  • http://twitter.com/mmhemani Moosa Hemani

    Ouch! Mike, You put too much sugar in it but very true about hemanshu.. I started to follow him not so long and his analysis and thoughts about search are mind blowing…

    On Post long time back I checked SEP Rush and found it interesting but the fact that its paid make me turned off… but you are right… FREE and useful often have a inverse proportion!

    Over all a good read!

  • http://www.twitter.com/alex_penyauski Alexandre

    Nice topic ! There is another tool that can do approximately the same thing http://www.searchmetrics.com/en/

    • http://seohimanshu.com/about-himanshu/ seo himanshu

      Thanks. I will look at it.

  • http://www.idslogic.com/seo-services.html SEO India

    It is good, isn’t it? Webmasters will be aware of the keywords their competitors are trying to pump in. So perhaps one would get more insights into the business of his direct rival and do better!

  • http://www.google.com/ Gina

    A bit surprised it seems to simple and yet uesufl.

  • http://www.interprise.co.uk Antony WIlmer

    wonderful tool for webmasters and seo for analysis our websites.it is really helpfula nd and unbeatable tool so far

    Thanks

  • Stromrechner Vergleich

    Thank you, I have recently been looking for information approximately this subject for ages and yours is the greatest I’ve discovered so far. However, what concerning the bottom line? Are you positive in regards to the source?

  • http://www.sepouk.co.uk/ Jim Watson – SEPo

    Excellent article (as ever) Himanshu – Have been using SEM Rush when it was called SEODigger.. Superb app..

  • http://www.spyfu.com Sidra Condron – SpyFu

    Hi, Himanshu I like your thoughts on why SEOs should have a good PPC understanding, but there is a lot here that still shortchanges SEOs. May I suggest that you try SpyFu? We respect what they are doing at SEMRush–it’s a good product with nice metrics. We just think that the metrics should give you real actions you can take right now.

    Instead of a % of PPC costs, we put the emphasis on dollars — what would this cost in actual PPC dollars? It’s easier to understand so you know where to prioritize. It is our goal to help SEO gain more respect (and dollars) from those who underestimate its incredible value. Let me know if I can arrange a tour for you.

  • http://www.kewl.in sanjeev

    Excellent one!! which will be best tool we can use for analyse on indian website/market ?

    • http://www.seotakeaways.com/ Himanshu

      I have no idea to be honest.

  • http://lajm.al/Shqiperi Maqedonia

    Very good write-up. I absolutely appreciate this website.
    Continue the good work!

  • http://www.mkmsinfotech.com/ seo services

    Nice Post!! I will be a successful entrepreneur one day.

    -seo services

  • http://This1That1Whatever.com/ David Wong

    Competition is fierce for ranking in search engine results. It takes a lot of effort and resources to get good rankings, and to maintain the positions.

  • sameerr allii

    Thanks for your great information, the contents are quiet interesting.I will be waiting for your next post.
    seo company in delhi,.