Keyword Scraper Tools I Love to Use and How I use them

by Skeffling-ton on June 4, 2012 · 74 comments

I am a software junkie for keyword stuff and have tried quite a few cheap programs since Oct last year.  I never owned Market Samurai, I bought it straight out, no trial, and they never sent a link to get it, or replied to a couple of emails from different addresses (in case of spam blocking on either end) so I had to ask for a refund.

I am not up for paying a monthly subscription, so right now I use Stealth Keyword Digger, the free Keyword Optimizer Pro and Keyword Researcher to find long tail keywords for me.  I use all three because I know all three bring me different phrases and allow me to search a different way.  One thing I constantly chuckle at is how much “weed” comes up in these, like “best toaster oven for drying weed”.

This is not a simple step by step guide; it’s roughly what I do.  I may ramble or assume stuff, so ask if something is not clear.

How I started

I did start using Google keywords tool, then the first piece of software I bought as Stealth Keyword Competition Analyser.  Not a scraper as such, but helped me narrow down what Google Adwords Keyword Tool gave me.  I think it is an excellent tool for beginners and the perplexed.  I have some very high traffic to Web 2.0 articles after 6 months, thanks to this program narrowing it down for me.  You can use it to analyse competition on the output from Stealth Keyword Digger.  It gives you a competition ranking for each kw as Amazing, Good, Fair, Bad, and Terrible.  I only ever used the “Amazings”

I knew the AdWords tool was not ideal as it gave limited numbers of keywords that everyone else was getting too. So I was on the lookout for auto suggest keyword scraper programs by the New Year.  The biggest dilemma using these kw scrapers is knowing how significant the returned results are.  Has one person or a million people searched that phrase in Google, and when?  There are thousands of phrases and keywords generated very quickly, so I learned to make sense of them and rule most out, with Google Adwords and Estimator and allintitle searches.

I have a few that have a handful of monthly searches in Google Adwords Estimator (sometimes less than 30!) with zero reported in AdWords, but I went ahead anyway, because I had the perfect Amazon items I wanted to sell.  It was a definite risk and I think it is working.

I strongly suggest you watch any and all free instructional videos with these programs, and take notes if you have to.  These people produce a good product and they know how to use it.  Take in all that info, copy the methods if you want, or pick your favourite, or evolve your own based on what you learn and see with your own results as you go along too.  You need to get the most out of the software as quickly as possible and have the purchases paid off quickly with your earnings.

Stealth Keyword Digger

The Stealth Keyword Digger is a program that can easily generate 2000-3000 keywords and phrases when left unattended.  Which is great for the lazy among us!  It is more hands off than the other two programs and does generate the lists which record the number of Google searches too.  The longer and more specific your phrase the better, and for Amazon, I use US – English, at least 50 searches.  Occasionally it goes off track, but the bulk of the results are good.  E.g. Power drills gets you soccer drills, basketball drills etc.

Only downfall is I suspect from the hammering on Google’s door, I can run a search once or twice every few days that’s all, maybe 3-4 times a week max.  I wish it was more often.  It does occasionally want a Google captcha.  I suspect a proxy IP address would help, and multiple Adwords accounts, but I can’t be bothered figuring proxies out.  I have enough keywords words to collect with the other two programs below as well.

If you have a netbook, forget this program, unless you want to cut and paste the results with the commas still in.  I have done that, then “search and replace-d” the commas and search numbers in Excel and had to search the words again on Adwords for numbers, it;s more work but still less than doing it all by hand.  The screen seems to be set a fixed size and the export button is off the screen with no scroll bar, so no way to export.

Keyword Optimizer Pro

This is a free program and is great.   I got it by signing up on a waiting list for some Amasuite software, here’s the free link.  I think the entry fee is the banner ads up top for the odd WSOs and other products by Dave Guidon.  This has an in depth 10 min + search or a quick “all through the alphabet” search that takes under minute.  It is searching from your seed kw phrase for related auto suggest phrases for Google, Bing and Yahoo. I find the longer more specific the phrase to start, the better the results.

The beauty of this system is there is a ranking or point score for which records whether the phrase scraped is the ranked 1 to 10 of the drop down list, giving a relative measure of popularity or volume, but no actual search volume numbers.  The accompanying videos are very helpful and help you see what to do.  This is where I learned to use Google Adwords Estimator rather than the regular Keyword tool to look for numbers of search traffic.  I find Adwords keyword tool gives a lower traffic figure than can be occurring.  Some of this I know from looking at my own actual Squidoo analytics.

Keyword Researcher

Keyword researcher has a useful free version that does the first half of the alphabet, but I think is well worth the $29 or so for the regular version.  It’s a one-time fee, and personally I hate the idea of missing out that diamond of a keyword that was at the wrong end of the alphabet, like “Reviews” or “Sale, for the same amount of effort.  I guess I am addicted to keyword research and love uncovering a hidden gem.

What this program allows you to do is use a wildcard or * to replace on or more words in a phrase, like “How to * Camera” might give “How to clean a Camera”, “How to focus your Camera”, “How to choose the best Camera”.  I think it’s the best for long tail questions for blog posts or articles.  I wish you could get another second wildcard word ending in there for the “Camera*” so it did Camera and Cameras and any others, as I find myself having to remember all the combos I put in.  The master list you send the results to, allows easy removal of duplicates, so no big deal really.

What next?

When I get the list of keywords they all go into Excel and I delete the irrelevant ones and this takes some time.  If I am selling online and people are looking for Lowe’s and I do not want to do affiliate stuff with them, I filter, for contains “Lowe*” and delete what comes up.  Pretty well any place names on buyer keywords get deleted too; people are looking for bricks and mortar in my opinion.  Maybe not, but there are many better keywords out there for Internet affiliate sales.

What is left, you can use Adwords Keyword tool or Estimator to assess the number of monthly searches.  Both have their limits as when logged in Estimator now wants [ ] brackets round the keyword for exact match now instead of just hitting a radio button.  I got an Excel macro gig on Fiverr to fix that and add [ ] to everything in an excel file so I can paste them in.  You can do about 2000 at a time when logged in.  90% or more will have no traffic.  Keywords tool lets you have maybe 200 entries, so it takes forever if you need to do it in batches.

Assessing Competition

Competition analysis is simple.  Either Stealth Competition Analyser or Allintitle: xxx in Google, the lower the better.  I can’t tell you numbers, it depends on the niche, how buyer oriented the keyword phrase, the number of monthly searches and how much linking you are willing to do etc.  Start with something reasonable like 10,000 or below and then chart your results.  If they are great, maybe you can go up.  If they are horrible, choose kw with less all in title page competition.

There are other ways to asses and look at the top 10 on the SERPs but mine are very low numbers of search so I don’t bother.  If I decide to do more AdSense sites, I will need higher numbers and will assess the top 10 more regularly for Page rank, authority, links etc.

I hope that is clearer than mud.  I think you either love keyword researching or you find it boring.  It is easier to love it with the right tools that give you a fighting chance or finding something good that will make money for you.  There are easier all-in-one keyword systems, but I like a challenge and figure very few people will find the same keywords this way.

Finally,

I really should be more methodical, but I am not.  That is too boring, so if am not liking the results from one program I will use another and combine them.  I do this by pasting them all into Excel and using the remove duplicates.  If I used similar seed phrases for all 3 programs, I usually have only 30% are duplicates out of about 3000 phrases, that’s why I use them all!

Anyway ask any questions in the comments, I’ll try and clarify or make it another whole post for another day, you know how I can run on a bit!

Jeni10 June 13, 2012 at 3:28 pm

Thank you, Skeffling, for a wonderful, informative article. I’ve already downloaded Keyword Optimizer Pro. Thank you for your help.

Skeffling Lavender farm June 13, 2012 at 3:31 pm

Thanks for stopping by Jeni10 and commenting. I hope you like the program. Make sure to watch the videos, I learned a lot from them.

Ash June 23, 2012 at 1:34 pm

how are you managing with so many domains?

Skeffling June 23, 2012 at 10:11 pm

I only have 5 or 6 set up right now, the rest are Cash-parked. I am not going to do too much in the way of backlinking for a bit, just to see how they do on their own. And I am waiting for more content. I will post some more up when I get some more time.

I have had more work comittments than usual lately and it has slowed my online work a bit this month, though I have got over 20 lenses added this month and hope to get at least another 20 of not more by month end.

July will give me more spare time and I can start on more websites! I have earned about $4 in adsense between 2 of the sites and another earned me a sweet $22 commission from an Amazon sale so I think I may be on the right track!

Ash June 23, 2012 at 8:23 pm

do you manage to find keywords with 1000+ searches that are low comp or do you use loads of longtail keywords in one lens/blogpost

Skeffling June 23, 2012 at 10:19 pm

Nowawdays, if I find a 1000+ search buyer keyword with low enough comp I will do it, but they are hard to find. So most are in the 100-200 a month range with some as low as 30, if there is zero competition. So it’s more long-tails making up my traffic on the really high traffic ones (100+ views a day).

In the beginning I tried to incorporate other good competitive keywords (based on numbers) in an article too, but it took so much longer, so now I or the outsourcers write naturally with a few synonyms and that seems to work just fine.

This works best with longer articles, though it looks like the sweet spot for Squidoo is 500-600 for the 3 product format I am doing. It may need to be longer in future to compete in the SERPs, but I don’t want too much text before the product images.

Ash June 24, 2012 at 11:55 am

most of the keywords i use are in the 30/month range very high ctr but slow progress i really need a better keyword research tool i have slowed down a bit as i want to start my own blogs and other things i dont want all of my work on squidoo as i am aware you have to not put all of your eggs in one basket

Ash June 24, 2012 at 11:57 am

skeffling another thing a homepage button on the navigation tab would be great

Ash June 24, 2012 at 2:51 pm

Thanks for putting the home button up

Skeffling June 24, 2012 at 3:01 pm

No problem Ash. I am working on a few things to improve the site, so don’t be surprised if it changes quite a bit more in the next little while!

Ash June 24, 2012 at 3:43 pm

if this is your main site you could get someone to customise your theme from elance

Skeffling June 24, 2012 at 11:26 pm

Good idea. I am changing the theme first anyway and well see how it looks and performs, as I know this one is basic.

Dave August 14, 2012 at 1:10 am

Hi! I was wondering how amasuite worked for you?

Skeffling August 14, 2012 at 5:06 am

I love it. It helps you cut through the chaff that Amazon wants to distract you (shoppers) with and allows you to get to the top items quickly with very few page views or searching. I use the Product Inspector search to get a shortlist of products as it makes that makes navigation faster, then quickly evaluate based on actual reviews of users as the star rating is a good indicator but not perfect for product quality. I use it when I am choosing and writing items to review, which I do less of myself now. For my oursourcing, I have the writers choose if it is a niche I am not especially qualified in. There are times where I have the 3 or 5 items picked out and have the reviewers use the products I have chosen with Amasuite and I do this more with the Niche website content too.

I totally use the Top 100 lists for inspiration for niches and use it to come up with a new product type to do keyword research about then I can organize some articles on those. I worry less about the number of monthly seraches than the type of porduct being searched for and how well it does. If it is a best seller, it is something that sells and one way or another, poeple will find it. I could just use the top 100 lists on Amazon itself (and used to) but it is slow, you have to go through numerous pages before seeing the whole list. Amazon wants you to have time to buy while you are there.

Dave August 14, 2012 at 8:02 pm

Cool! So I gather the ROI on amasuite is worth it then.

I’ll give it a shot. Thanks!

Skeffling August 15, 2012 at 8:39 am

I think so! I’d buy it again for sure! It going to be a great help for gift lists too this Christmas! Good luck!

CashFlowsTo.com August 14, 2012 at 4:18 am

Great article! It’s a true help to gain perspective.

Just to be clear you did not and do not own Market Samarai?

How soon after doing key word reserarch did you see your performance and income got up? Also do you hav va’s doing key word research too?

Skeffling August 14, 2012 at 5:28 am

Hello there. Thanks so much. Ihope it helps. So no, I never had market samourai due to a mess-up. I realized quickly buyer keywords were very important and I needed to find them. Then I realised if I could figure out my own way, less other Internet marketers would have found those same words and phrases. So I just figured it out. I knew the more longtails I could start with, the better chance I would find good phrases, so I scrape with a few programs, and those programs were brand new out when I started using them, and as I love kw research, I was able to cover a lot of ground quickly.

By doing a lot of it manually to start (and still really), it gave me a better undertsanding of how people type queries into Google and there are definite patterns of keywords that people ask but most IMers don’t write about as such. Also many programs have preset parameters and my search threshold is way lower than most of those like. I think kw that do well with Adsense do not always work with Amazon and vice versa. You do not need as many monthly searches to do well with Amazon as you do Adsense IMO. Most kw work programs seem to be optimised for Adsense.

I know I am onto a winner when I check the front page of google for a phrase and it is ancient forum posts asking about what products should someone buy! I should outsource kw research to VAs, but I actually like it, I love looking for a needle in a haystack. I need some allinone searching software for others to use as one of my programs is unique and only works on my computer. My husband is just getting laid off from project management work and is used to writing procedure lists etc, so I expect we will be using VAs more as he will be managing them. I know I need them, but managing poeple is not my style really!

As far as how fast I saw results, I guess I got the Christmas boost that encouraged me (even though I was still blundering about, and any success is totally down to Alex Safies Stealth Keyword Competiion analyser), and I took a bit of time off in Dec/Jan to set up some Zazzle stores, so Feb and March I joined a “500 article challenge” for the year and just started to write reviews based on what I thought would sell over the summer, then written with my extremely strict kw ratios, so the income has been rising since, some of it is still from some of those maturung, but for the most part back then it took about 2 months for some of them to really earn and my income went up the most then. It keeps going up because I am adding more of the same type of articles as earlier articles still mature.

Ddraig August 14, 2012 at 12:12 pm

Have subscribed Skeffling and I have always been grateful to you, you are an amazing marketer and writer.
I wish I was more organized, it would make for an easier life,really would! I have no system when it comes to my writing and I fear that is why I am still poor!
Emma x

Skeffling August 15, 2012 at 8:47 am

Hey that is great Emma that you subscribed! I hope it helps. Thanks and cheers for your kind words. Isn’t this insane? A long way from a year ago! It is possible to do this when you scatterbrained, at least get this far! I am not that organized, it’s seeing those results that drive you to do more in bursts, even if it is piecemeal.

I am so glad Nathan will keep things organized for me now, and manage the business it is turning into, so I can still do the fun stuff that I like and am better at.

You should just get going like you were in the Spring, you know those articles earn now, you just need to think of more topics like that, and have fun writing them! If you want to do serious writing between on your regular profiles, then go for it too, just treat the product reviews like a chance to window shop! I know you can do that!

Joanne August 15, 2012 at 11:45 am

I started to read this article until I saw that you’re only linking either to the Adwords tool or to some tool that you can get commission from. Why don’t you like to the mother of all scraping tools, Scrapebox, or a few others that (sadly) don’t offer affiliate commissions? Scrapebox is way more worth mentioning than the Clickbank tool you mentioned. Bah, this is one place I’ll not come back to. Pity because I really started to enjoy your writing, but your pushy marketing plainly sux.

Skeffling August 15, 2012 at 12:34 pm

I was disappointed to see your negative comment. I’d love to hear more about scrapebox, maybe you could explain about it. I’ve never heard if it. If you know of other free ones, I’d love to hear about those too. I don’t write about or recommend products I have not used or liked. It doesn’t seem right. I only write about products I use daily or weekly. 2 out of the three (not including the two free Adwords Estimator and Adwords tools) are available for free and the third SKD) is the one that saves me the most time as I do not have to manually get search numbers for adwords from it to calculate search/competition ratios. Funny, I am running it as I write this.

I hope to see you back, but I know you can’t please everyone and it sounds like you have plenty that you can share with others elsewhere if you want to.

Tom August 16, 2012 at 2:39 pm

How is Scrapebox a better tool for accessing keyword research? I would love to know Joanne. Please share, because while it works great for certain things I have never heard of people using it to access competition and keyword data to the level of detail that the product Skeffling has mentioned.

Recommending products that work is something that any smart person will do (whether or not they get a commission from it). Skeffling is not pushy at all for this and even recommends free work arounds (i.e. Allintitle). Keep up the great posts Skeffling.

zinnee August 16, 2012 at 9:32 pm

Hi. Sorry for the stupid question I’m about to ask, lol, but can you explain what you mean by allintitle? Thanks!

Skeffling August 16, 2012 at 10:26 pm

No, it’s not a stupid question at all. If you have a keyword phrase you want to know about, I think this is considered an old fashioned way to do it (but it works for me!!) and it may not work for more competitive regular searches phrases (when looking at the first page of the Google search results for it will give an idea of how hard those are to compete against), but this is a way to look and see how much competition is likely really competing with you. I mean, some pages may just coincidentally have the same phrases in the whole document, so narrowing it to the title means it’s more likely related to the phrases you are interested in writing too.

So to do it, you go to google.com search
type in allintitle: “your phrase here” and see how many pages are returned.

I find if there are less allintitle pages with that, than the number of monthly searches for that phrase (as reported by adwords), it as likely a good bet. If there are 2 or 3 times more allintitle, it may still OK to do (especially for more generic everyday products). These are tough to find, but if you spend the time doing that, you have to write less articles to earn the same or more!

All this is assuming you are using Squidoo’s authority and don’t want to backlink or you can probably pick a phrases that has way higher allintitle pages as it likely has more searches too and more potential for lots of traffic. I am just super-conservative and it hasn’t hurt so far.

Quinton Hamp September 26, 2012 at 7:57 am

Really, really, dumb question, here.

Do you do allintitle searches with, or without, quotes

so allintitle:”XXX XX”

or just allintitle: XXX XX

Skeffling September 26, 2012 at 8:17 pm

Hi again Quinton. It’s a good question. I do it without quotes. allintitle: XXX XX

I know Tyler on Seekyt (aff link ;-) ) had them in quotes when he was experimenting in the spring and likely that is how he as them set up in the kw tutorial (I haven’t checked). Mine is less strict, but that is what my guidelines and threshholds are related too.

Dave August 17, 2012 at 9:24 pm

Skeff! You should write about how you do your Christmas/Holidays Gifts articles etc. I’m dying to know! :)

Skeffling August 18, 2012 at 9:31 pm

Hey Dave, great idea. I will write something when I get the chance. I haven’t tested any different versions of lists for earnings or click through, but can certainly share what I did last winter to get the earnings rolling!

Dave August 18, 2012 at 10:31 pm

Oh yes, yes! Please do. Please share what you did last season :)

Bethany August 25, 2012 at 1:28 am

Hey Claire FYI your Stealth Keyword Digger link points to the Amasuite page… :)

Skeffling August 25, 2012 at 8:18 am

Hi Bethany, Thanks. I just fixed it. ;-) Have a lovely weekend

Bethany August 27, 2012 at 10:49 pm

Okay so – I’m just curious about something. I already had Keyword researcher and Optimizer Pro (and amasuite, actually) and so I’ve been looking in to a few products. I would like to try out your method, at least one lens, before I start focusing on my kindle books.

One product, I pulled a list of literally over 10k scraped keywords. I realized at that point that this is probably a really popularly promoted product so I didn’t really go through with much analyzing. However, it is also a very popular product itself, a consumable, and I don’t see an active lens on Squidoo about it.

The other product is a new one on the market with high demand but I couldn’t really find many stellar keywords with allintitle results that were lower than the search volume.

So I’m starting preliminary research on another niche but I’m just curious – do you start with a niche itself, a product, or how do you identify the direction you do your research in when it comes to scraping these keywords?

Skeffling August 28, 2012 at 9:19 pm

Great questions. I was just working on this in my book. I look at what is on the top 100 lists, gift, best sellers for various categories etc (using amasuite, sometimes I sort by dollar value, highest at the top! And don’t worry about actual ranking in the 100, just that type of products are there), just browse around and pick a general product type like big screen tvs or wine openers, then for Keyword Researcher enter something like ” * wine opener reviews ” or ” best * wine openers ” on .com, an to the Aa-Zz and see what it brings up. With SKD, I use 3-4 words if I can, it seems the more specific you are the better longtails. I don’t think this works as well for new on the market items (as the google search number is likely lagging on search numbers) though there is a guy of the warrior forum Wolff that has a pretty good simple and short WSO if you are leaning towards the new stuff. He doesn’t kw research for it, but it is more for niche websites.

Part of why I do go ahead and promote the popular stuff (like everyone else) is they sell, people buy fast, can see value fast and the social proof is there in reviews and ratings. So I don’t choose the actual items I want to write about until I know what type of wine openers etc are called for due to the kw phrase. Like say “best hand held wine openers”, or “best easy to use wine openers” etc may warrant different products being listed, so you can’t do it first unless you want to take your changes and ignore the keywords stats. Something that may be worth doing if a new type of product (with many brands and models) is taking off on Amazon.

I’d love to hear about your scrapers. I am on day three of a Longtail pro trial, but it shut down my google searches, couldn’t do a hand one even..beyond the captcha stage, so need to change some settings I guess. It is nice and easy to use but I am not 100% on the scraping algorithm and it seems impossible to import into it. I’ll try it again, as it did manage one complete search for me so far and I was impressed.

Bethany August 29, 2012 at 12:01 am

Oh gotcha! Okay that’s kind of how I was choosing but I think I have been going too specific product-wise.

And with the scraper – it isn’t the scraper, it was the product itself that pulled 60k longtails! I might see if I could book a fiverr gig to use stealth competition analyzer just to see if any of them are worth looking at, but squidoo has about 170 lenses on that particular product so I’m pretty sure it’s saturated. Very high and very repeating demand, though! I need to just buy SCA – I have market samurai, but it is now using Bing to estimate competition and it is completely different than what’s in Google so pretty much worthless.

Skeffling August 29, 2012 at 10:02 pm

Wow, a popular product! I bet there is still a way to index another page if you have a different enough title and take on it. You do get a feel for what can be ranked along side existing lenses when you type in a keyword phrase for Squidoo and see what comes up. SCA has another measure of real competition other than all in title (as well as the actual # competition pages), I can’t remember what it is now, but you can always do a manual allintitle search on the ones with low comp with that tool anyway-the “amazings”. It may falsely classify some as “poor” that have low search volume, but I find if it is a best seller on Amazon, the # searches can be low and it will sell just fine.

Bethany August 27, 2012 at 11:38 pm

Scratch that – the first product has about 60,000 scraped keywords on my list… Oy vey!

Gina August 28, 2012 at 5:42 pm

Hi, Claire. This is a very informative article. Does Stealth Keyword Digger work with Mac computers as well?

Skeffling August 28, 2012 at 9:30 pm

Hi Gina, I am not sure, but I emailed the developer to find out for you. Hopefully we hear back soon.

Skeffling August 29, 2012 at 10:03 pm

Gina, Alex says at the moment SKD is only for PC’s, but you can use http://www.winehq.org/ (FREE) http://www.codeweavers.com/ or http://www.parallels.com/ to run it on a MAC. I hope that helps. ;-)

Ash August 31, 2012 at 10:35 am

I was looking on google for something to automate allintitle searches I found something called the allinscraper have you heard of it? You mentioned on info barrel you were using push button marketer at one point

Skeffling September 1, 2012 at 12:55 pm

I think so. Is it on the Warrior forum? I think that was one I was going to try to do it all in one. I found the Push Button Marketer a bit hard to program, it has amazing potential, but i don’t think I have the troubleshooting patience for it. I am a bit lazy and if something doesn’t work first time or intuitively, I often don’t bother with it if another solution presents itself.

Ash September 1, 2012 at 2:28 pm

It could be on the warrior forum although I found it on a standalone website

Bob Beckley September 3, 2012 at 11:22 am

Great post, Skeff. I love the way you write about keyword research.

I just bought Stealth Keyword Competition Analyzer and Stealth Digger as a package for $37. I’m just getting into them, and I especially like SKCA for its Keyword Rating column. I also like its REAL Competing Pages column, which combines the techniques of allintitle and adding &start=990 at the end of a url search string to bring back a more accurate number of competing pages.

Now I’m on to read more of your posts. I’m definitely bookmarking this site for inspiration and solid advice. Keep up the great work. :-)

Skeffling September 3, 2012 at 7:29 pm

Thanks Bob and thanks for reading! I am so glad when people start to take action. I didn’t know there was a deal on the two. Good for you for finding it! I hope you continue to enjoy the site. It is slow going for me as it takes me ages to write a monster post and we are moving house too so everything is in upheaval!! Hope the keyword tips help! Claire

Bethany September 4, 2012 at 10:44 am

hey I got another question for you – how long does it take you to narrow down a keyword or set of keywords? I see you are posting LOTS of new lenses (with outsourcing and your husband’s help) so I was just curious how long it takes usually.

Skeffling September 4, 2012 at 4:17 pm

I’d say about 30 minutes for the stealth Keyword Digger (took over an hour with our slower internet), and maybe an hour for the competition program. You can use SKCA then do manual allintitle on the Amazings. Or you will learn to just look for promising looking phrases and do those manually. So that will scrape 2-3000 words, maybe 200-1000 will have any search volume, and I allintitle those (using a custom program I bought on Fiverr that occasionally makes google lock me out!), calculate ratios and and usually pick maybe 5 good ones from each batch. So about 1 3/4 – 2 hours.

If I do the Keyword Researcher or Keyword optimizer pro scrapes, I paste the potential kw into Excel where I got a macro to add [ ] for exact match and cut and paste them in the Adwords estimator (US and Max click $99, no more than 1000 at a time) then download the results, sort and multiply the daily # by 30 to get monthly searches, Then work out allintitle then do the ratios or allintitle/# monthly US searches. So closer to about 3 hours with sorting and selection. Quite a bit of it is manual, but I tried Longtail Pro this week and it is triggering Google to lock me out of searching for automated queries, so I must have messed something up. It will speed it up a lot of I can get it working OK.

I found with the SKD (as it looks up search numbers automatically) I could run it 3 times back to back then have to wait a few days as adwords did capchas exponentially closer together. Sorting and outsourcing, checking and editing them in groups often 10-15 at a time took a few days, then after a few days I’d run it again. Sometimes between times I’ll run the other two scrapers, but I am getting lazier and using them less often. Most of the speed limit is not the actually time to run and porcess the info but how often you could run these! I need to learn about proxies. ;-) Now SKD just added a capability where you can have it switch Adwords accounts 70% through the search to keep them going, so I need to see how well that works.

Bethany September 5, 2012 at 12:55 am

interesting! I bought the program this morning and compiled my list that I’ve been scraping for a few days in between boxes (lol) – I ended up with 1466 keywords to put into SCA and it has taken about 8 hours to do half of them. I do have VERY slow internet though.

Something I haven’t seen you mention is the Keyword Researcher tool has the ability to check “related searches” in Google on every keyword in your master list. So, for instance, you get a master list of 500 scraped keywords, you can click over to the related keywords tab, import the master list, and it will pull related searches for each keyword (You know, they used to have them listed at the bottom of the SERPs). I found some related niches that way – it does “dilute” the keyword list a bit, but I’m glad because so far I’ve found two different (but related) niches that have some really awesome keyword opportunities.

Skeffling September 5, 2012 at 8:41 am

Bethany, it is bound to vary! I used to set mine overnight! Didn’t go to bet without it running!

But we went from the 3rd fastest to 2nd fastest internet about 6 weeks ago. We live in the country of course and the SKD scraper was taking about 2 hours at that point, and now is way less. I guess over those thousands of searches it makes a difference. I had to get a new laptop back in April and got a Quad core i7 processor because I wanted the computer to be fast too, and not stutter when I was typing or working on it! I had been using a netbook with 1 GB RAM from Jan to April and OMG it took days to run search stuff, that was when I ran it overnight! I guess I’ll wait til we move before we get the fastest internet. No or slow internet is a deal breaker for us on a new place, which is tough as we want a remote location!

Thanks for the tip on Keyword Researcher. I have never used that or knew it could do it. Guess I better watch some videos over there! Can you import any other saved .txt list of keywords, maybe scraped from elsewhere?

Bethany September 5, 2012 at 10:16 am

Yes it was a feature added in the last update… I’m SO glad they did too because (as SCA is *still* running) I keep seeing new niches and some really awesome keywords in this list I pulled. One keyword I found had 8 allintitle and 173 searches per month… yeehaw :)

Oh and you can’t technically import a list, but you CAN paste into keyword researcher. So I actually will compile my list of keywords from other scrapers, then paste it into the master list & hit cleanup to delete the duplicates.

Now if we can only bug Ton enough to add custom search strings and even better… some kind of script that would allow us to program which search strings we want to scrape so we don’t have to enter them manually each time… would be super awesome.

Bob September 8, 2012 at 8:22 pm

Hi, Skeff/Claire. Sorry to bug you again, but what kind of ratio are you looking for when doing allintitle/# monthly US searches? Many thanks for your thoughts.

Skeffling September 11, 2012 at 11:30 am

Hi Bob, it varies, but for Squidoo and no back-links, the highest I have done and been happy with so far is about 10 times with less than 200 all intitle pages. Less than one is the best though. The higher the ratio (more allintitle or lower searches) the longer they take to mature as sell. You can likely go way higher and think about backlinking but I find enough good kw in at those levels for me to continue there.

Bob September 6, 2012 at 11:37 am

Hi, Claire. Got a couple of questions for you:

(1) what’s the difference between allintitle and SKCA’s “REAL Competing Pages” (known in some circles as GCode)? Aren’t they essentially the same?

(2) now that Google Traffic Estimator doesn’t return search volume anymore, where do you get it from? Google Keyword Tool? Market Samurai? Somewhere else?

You are a great voice/helper/teacher for us little people out here who are trying to get a toehold on IM. Thanks for sharing your time and research and writing.

Skeffling September 8, 2012 at 8:32 am

Hi Bob, Thanks for the kind comment. I am trying to help as much as I can, and I know the way I do it is not straight forward but it is working for me! Keyword research is definitely a learned skill and constantly evolving as far as I can see, that’s the challenge. Making it faster, and more accurate it something to strive for, so testing new software here and there every so often is a must.

I have ended up using allinitle myself as I know how it is calculated and it makes sense to me as a measure of relevance, I started to use it as a final check after a SCKA run, then ended up going with it more and more.

Adding the “&start=990″ to the end of the url to give actual Google competition like SKCA does is a smart way to do it also, especially as it is “asking” the very group or company that will be ranking your page. I find the allintitle simpler and I can somewhat predict it now based on the keywords as I have seen so many results, so I am more comfortable with it. Finding it automatically calculated with bought software is not as easy to find, and I was doing it manually for a long time.

For Adsense Estimator you can still get numbers back if you
1) Put your phrases in (in [] brackets for exact match) in the US
2) After it returns the messed up looking graph, enter the max click amount as $99 and it will bring up all your numbers
3) I sort by daily impressions
4) Download/export, then multiply the daily impressions column by 30 for monthly searches.

I know this is convoluted and not simple and I’d love to have an easier way to getting non-google adwords data (and am considering wordtracker) but I know going with straight Google adwords gives you a lot of false negatives. What I mean is I know from my Squidoo stats that a word is being searched for quite a bit more that Adsense says has 0 monthly searches on. This is ‘specially evident with buyer very long tails.

Bob September 8, 2012 at 8:23 pm

Wow! Super helpful answer. Thanks so much.

Chis September 23, 2012 at 9:51 pm

I too am a keyword software junkie :-) but that is part of the fun, right?

I also own and use/have used KeywordResearcher, Keyword Optimizer Pro, Stealth Keyword Competition Analyzer,Stealth Keyword Digger,Market Samurai
,GKT and countless others. These days my main tools are GKT and SECockpit along with KeywordResearcher (can’t wait for verson 6) and ScrapeBox (I know, black hat, but you won’t find a better tool for long tail keywords discovery).

So I use GKT, KeywordResearcher and ScrapeBox to collect a big list (10K +). Enter those into SECockpit (can be a chore because you are limited to 800 keywords per list entry) and let it analyze. There are also Squidoo, Youtube, Adsense and Amazon columns to help you zero in on the keywords that meet your requirements. Kind of a intense research process but I like the “looking for a needle in the haystack” part also.

In the end I can filter for low competition and have hundreds (thousands) of good keywords to use on my website :-).

So the down side, cost (no surprise :-)). I got in at the very begining so I’m not paying the current price but still expensive. I’ve looked long and hard for something that would replace SECockpit, and save me that monthly fee, but no joy (yet). The closest one is Stealth Keyword Competition Analyzer. Great approach and Alex is good to work with. Its just slow to analyze big lists.

There is a guy building bots (StealthBot) that can automate SKD and SKCA. Another great idea but its a little complicated running the bots because they use sendkeys, so they really need to be run on a dedicated pc.

I just recently discovered Squidoo and I’m excited to get my first lens published (should be in the next couple of days). Hopefully a great place to use some of my keywords :-)

I look forward to your further musings on Squidoo Amazon marketing.

Chris

Skeffling September 24, 2012 at 11:49 pm

Aaah, I definitely see another keyword addict, Chris! Sounds like you are well set for Squidoo. As a last check, make sure you check there aren’t too many lenses on there with the same kw. I do a site search with quotes. When looking at results, realize you can add extra words to beginning or end and they may still rank just fine. Once you check a few you’ll see how much variation in titles will still get ranked with same main kw phrase.

I just tried Long Tail Pro and need to test further and ask Spencer some questions. There is another “All In Scraper” I want to try but not sure if it scrapes kw or just does allintitle etc. I haven’t tried SEO cockpit but hadn’t heard too much about it. I certainly like to pick my own parameters. Many of these rule out low search numbers for Adsense, but those phrases do sell Amazon anyway if there is good buyer intent and low competition! I sort mine manually in Excel with ratios and filters then pick based on buyer intent.

I have never tried Scrapebox, why is it so Blackhat? The Sendkeys on the Stealth bot, does that ban you, or is it slow? why do you need another computer? We have a spare laptop here, but I am curious.

It is funny how fast the time flies when you are working on keywords, but it is worth it to find those gems! Keep me posted if you find the dream kw system! Good luck on Squidoo! Let us know how it goes! Claire

Oloo September 30, 2012 at 12:51 pm

Hi Claire, I’ve read the whole blog, its fantastic – both the method and your progress! I’ve actually read some posts and comments twice/three times, made notes and I definitely want to try this! The biggest problem for me is your keyword research process and this post brought some light into it. I’m still a little bit confused though, probably because I have no experience with neither Squidoo nor Amazon. Would you mind answering a couple of newbie questions? Have I understood the phases of your keyword research correctly:
1. Find seed phrase in amazon (must meet the following criteria: price $100-$300; rating above 4) using Amasuite,
2. Use keyword generation tools to obtain a list of keywords candidates,
3. Keyword analysis based on: allintitle (must be below 300), allintitle/# monthly US searches ratio – must be below 1 for quick ranking, buying intent (buying keyword).

Do I have it right?

What tool do you use to bulk check of allintitle? SKCA? Are you checking allintitle for USA or global? Do you check allintitle with “” or without (with “” usually gives me lower number)?

You’ve mentioned this a couple of times and it is probably obvious to Amazon affiliates but what is a “buying keyword”? Is it a keyword with high (over 0.75) adwords competition? If so do you check the level of adwords competition in the US? Or does this simply mean that the keyword has to include “buy”, “best”, “top”, “review” in it?

Any luck with LTP? I’ve been using it since early beta version and for Adsense it is fenomenal.

Skeffling October 1, 2012 at 8:39 pm

Hi Oloo, Thanks so much. The keyword research is certainly not tidy, but it looks like you have the jist. I just use Amasuite for top 100 list ideas in random categories, then use that type of product of something promising as a seed kw. For allintitle, I use a program I had made on Fiverr in May, but LTP does all in title too and another called Allinscraper will do that too. I had Macromarketer do it too, but I was too lazy to learn to set it up properly, super steep learning curve there.

I had some wierd stuff happen when using LTP, I really want to be able to use it as it looked good on the first trial. I think the problems were maybe coincidence but haven’t had chance to inquire about it, and have been nervous to try it again. Allintitle is on google.com, no quotes. You could use quotes, but just know any ratios or numbers I mention are for without quotes.

Buyer keywords, technically you are probably right, likely more in an Adsense scenario. I see buyer keywords as what you type into Google when you are about to buy something, it is as simple as putting yourself in the buyers position. I believe the best sales (with lowest returns) are for items where the buyer wants something and can save him time, hassle or money on the purchase. Time savings keywords are top rated, best-selling, best short list type phrases that imply selection has been done and the reader can save time by catching the highlights. Hassle savings are keywords with consumer reports, reviews, top rated items, people don’t want to have to return dud items. Saving money things are deals, refurbished, cheap, low cost etc. People buy items for other reasons, to look cool or feel good, but I personally think those items are more likely to be returned. There are many, many more examples of these phrases if you start to think about it and many can be applied to multiple niches. Hope that helps.

As I said it isn’t tidy and isn’t set in stone, you can go over 300 allintitle if you want to back-link or wait longer for articles to mature, it is up to you. I mean this really is an art as well as a science and it works best when you look at your results, listen to your gut and evolve your own plan that makes sense to you and it worth your while.

Tom October 2, 2012 at 12:59 pm

Not sure I understand the Allintitle reference here. Are you basically saying that when you check allintitle on Google.com you type:

allintitle: keyword phrase

You don’t use quotes around your keyword phrase? I find that if you use quotes it gives you very different numbers than if you don’t use quotes. Just wondering which is more accurate or a better gauge for assessing competition.

Skeffling October 4, 2012 at 10:56 pm

Yes the numbers are way different and make no sense sometimes! I think it depends what you want to know or how you want to compete. This is my theory.

I err on the side of caution when choosing kw and an in essence an article title/topic. I prefer no quotes. I think if those words are in the title of any page, it is likely about the same subject as my article. So I am being more critical of the keyword phrase and evaluating it as more (imaginary) competition that if I compare to the phrase in quotes. It works for me. Much of my traffic is from many longtails at these search levels, so competing for one specific main keyword isn’t quite as critical as it may be for a higher search volume monthly search.

I think in quotes may be better used if you know you are say writing something shorter, for a more competitive search term, where you are totally going for that phrase (maybe with higher kw density?) then, when you really need to know and how many you are up against.

I find when any of these allintitles are in single digits, they are way more accurate to gauge and either way you are probably safe! ;-)

Alana November 7, 2012 at 1:07 pm

Hi

Congrats on all your successes so far, I have wanted to try your method of keyword research for a while now so I purchased SKD ad STCA but no matter how many times I reread this post I am still a bit confused and hope you can help me..

After I have a list of amazing kws from SKCA, do you use allintitle: on the “amazing” keywords to find results with under 200 pages? without adding &start=990 to the results?

Is the ratio you are looking for simply more searches per month than the results of allintitle alone or allintitle then &start=990?

Thanks!

Skeffling November 7, 2012 at 1:55 pm

Hi Alana, thanks for reading. That is great you have some good keyword tools. On Google.com I just do a plain

allintitle: keyword no quotes or any other code

I generally use a ratio in a Excel spreadsheet to figure out which keywords have the lowest competition and the highest searches. If you can keep the competition pages under 200, and the ratio of alltitle pages/searches under one in that ratio, you are in pretty good shape for Squidoo ranking.

Sometimes I will go higher with the allintitle (say 300) if the monthly US exact search is 1000 or something. I have come to expect that these lenses will get traffic/mature a lot later than those with all entitled competition in the single digits.

Generally as long as the exact monthly US searches are say 50 or so and above, and there are very few allintitle competition pages, I go ahead with that keyword. I hope that helps and does not make things even more confusing. ;-)

Alana November 8, 2012 at 7:39 am

Thank You!
I am easily confused! :) I have it now, I just need to get a list of keywords in time for Christmas now!
I am already seeing results with squidoo, I am working on my lenses since July this year, in August I had about 10 sales, in September about 24 and last month over 80! so I hope to beat that this month!

Thanks again

egdcltd November 17, 2012 at 1:16 pm

Are Stealth Keyword Digger, Keyword Optimizer Pro and Keyword Researcher essentially the same type of tool? Or do they do different things? So, would you take the results from any of the three tools and then run them through some different tools, or would you take the results from one and then run it through a second?

Also, do any of these come with actual written instructions and tutorials, or do they all come with just videos? If there’s one thing I’ve learned from video tutorials over the years, it’s that I can’t use them to learn things.

Skeffling November 24, 2012 at 11:22 pm

Hi egdcltd,
That is a good question, yes they are all essentially the same type of tool I am sure they work differently, but they scrape to find you keyword phrases that the Google ad words tool will probably not bring up do to lower search volume. The outputs of each one a slightly different Excel spreadsheets. Most of the time I run them separately, but if I am starting a new niche, and I want all the information I can gather (alright no a niche is really small and has very few keywords), I will put all the output together. It is pretty well just the words and none of the other measurements, then use the ad words estimator to get search volume. Once I have that, I run them through my allintitle custom program I had done on fiverr, but there are other allintitle searches on longtail Pro.

I think they are nearly all videos, I think Dave Guidon’s software may come with PDF files as well, but I cannot remember for sure. I can look into that for you, but I am going to the in the UK as of Monday (in Skeffling!) and I have a feeling I am not going to have any Internet for the duration, so it may be a while!

egdcltd November 26, 2012 at 9:00 am

You’re going to be passing pretty near me then. At least there aren’t any flood warnings in that area.

Skeffling December 4, 2012 at 8:46 pm

Yes, I guess so. I am safely back home now though. I guess there were over 260 flood warnings the one day we checked as we had planned a trip to York, but did not end up going.

Brandon December 6, 2012 at 10:06 pm

I’m thinking of getting some of these tools, but I was wondering if you could list all of the tools you use on an on going basis with links to all of them in one place and like one sentence that explains briefly how you use them. I just sorta got confused or lost with all the different names that were listed. If you would I would be grateful!

Skeffling December 7, 2012 at 8:35 am

The bare minimum ones I use the most (if I am being lazy) is the Stealth Keyword Digger and Keyword Researcher to generate keyword phrases by them scraping (searching the internet) for drop down menus. Once I have search numbers, and they are over about 20 monthly (on buyer phrases), I search them through an allintitle search.

In Germany, Malonie on Fiverr made a program for me to do allintitles but I have to put in captchas manually (cost me $30 and it was lot of work for us, testing it, so I think he has asked for $50 since), but there is an allintitle scraper program on Warrior Forum that some are using. I haven’t tried it. I like mine! Longtail pro has an allintitle too, but I don’t think you can put your own keywords in to do that, you use the scraped ones from LTP, I may be wrong, I know Spencer has a new version out I may go and check out right now….

Markwell March 21, 2013 at 11:22 am

Howdy!

Just letting you know i enjoyed your articles. A question for you though!
What is the search volume you use as the minimum?

Have a nice day,
Mark

Skeffling April 17, 2013 at 7:31 am

Thanks Mark! Honestly, I have done search volume of local of 12 for USA, which really begs the question, is is really keyword research at that point? I am happier with 50+ but if the product is expensive (good commish) with maybe only 1 or 0 allintitle pages for a buyer phrase, I have been known to do it, especially if I have had success in the niche already. I hope that helps. I know I am overly strict, and overestimate my competition, and am probably missing out on opportunities, but it has worked so far!

Skeffling September 5, 2012 at 10:50 am

Awesome! I bet you get a sale in the first week on that one! I know the search strings would be great, would certainly save time! I know it is not the norm, but I love this side of the of line stuff! When I had a store on ebay I would look through thousands of listings of antique linens for sale on Ebay, to find some that were incorrectly ID’d and cheap enough to buy to resell there and make money on! Very similar to this though but we have tools to help us do it faster (or more conveniently anyway, haha!)

{ 1 trackback }

Previous post:

Next post: