Now, I'm going to spill some beans here. Some secrets out in plain site for anyone who has taken the hours, weeks, and years of research to find them. Others are simply specialized and unadvertised techniques. Simply, this is a sequence I currently use to find the long tail and potentially profitable niches which can be easily and readily approached with various web publishing strategies to gain the most SERPS you can for a given KW, or to raise your site by optimizing your content for the ones you find.
Some basics you should know first:
Long tail keywords (LTKW) describe very narrow niches. They are not the niche itself, but just a way to find it. LTKW are usually four or more words long. And you'll quickly see when you run down the below sequences exactly why this is.
Google competition isn't real. Google gives you the total competition of all the individual words in various combinations and with various word endings. What you are looking for is a very exact phrase and nothing else. This is found by adding quotes (") before and after the phrase you are looking for. That will give you a much narrowed down view of how many are actually competing for that phrase. But that's not all.
The vast majority of sites out there don't use SEO - and the number of sites that do varies widely. So this means you aren't going to be competing with as many as you think. The basic points of SEO are using the exact keyword in titles, body, and incoming links. Some keyword phrases are more optimized for these than others. Particularly if people find it profitable. But usually what happens on profitable keyword phrases is that the demand jumps up, as well as the competing pages, and quickly this is no longer a profitable keyword phrase. But no worries - the web is fragmenting all the time and new niches are starting up out of the others. So you can always find niches in any particular area.
So our approach it to then:
1) Find long tail keywords based on our subject area.
2) Find out if these are profitable.
3) Find out the ones which get enough traffic to be worth promoting to.
4) Find those what competition exists for those keywords to see if it's going to be worth it.
Now, however you are looking for your subject area, it's probably based on a business plan where you have a product you want to offer to a wider public than you have. So you can come up with a basic term that describes that product or service succinctly. We'll ask the search engines to help us find related terms, so it just has to make sense to you.
So you go to Google and use their Adwords Keyword Tool (https://adwords.google.com/select/KeywordTool). This will give you alternative phrases for your term, as well as the various similar words and terms related to this area. You are best having Google account, as otherwise, it will only give you a hundred terms - which isn't always what you want. A Google account is free, however.
You've typed in the subject area and now have a nice long list of terms. Go to the "Columns" button and either select all or at least get "Estimated Average CPC" (cost per click) column showing. What this does is tell us which of these keywords is profitable - in the eyes of those who are advertizing for them.
Sort by this column by clicking on it. Now you have them high to low (or click again so you do.)
Points you are looking for:
a) Anything that has an estimated .05 as your CPC means no one is using those terms. So you can throw these away as basically not viable.
b) You have to pick how much traffic you want. I settle on 100 per day, which gives me 3000 as the base number of people I want looking for my site daily from this keyword phrase. You can pick any number you want.
Using those criteria, select anything above .05 CPC which also has a minimal amount of traffic (above 2900 or so).
Now, export these as CSV and save them to your computer. Open this up in your spreadsheet program.
Next is to copy the list of terms you've found and find the total competing pages in Google for these. There are several programs out there which will do that. I use Keyword Research Pro - but you could also simply do this by hand for every keyword nearly as fast (since KRP has to pause and mimic human action). Whether you use a program or not, make sure you can export or record the data to a spreadsheet.
Once you have your list of keywords with their Google competing pages, you export and then combine these two spreadsheets (copy/paste), making sure that your numbers align to the same keywords in the same sequence. (If you don't, then sort them alphabetically and align them that way.)
At this point you have how many searches and what the demand is, as well as CPC. From experience, I've found that you'll have too many competing pages for anything higher than 50,000. So sort these by Google competing pages from lowest to highest. Split off (insert row does it in OpenOffice) anything higher than 50K.
Now, what I do to keep this simple is to add a column called Number of Keywords (#KW) and individually assign a number by counting the number of keywords (including a, the, to, etc.). When you sort by this column (high to low) you'll see some relationships starting to be visible.
Next is to find out the actual competing pages. Go to http://www.web1marketing.com/resources/tools/keyword-competition-estimator.htm and you'll be able to enter this short list of keywords one at a time. Now, spread this out over time, since Google is sensitive about this information, and they will give you a time out if you use it too much. This tool will give you the actual pages (use "Google combined") for pages which have that exact phrase in the title and the incoming links.
Now you have the really short list of what you want to work with. This is an eye opener.
After that, you can then start looking at who is actually competing in these pages and find out more about what they are doing. Since you have some very long tail keywords at this point, this might be an optional step. But go to http://www.nichewatch.com/ and you can find the top 20 competing pages for any single keyword. And then drill down into those pages to find out what they are saying and how their pages look, etc. But what Niche Watch gives you are not only anchor and title, but also how many times your keyword is used in the text of that page. You can also see how many pages are linking to that page according to Yahoo. It's nice tool.
Another nice tool is to check out the Yahoo! Keyword Phrase Term Extraction page at http://tools.seobook.com/yahoo-keywords/. Here you can copy/paste the given text on a page and this page will use the Yahoo API to find what keywords are on that page. If you take the obvious ones and plug them back into Google's Adwords tool above, you'll get a nice idea of how to compete with that page - and outdo their writing, grabbing several related terms they weren't using.
The benefit you have is from all my experience and looking for efficiencies. Instead of checking long lists of keywords, you can now speedily narrow down to just the handful which will prove useful - and ones you can immediately produce content about which will improve the SERPs of your site.
The practical approach is to assemble your short list as above, then work it in terms of the highest number of keywords first (do the 6-word phrases, then the 5-word phrases, etc.) Essentially, while there are differences and isolated anomalies, the number of actual sites which are SEO'd closely follow the length of the keywords. Fewer exist for the longer keyword phrases – less competition, higher possible SERPs. Of course, longer keywords include the shorter ones within them, so you are really writing for several keyword phrases at once if you put the long one in your title.
And after all the above keyword research is followed, you are then simply able to get into your various publishing strategies in order to promote your product or service via these keyword phrases.
I've found that the keyword I've been wanting to rank for isn't necessarily immediately available. But other long tail phrases are - and Google is more than happy to send me that traffic instead. I've even found that a little work down these lines will result in some affiliate sales, which are always nice when I don't have a direct product in that area.
Again, this is complementary to your own analytics. Google Webmaster Tools and Google Analytics, as well as your server analytics, will show what traffic is coming in to your site and break down the terms they've been associating your site with. Keep an open mind about what you can rank for, and what Google associates your site with. It's often easier to swim with the current than against it. Being able to search out the value of the long-tail keywords people are searching your site for can bring new ideas for content - which Google is always looking for.
I don't say that you can't work at getting placed for shorter keywords. Or that you can't compete where the number of competing sites is greater than 50,000. Knock yourself out.
And this doesn't deal with local keywords which are "[product/service][location]" these will obviously have less traffic or demand than global ones. But the keyword "best house cabinet painter San Francisco" will have it's own niche to serve.
Good Hunting!
Sites referenced:
https://adwords.google.com/select/KeywordTool
http://www.web1marketing.com/resources/tools/keyword-competition-estimator.htm
http://www.nichewatch.com/
http://tools.seobook.com/yahoo-keywords/
