Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.
  • I added my url to Google http://www.google.com/addurl.html

  • I added a meta tag with description and keywords

  • I added it in Google web master tools and submit the site map

  • There is no any robots in my site

But when I create the sitemap.xml automatically with http://www.xml-sitemaps.com/ or other tools they don't find all the pages!

Are there any better tools, how can I fix the problem?

share|improve this question
1  
[This is two questions, by the way; try to keep threads to a single topic.] How long have you waited after creating/submitting your site? It's not automatic. For the second part about your sitemap, see this previous question. –  Su' Dec 24 '11 at 15:42
    
Are you using a CMS like wordpress? –  toomanyairmiles Jan 25 '12 at 10:30
    
And remember, meta information is either completely antiquated, or moderately antiquated. As John indicated, quality, relevant backlinks from quality, relevant directories and websites is the #1 rule in SEO. See my post here for more information webmasters.stackexchange.com/questions/23810/… –  Jason Weber Nov 12 '12 at 19:16
add comment

migrated from stackoverflow.com Dec 24 '11 at 15:06

This question came from our site for professional and enthusiast programmers.

6 Answers

You're doing the right things. Keep up the good work, and don't get too antsy. If you did it right, the googlebot will come. In the meantime, consider this free product: http://www.vigos.com/products/gsitemap/ ... I'm not an advertiser for 'em. I just use the product.

It generates a perfect sitemap. You'll have to make some manual adjustments once it's completed, and delete things you don't want spidered. Also, understand the Google Sandbox. Here's an SEO article I wrote awhile back:

The Problem: I hear many people ask why they're not listed in Google yet, and I wanted to offer a posting that clears the confusion of why this is happening. There is no doubt anymore about the existence of The Google Sandbox, which came into existence in March of 2004 in an effort to clamp down on SPAM websites, among other things.

The Symptoms: Basically, if your website is relatively new, and you're ranking in Yahoo, MSN, and others, but not Google, you're caught in the "Google Sandbox", which is referred to in other terms such as "Google Time Delay", or "Google Age Delay". Most refer to it as the "Google Sandbox", however.

What is the "Google Sandbox"? This is a probationary period where your website is checked out. The Googlebot will index your website, but will not list you immediately. Do not panic! Things will be okay, although there is really nothing you can do to get out of the Sandbox or avoid it. Some people will suggest putting your site on a subdomain of an old trusted domain, and then setting up a 301 redirect to your new website after google has indexed the subdomain.

share|improve this answer
add comment

but in creating the sitemap automatically with that site http://www.xml-sitemaps.com/ and others it dosn't reach all pages!

I'm assuming the question is, "Why aren't all of the pages of my website indexed by the search engines (or specifically Google)?

Just because you submit your website to the search engines, or provide an XML sitemap, doesn't mean all of your pages are guaranteed to be crawled and/or indexed or included in their search results. By notifying the search engines of your website and all of your available pages you give them the opportunity to find and crawl your content. But they have the final say about whether to include and how to rank your content. The best way to influence this is to get quality links to point to your pages (both your home page and internal pages). The higher the quality of the links, the more pages you'll get indexed and the better they'll rank. (And to take it a step further, if you do a good job with your internal links your well ranked pages can help prop up other pages in your website).

share|improve this answer
    
Google does find some pages on a website to have no relevance and refuses to index them. As John points out, rethink the link structure to and content of those pages. One thing that can cause this is having too deep a link path, these pages are viewed as internal supplemental information to their parent page and if that's the only link, may not be seen to be relevant search fodder. –  Fiasco Labs Dec 24 '11 at 18:59
    
Good tips, but it sounds more like he's saying the sitemapping tools don't reach all pages. This is indicative of broken internal link structure. There are workarounds with other sitemap generators (such as a side project by some Google developers that monitors user traffic instead of crawling the site), but it's still better to fix the internal link structure so all pages are crawlable, with higher priority pages taking less clicks to reach than less important ones. –  Lèse majesté Jan 25 '12 at 13:35
add comment

There must be links to the pages somewhere in your site. The sitemap generator will start at the root and follow all links on the pages and so on until it has exhausted all links. If there are no links to the page it will not be found.

You could manually add the entries to the XML file to create a complete sitemap.

Alternatively create a static page in your site with the links to enable the sitemap generator to find them if there is no genuine reason to link to them.

share|improve this answer
add comment

txt file via Google webmaster account and put it in root domain.It will enable google crawlers to spider your contents and If you use CMS you can create XML sitemap with some plugins available.

share|improve this answer
add comment

Try Xenu's Link Sleuth, it's a much better tool than xml site maps, and can generate sitemap style link lists.

share|improve this answer
add comment

It's 2012 you don't have to submit your website to search engines. Google is a registrar and knows when new websites are registered. If they decide to crawl you is another matter.

Google doesn't pay any attention to meta keywords and often changes the meta description they display in search results based on the search executed.

Verifying your website and submitting sitemaps is good. Most free sitemap generation tools have limits.

If you are not using a CMS with a built in sitemap generation tool then $19.99 for the unlimited version of http://www.xml-sitemaps.com/standalone-google-sitemap-generator.html is well worth it. It will crawl all your pages.

If you have submitted a sitemap to Google, and you are not blocking robots. Then there is no problem. Simply put you have to be patient and wait for Google to crawl, index, and cache your websites pages.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.