A site map (or sitemap) is a list of pages of a web site accessible to crawlers or users.

learn more… | top users | synonyms

1
vote
1answer
4k views

How to fix Google 404 not found Crawl Errors?

I was checking on Google webmater tool for my blog site to see if there's any indication on why my blog traffic decreased to half in one day and i saw 43 Not Found crawl errors and 5 in Sitemap Not ...
1
vote
1answer
240 views

Is this Sitemap code correct? [closed]

I'd like to check whether I have any of the code not as it should be in my Sitemap file. Not sure if this is too general a question for here... If it's an ok question, here's the sitemap URL: ...
1
vote
1answer
206 views

I want the links in my HTML sitemap to be crawled but I do not want the sitemap in the SERPs

I have a website that has some pages and then it has a folder that has the majority of the pages. All the pages that exist in this folder could only be accessed by using my website search engine, then ...
1
vote
1answer
466 views

Problem with Google indexing URLs in my XML SiteMap

My XML sitemap has a lot of links - for example: nanoc.gigfa.com/post.php?id=1 nanoc.gigfa.com/post.php?id=2 nanoc.gigfa.com/post.php?id=3 nanoc.gigfa.com/post.php?id=4 ...
1
vote
2answers
353 views

When does Google Webmaster Tools re-evaluate re-submitted sitemaps?

When I alter and re-submit a sitemap does GWTs immediately re-evaluate it or do I have to wait 'til GoogleBot crawls our site again? I just re-submitted a problematic sitemap and it now displays ...
1
vote
1answer
94 views

Why is there a discrepancy between Index Status and indexed from Sitemaps in Google Webmaster Tools?

I've submitted a sitemap to GWT and things are starting to index as shown under the Crawl → Sitemaps menu. I've also confirmed indexing by doing a Google "site:" search. However going to the ...
1
vote
1answer
36 views

Should I include product URLs in my Sitemap, if I have Google Shopping?

Well, the title says it all, really. We already have Google Shopping, where they have data for all of our products in the product feed, including URLs. Should I still include the URLs to the products, ...
1
vote
1answer
84 views

Can't find 404's that are listed

I've been asked to look at the "Not Found" tab in Google Webmaster Tools, to try and eliminate as many of the thousands of 404's not listed. There are several reasons for these, but this post will ...
1
vote
2answers
134 views

Sitemap.xml generates 404's for URLs with single quotes and commas

I'm going to try and keep this as terse as possible: when it comes to single quotes and commas in the URL, I'm damned if I encode and damned if I don't. If I leave the single quote unencoded in the ...
1
vote
1answer
167 views

Are HTML sitemaps needed in modern websites? [closed]

Most of the articles you find regarding HTML sitemaps these days seem to be based on old and outdated sources, especially this 6 year old video. 6 years in the SEO world is ancient. So SEO experts; ...
1
vote
1answer
55 views

How to tell googlebot crawl more page from my site? [duplicate]

I have a small local news site, every one increase hundreds of articles. I have set a crontab, list new articles' url into a sitemap by hour. for example, I have 24 sitemaps which is generated by ...
1
vote
2answers
143 views

How to let Google's crawler know about new Atom/RSS updates?

What is the right way of letting Google's crawler know about new Atom/RSS updates? The documentation about sitemap is clear about how to ping the crawler, but when it comes to Atom/RSS, there is ...
1
vote
1answer
73 views

GWT- Should I add separate URLS as separate sites in Webmaster Tools?

We're using an e-commerce platform that's making things a bit unclear on how I should proceed with Google Webmaster Tools. Let's say we have a store domain at: https://store.mystore.com and within ...
1
vote
1answer
521 views

I get General HTTP error: 404 not found when submiting sitemap?

I just bought 2 domain from domaindiscount24.com. I get an error: "General HTTP error: 404 not found" when submitting sitemap. Then, I use "fetch as googlebot" and my website URL, it showed "not ...
1
vote
1answer
139 views

Submitting an Index Sitemap to Bing when the individual sitemaps are hosted on AWS

Been working on the site map recently and now I want to submit it on Bing Webmaster tools. The main site map includes several others, each one for a different country. Like this: sitemapindex ...
1
vote
1answer
45 views

Is it possible to include news into Google News if there is a non-news content under the same section?

We want to include our news into Google News, and now performing research how to properly do this. At the moment we have the same base URL for news and non-news articles (analysis, interviews, etc.), ...
1
vote
1answer
101 views

How to instruct Google to crawl old, non-linked pages?

I am working on a website for a client who wants to launch with at least 2,000 articles, where in terms of links not all 2,000 articles will of course be available on the homepage or elsewhere. But, ...
1
vote
1answer
73 views

Which search engines prefer an HTML sitemap over an XML sitemap?

Is there a search engine out there that prefers a full HTML sitemap over an XML one? On a site with thousands of articles, does it make sense to include a page that has links to every single page?
1
vote
1answer
329 views

What is the meaning of this sitemap error in Google Webmaster Tools?

In my Google Webmaster Tools, I received an error which I am not understanding: When we tested a sample of the URLs from your Sitemap, we found that some of the URLs were unreachable. Please check ...
1
vote
1answer
105 views

Can I submit multiple domain sitemaps in Google Webmaster Tools?

My site is setup in Blogger with a custom domain name. Currently, url.blogspot.com is 301 redirected/forwarded to www.url.mydomainname.com. The naked domain mydomainname.com is returning a 404 status ...
1
vote
1answer
103 views

Google not indexing new forum

We installed a new forum a few months ago now. The URL is: https://www.scirra.com/forum I've 301'd the old topics/threads, as well as included all the new URLs in the sitemap. Yet they still are ...
1
vote
1answer
141 views

Creating more than one sitemap and relating it to a subdomain - will I be penalized?

I'm thinking in creating a subdomain containing all the reviews from my products, for example: http://reviews.mystore.com At this subdomain, I have my sitemap.xml pointing for the reviews page about ...
1
vote
1answer
290 views

Is it a problem for Google if robots.txt blocks URLs in the sitemap?

I had submitted my site's URL to Google Webmaster Tools. I am using OpenCart. I also have robots.txt file which is accessible to Googlebot. Now issue I am facing: Googlebot is blocked from accessing ...
1
vote
2answers
215 views

Do Google sitemap URLs need to be double urlencoded?

Do Google sitemap URLs need to be double urlencoded? I am submitting my sitemaps to Google and Google is having difficulty navigating to URLs that contain urlencoded characters. An example of a ...
1
vote
1answer
63 views

Will there be a conflict between my Sitemap.xml and robots.txt?

I have a robots.txt with the following in it: User-agent: * Disallow: http://tests.compkerworld.com/report_question_error.php?question_id=*&url=* Sitemap: ...
1
vote
1answer
23 views

Sitemap for WordPress extension

I'm using an extension to integrate a WordPress blog into my eCommerce site. WordPress is installed in its own directory with a link called blog on my home page. I used a free extension to do this. ...
1
vote
2answers
79 views

Sitemap structure for network of subdomains

I am working on a project that's a network of 2 domains (domain.gr & domain.com.cy) of subdomains similar to Hubpages [each user gets a profile under a different subdomain & there is a ...
1
vote
2answers
99 views

Should expired or deleted eShop links be removed from an XML sitemap?

If I delete a product should I remove its link from the sitemap.xml also? Is there any penalty from Google for leaving a lot of dead links in my sitemap? What is the most common practice and why? It ...
1
vote
1answer
94 views

Moved sitemaps to a different subdomain and losing search referrals around the same time. Red herring or correlation?

We started to lose search referral traffic around the same time that I moved some of our sitemaps to a subdomain. Could this have hurt us? I followed Google's steps to creating a sitemap under a ...
1
vote
1answer
218 views

Is there any advantage or disadvantage to add subdomain twice in search engines?

I have a domain (example.com) and some subdomains (e.g., sd1.example.com, ...). I added example.com to Google and Bing Webmaster Tools. I added example.com/sitemap and sd1.example.com/sitemap (with a ...
1
vote
1answer
592 views

My Domain Name Appears Twice in Most of My URLs

I've recently been adding some backlinks to other articles within my site. I don't know if it's this that is now causing me a problem. I now notice that Google has indexed my sites pages again but ...
1
vote
1answer
54 views

Sitemaps for growing content and googlesitemapgenerator

The number of URLs on my website is constantly growing (by 10-50 per week or so). Should I: Regularly submit an updated sitemap including all URLs. Have a sitemap of only the static content of my ...
1
vote
1answer
78 views

What is the proper SEO way to add and remove links to our site and sitemap?

As content fluctuates on our site, we will obviously add the titles to the new pages to our sitemap, and link list for search engine indexing. Through time, certain links will become less relevant, ...
1
vote
2answers
95 views

two sitemaps - possible problems?

I am using a web shop software and for some reason the main page (http://www.mysite.com/) is not being listed / generated within the automatic sitemap.xml while every other page is. My question now: ...
1
vote
1answer
186 views

Multiple Sitemaps Doesn't Index Quickly - How To Get Quick Index

I have a serious problem in my site: http://www.coimbatorematrimony.com I will create multiple sitemaps.xml with all terms & conditions of Google Webmaster and some other search engines. I have ...
1
vote
1answer
297 views

How to create and submit multiple sitemaps into categories?

I have a questions. I need to know how to create and submit multiple sitemaps into categories? This is for a big online auction site with a lot of products that needs to be indexed.
1
vote
2answers
644 views

Google Webmaster Tools URLs in the index differ from the sitemap. How comes?

Google Webmaster Tools says ~14000 URLs in the index but as you can see in the following picture there are only ~6500 from the sitemap. How comes? I mean: my website is not properly "crawl-able". So ...
1
vote
1answer
75 views

Sitemap creators not visiting most pages

I am trying to generate a sitemap for a customers website, http://www.ruralmarketing.co.nz . When I use the online generators it only ever creates a sitemap with two urls, the index and contact page. ...
1
vote
1answer
96 views

Making search engine crawl my site

I have a site with visitors from all over the world, and I'm trying to optimize indexed pages on search engines beside Google (he indexed most of my pages). I see that on Japan search engines I'm ...
1
vote
1answer
77 views

What's Required for the AudioObject in Schema.org

We're trying to paint our site with the correct Schema.org information, and we're getting the following error: ...
1
vote
1answer
258 views

How Does One Make a Sitemap for a Flex Website?

I've got a Flex website and I'm not sure how to make a sitemap for it. For a standard HTML page, sitemaps are straight forward. As Flex sites use a "pageless" architecture, I'm not sure what I should ...
1
vote
2answers
688 views

Yahoo sitemap validation

I am trying to submit sitemap.xml (Index) to Yahoo Site Explorer but with no luck. I tried using website feed option in the site explorer to submit the sitemap, but I got validation errors. However, ...
1
vote
1answer
88 views

What are Sitemap content tags?

i have recently generated sitemap for google. I included following loc, lastmod, changefreq. My site is in two languages. Do i need to add some other tags for specifying languages too
1
vote
3answers
1k views

Where can I find a list of search engine crawler user agents and their domain names?

I'm trying to follow stackoverflow's approach of using a white list for the sitemap. I have tried to find an extensive list of search engine crawlers (and other crawlers) to use in my white list but ...
1
vote
1answer
19 views

Submitting more links than usual to Google in a sitemap

I recently submitted to Google 17000 links to businesses in my database but in the past i've only ever submitted 2 or 3 hundred. The sitemap has for the past 5 days been listed in Webmaster Tools as: ...
1
vote
1answer
16 views

Can a sitemap be hosted off-site (e.g., in s3) if you 301 redirect to it?

We compute our sitemap periodically with a chron job and store the results in s3. Can we 301 redirect https://oursite.com/sitemap.xml to the sitemap.xml hosted on s3? I know that the sitemap spec ...
1
vote
0answers
16 views

How to design sitemap for multi-language sites? [closed]

Now I have a site example.com And I want to create the Japanese and English version as ja.example.com en.example.com How should I design my sitemap.xml for the best practice?
1
vote
2answers
39 views

Duplicate Sitemap URLs

I'm in the process of replacing my old sitemaps with a new form which should better represent our site. However, as bots can take days or even weeks to scan all of our urls after submitted I do not ...
1
vote
1answer
60 views

Does the load time of an XML sitemap matter?

I have a dynamic sitemap that will change on a daily basis. Because I'm hosting on Heroku, I'm having some difficulty getting it to persist. I'm considering having the sitemap generate on request, ...
1
vote
0answers
32 views

Google Indexes are getting reduced day by day [closed]

This is what I did, first the site was indexed and we had over 10 lacks indexed entries, but some time later, our site links yields 404 for some of them and indexes were also getting reduced. Later we ...