A site map (or sitemap) is a list of pages of a web site accessible to crawlers or users.

learn more… | top users | synonyms

1
vote
2answers
765 views

Why does google crawl my page only once even I set changefreq:daily?

I have built a new site, submitted my homepage URL to Google, and my sitemap(I have about 6,300 pages.). Now when I type site:mydomain.com, there have 1,992 pages have crawled. But they were all ...
1
vote
2answers
422 views

Creating Google sitemap.xml , is it okay for the images to be wrapped in URL tags?

I'm using a tool to generate the sitemap.xml file for me, it started to crawl my website, got the pages and all images, but when exporting it, I review the XML (to make sure nothing is wrong) and I ...
1
vote
1answer
176 views

article url format for google news sitemap

I have 2 questions about url for google news sitemap. Is it good that I use= url for sitemap witch like http://www.mydomain.com/news?query=football&title=Poland-coach-Smuda-names-Euro-2012-squad,...
1
vote
2answers
131 views

Robots.txt and sitemaps

If i where to block a web page, say 404Error.aspx, in robots,txt (i see no reason for this page to be indexed by google). However, do i still include this page in the xml sitemap or not? Edit: If i ...
1
vote
1answer
703 views

ASP.NET MVC SEO friendly URLs and Sitemap

I've created an ASP.NET MVC 3 application and generated sitemap.xml using some free online tools. It contains the following URLs: http://www.mydomain/ http://www.mydomain/About http://www.mydomain/...
1
vote
1answer
4k views

How to fix Google 404 not found Crawl Errors?

I was checking on Google webmater tool for my blog site to see if there's any indication on why my blog traffic decreased to half in one day and i saw 43 Not Found crawl errors and 5 in Sitemap Not ...
1
vote
1answer
246 views

Is this Sitemap code correct? [closed]

I'd like to check whether I have any of the code not as it should be in my Sitemap file. Not sure if this is too general a question for here... If it's an ok question, here's the sitemap URL: http://...
1
vote
1answer
252 views

I want the links in my HTML sitemap to be crawled but I do not want the sitemap in the SERPs

I have a website that has some pages and then it has a folder that has the majority of the pages. All the pages that exist in this folder could only be accessed by using my website search engine, then ...
1
vote
2answers
384 views

When does Google Webmaster Tools re-evaluate re-submitted sitemaps?

When I alter and re-submit a sitemap does GWTs immediately re-evaluate it or do I have to wait 'til GoogleBot crawls our site again? I just re-submitted a problematic sitemap and it now displays ...
1
vote
1answer
44 views

Ask Google to refresh URLs in sitemap

I submitted an updated sitemap via Google Webmaster and I want Google to rescan URLs in it, as some of them are not yet found in Custom Search. I found an instruction for for CSE https://support....
1
vote
1answer
110 views

Submitting sitemap to Google Search Console: “General HTTP error: Domain name not found”

I'm having a lot of trouble getting google search console to work with my wordpress website. I'm hosting the website myself on VPS. Everytime i try to insert xml sitemap i get General HTTP error: ...
1
vote
1answer
42 views

The Yandex index robot is redirected when requesting this site so the robots.txt file will not be used. What's the fix?

We have a website: www.example.com which redirects to https://www.example.com. Therefore when you type www.example.com/robots.txt you are redirected to https://www.example.com/robots.txt. Another ...
1
vote
1answer
40 views

Robots.txt disallow all except one directory and access to sitemap

robots.txt containts following instructions: User-agent: * Allow: /public/ Disallow: / Sitemap: http://sitedomain.com/sitemapindex.xml it seems that in this case sitemapindex won't be accessible ...
1
vote
1answer
31 views

What Gzip compression level can safely be used for sitemap files?

I can use PHP's gzencode function to set the compression level of data to compress and I want to compress my sitemap files then submit them to google. Currently my compression level is set to 2 ...
1
vote
1answer
26 views

Google Webmaster Tools 'index status' and 'sitemaps' are not consistent

In Google Webmaster Tools, Google Index -> Index Status shows 40 indexed pages: However Crawl > Sitemaps shows 5 indexed pages, and 35 submitted: There are no crawl errors in the sitemap. Why ...
1
vote
2answers
60 views

Google Structured Data and sitemap resubmit?

I've submitted sitemap in google search console a few weeks ago and slowly being indexed. I just implemented Google Structure Data to all the pages. How strong does it for the data to show up in ...
1
vote
1answer
108 views

Sitemaps & <link rel=… for quad-directional websites

I realize there are many similar questions, but I think this consolidates multiple points of confusion. This question follows from the great advice given here. I have a site that has a separate mobile ...
1
vote
1answer
76 views

Image sitemap. Ok to use the ALT text as <image:title>?

Screaming Frog uses the alt text of an image when i create a sitemap of a crawled site and places it in the image sitemap as . Is this correct? Or should the actual image title be used as in the ...
1
vote
3answers
76 views

How to have multiple sitemaps for multiple domains hosted on one hosting account?

I have two domains pointing to the same hosting account. My code in the index.php page recognizes which domain and shows different content for each domain. However, both domains have the same sitemap....
1
vote
1answer
68 views

Sitemap in different domain and robots.txt crawling

I have the following subdomains: www.example.com api.example.com The subdomain api.example.com is supposed to be invisible to Google. Its robots.txt had a disallow / directive. But I don't want ...
1
vote
1answer
40 views

How to support content for logged-in users in a sitemap for Google Webmaster Tools? (Or should I?)

I am creating a sitemap for the first time - for a fairly small client. I am surprised that nowhere (that I can find) is there discussion of how to handle content that is only accessible to logged-in ...
1
vote
2answers
155 views

Is there a more efficient way to specify multiple languages for Google in a sitemap?

According to this Google help document, we can create a sitemap telling Google that we have more than one language. But this requires indicating each URL with different languages every time. That ...
1
vote
1answer
61 views

SEO multiple ecommerce website running off the same codebase

I was wondering the about the effect a duplicate/similar codebase of products would have on a sites SEO across 190 domains. We have a dealer group with 190 for a certain class of products e.g. ...
1
vote
1answer
69 views

Should I include product URLs in my Sitemap, if I have Google Shopping?

Well, the title says it all, really. We already have Google Shopping, where they have data for all of our products in the product feed, including URLs. Should I still include the URLs to the products, ...
1
vote
1answer
119 views

Can't find 404's that are listed

I've been asked to look at the "Not Found" tab in Google Webmaster Tools, to try and eliminate as many of the thousands of 404's not listed. There are several reasons for these, but this post will ...
1
vote
1answer
674 views

Are HTML sitemaps needed in modern websites? [closed]

Most of the articles you find regarding HTML sitemaps these days seem to be based on old and outdated sources, especially this 6 year old video. 6 years in the SEO world is ancient. So SEO experts; ...
1
vote
1answer
97 views

How to tell googlebot crawl more page from my site? [duplicate]

I have a small local news site, every one increase hundreds of articles. I have set a crontab, list new articles' url into a sitemap by hour. for example, I have 24 sitemaps which is generated by ...
1
vote
2answers
295 views

How to let Google's crawler know about new Atom/RSS updates?

What is the right way of letting Google's crawler know about new Atom/RSS updates? The documentation about sitemap is clear about how to ping the crawler, but when it comes to Atom/RSS, there is ...
1
vote
1answer
83 views

GWT- Should I add separate URLS as separate sites in Webmaster Tools?

We're using an e-commerce platform that's making things a bit unclear on how I should proceed with Google Webmaster Tools. Let's say we have a store domain at: https://store.mystore.com and within ...
1
vote
1answer
1k views

I get General HTTP error: 404 not found when submiting sitemap?

I just bought 2 domain from domaindiscount24.com. I get an error: "General HTTP error: 404 not found" when submitting sitemap. Then, I use "fetch as googlebot" and my website URL, it showed "not found"...
1
vote
1answer
61 views

Sitemap submitted to HTTPS profile also influenced HTTP profile in Google Webmaster Tools

Recently I upgraded a site to use an SSL certificate and noticed strange behaviour with Google Webmaster Tools. Before starting the changes Google WMT Sitemap for http://www.example.com was showing ...
1
vote
1answer
209 views

Submitting an Index Sitemap to Bing when the individual sitemaps are hosted on AWS

Been working on the site map recently and now I want to submit it on Bing Webmaster tools. The main site map includes several others, each one for a different country. Like this: sitemapindex ...
1
vote
1answer
54 views

Is it possible to include news into Google News if there is a non-news content under the same section?

We want to include our news into Google News, and now performing research how to properly do this. At the moment we have the same base URL for news and non-news articles (analysis, interviews, etc.), ...
1
vote
1answer
116 views

How to instruct Google to crawl old, non-linked pages?

I am working on a website for a client who wants to launch with at least 2,000 articles, where in terms of links not all 2,000 articles will of course be available on the homepage or elsewhere. But, ...
1
vote
1answer
88 views

Which search engines prefer an HTML sitemap over an XML sitemap?

Is there a search engine out there that prefers a full HTML sitemap over an XML one? On a site with thousands of articles, does it make sense to include a page that has links to every single page?
1
vote
1answer
166 views

Can I submit multiple domain sitemaps in Google Webmaster Tools?

My site is setup in Blogger with a custom domain name. Currently, url.blogspot.com is 301 redirected/forwarded to www.url.mydomainname.com. The naked domain mydomainname.com is returning a 404 status ...
1
vote
1answer
157 views

Google not indexing new forum

We installed a new forum a few months ago now. The URL is: https://www.scirra.com/forum I've 301'd the old topics/threads, as well as included all the new URLs in the sitemap. Yet they still are ...
1
vote
1answer
185 views

Creating more than one sitemap and relating it to a subdomain - will I be penalized?

I'm thinking in creating a subdomain containing all the reviews from my products, for example: http://reviews.mystore.com At this subdomain, I have my sitemap.xml pointing for the reviews page about ...
1
vote
1answer
425 views

Is it a problem for Google if robots.txt blocks URLs in the sitemap?

I had submitted my site's URL to Google Webmaster Tools. I am using OpenCart. I also have robots.txt file which is accessible to Googlebot. Now issue I am facing: Googlebot is blocked from accessing ...
1
vote
2answers
368 views

Do Google sitemap URLs need to be double urlencoded?

Do Google sitemap URLs need to be double urlencoded? I am submitting my sitemaps to Google and Google is having difficulty navigating to URLs that contain urlencoded characters. An example of a ...
1
vote
1answer
70 views

Will there be a conflict between my Sitemap.xml and robots.txt?

I have a robots.txt with the following in it: User-agent: * Disallow: http://tests.compkerworld.com/report_question_error.php?question_id=*&url=* Sitemap: http://tests.compkerworld.com/...
1
vote
1answer
24 views

Sitemap for WordPress extension

I'm using an extension to integrate a WordPress blog into my eCommerce site. WordPress is installed in its own directory with a link called blog on my home page. I used a free extension to do this. ...
1
vote
2answers
90 views

Sitemap structure for network of subdomains

I am working on a project that's a network of 2 domains (domain.gr & domain.com.cy) of subdomains similar to Hubpages [each user gets a profile under a different subdomain & there is a ...
1
vote
2answers
161 views

Should expired or deleted eShop links be removed from an XML sitemap?

If I delete a product should I remove its link from the sitemap.xml also? Is there any penalty from Google for leaving a lot of dead links in my sitemap? What is the most common practice and why? It ...
1
vote
1answer
96 views

Moved sitemaps to a different subdomain and losing search referrals around the same time. Red herring or correlation?

We started to lose search referral traffic around the same time that I moved some of our sitemaps to a subdomain. Could this have hurt us? I followed Google's steps to creating a sitemap under a ...
1
vote
1answer
273 views

Is there any advantage or disadvantage to add subdomain twice in search engines?

I have a domain (example.com) and some subdomains (e.g., sd1.example.com, ...). I added example.com to Google and Bing Webmaster Tools. I added example.com/sitemap and sd1.example.com/sitemap (with a ...
1
vote
1answer
331 views

Submitting vBulletin Sitemap to Google

I'm using Google Webmaster Tools to get my forum out there, and when I go to submit the sitemap I get the following error: Your Sitemap appears to be an HTML page. Please use a supported ...
1
vote
1answer
830 views

My Domain Name Appears Twice in Most of My URLs

I've recently been adding some backlinks to other articles within my site. I don't know if it's this that is now causing me a problem. I now notice that Google has indexed my sites pages again but ...
1
vote
1answer
56 views

Sitemaps for growing content and googlesitemapgenerator

The number of URLs on my website is constantly growing (by 10-50 per week or so). Should I: Regularly submit an updated sitemap including all URLs. Have a sitemap of only the static content of my ...
1
vote
1answer
82 views

What is the proper SEO way to add and remove links to our site and sitemap?

As content fluctuates on our site, we will obviously add the titles to the new pages to our sitemap, and link list for search engine indexing. Through time, certain links will become less relevant, ...