A site map (or sitemap) is a list of pages of a web site accessible to crawlers or users.

learn more… | top users | synonyms

2
votes
1answer
45 views

URL in sitemap with temporary/dynamic noindex

Our scenario is we've got multiple pages set up for category-based browsing, such as example.com/products/boardgames A list of the canonical URLs for the categories is in the sitemap, with a frequency ...
2
votes
1answer
23 views

Allowing Google to index images on s3

We host our sites' static content on S3. We also have a very open robots.txt; User-agent: * Allow: * This is because in Webmaster tools I get thousands of warnings Sitemap contains urls which are ...
1
vote
1answer
38 views

Ask Google to refresh URLs in sitemap

I submitted an updated sitemap via Google Webmaster and I want Google to rescan URLs in it, as some of them are not yet found in Custom Search. I found an instruction for for CSE ...
0
votes
0answers
6 views

How to Create a sitemap in MEAN stack? [duplicate]

Well I am creating a Application in MEAN stack but I don't know how to crate a sitemap for MEAN stack
1
vote
1answer
125 views

Must the image url in a sitemap file begin with the same domain the file is stored on?

I remember when making a sitemap, all URLs defined in between <loc> and </loc> must be part of the same domain as the domain the file is stored on. For example, I can use this line in my ...
3
votes
1answer
68 views

Sitemap.xml on separate server (can't use robots.txt)

We have several domains managed by the same code base. They share content and assets between each other but are separate entities online. I have started generating sitemaps for these sites and ...
1
vote
1answer
26 views

How important is it to include <lastmod> in a sitemap?

In a sitemap, is it important to include information about when the url was last changed in a <lastmod> tag? How does not including a <lastmod> tag in a sitemap affect how my site is ...
6
votes
2answers
3k views

Is it necessary to submit sitemap to www and non www?

I have submitted my website to Google Webmaster Tools with both www and with no-www. I had made the domain without www to my preffered domain and have submitted a sitemap. My question is should I ...
1
vote
0answers
66 views

URLs not allowed error for one property, but not for the other?

All of our XML sitemaps are located on another domain than the domain they're referring to, for instance: - Sitemaps would be located at sitemap-domain.com/sitemap-1.xml - URLs being referenced in ...
7
votes
3answers
946 views

How soon Google is supposed to include new pages to search

I'm using Google CSE on my site. As soon as new entity is published, it's immediately added to sitemap.xml rendering script too. How soon after Googlebot fetches new sitemap new entities should be ...
1
vote
1answer
44 views

Having trouble with Google indexing after extending SSL cert across whole site and resubmitting sitemap

I recently extended out SSL certification to every page on my website in an effort to gain some rank. I created and claimed the new https properties (both www and non www) in Google Search Console and ...
1
vote
0answers
24 views

Is sitemap in sitemap_index getting indexed by Google webmaster?

it sounds a silly question, but I actually could not find a reference to it. I have created a sitemap_index with about 100-ish sitemaps. No errors. I submitted only the sitemap_index to Google ...
3
votes
1answer
80 views

Submitting a URL to Google using PHP/cUrl

I've been successfully submitting xml sitemaps to Google using cURL. The URL is http://www.google.com/webmasters/sitemaps/ping?sitemap= The problem is that XML sitemaps take several days to be ...
2
votes
0answers
22 views

Which is faster for sitemap submission: ping or submit via search console?

Previously I asked a question about marking sitemap warnings as fixed here: Resetting sitemap warnings in Webmaster Tools I look in my account today and find such warnings and errors are always ...
1
vote
0answers
38 views

Google isn't indexing all of my sitemap URLs for an SPA website [closed]

I have a JavaScript (React) SPA website. To speed up the process of indexing, I submitted a sitemap text file at Google Webmaster Tools. The sitemap has a measly total of 70 URLs. In spite of the ...
2
votes
2answers
27 views

Sitemap compressed is in unsupported format but valid if uncompressed

I've build a sitemap_index pointing to different sitemaps "sitemap1.xml" If they are uncompressed, they are successfully validated and accepted (Google Webmaster). However, when compressed, they are ...
1
vote
0answers
19 views

Ping Sitemap or Individual URL

On self hosted wordpress blogs when you update a post it automatically pings services such as pingomatic etc. This sends the URL of the updated post and not the xml sitemap. What's more beneficial? I ...
0
votes
1answer
25 views

Should I be using both “rel=canonical” and “rel=”alternate" within my sitemap?

I followed this setup in my sitemap that Google outlines on this page: https://support.google.com/webmasters/answer/189077?hl=en . <url> <loc>http://example.com/en</loc> ...
3
votes
1answer
29 views

Internal links to pages vs. sitemap.xml links to pages

I have a page that is not linked to from anywhere, but it is listed in sitemap.xml. It will be crawled and display in the SERP. Would it be preferable, from a SEO perspective, to have the page ...
0
votes
0answers
14 views

Google sitemaps: Url is not allowed at this location

Hi I have made a sitemap for my website. I went to google search console and added my sitemap.xml when I click test it displays the details and says that all 8 URL's are not allowed? what have I done ...
0
votes
1answer
641 views

1000 page submit limit in Google Webmaster Tools sitemap

I'm currently managing Webmaster Tools for some Wordpress sites and I'm curious as to why none of the sites are showing more than 1000 pages submitted. Some of the sites are much larger but I can't ...
1
vote
0answers
52 views

Google can't crawl my rewritten URLs

I tried to submit my robots.txt and sitemap.xml for hours now - changing directories and .htacces rules like a horse. For some stupid reason Google can't crawl my pages and gives me warnings, that my ...
1
vote
2answers
90 views

Blogger sitemap for Webmaster Tools

I'm developing a new Blogger Blog, and I'm about to send its first sitemap to Google Webmaster Tools. The thing is, I've found 2 articles about Blogger sitemaps, in which one is suggesting to send the ...
2
votes
0answers
25 views

How to make nginx override web app route for sitemap and robots?

I would like to make nginx serve sitemap and robots. My problem is that route http://example.com/ is handled by flask, configured to handle request with certain patterns - ...
3
votes
1answer
148 views

Can HTML sitemaps be listed in robots.txt just like XML sitemaps?

I will have two sitemaps for my website. The first sitemap is the .xml version for the search engines. The second sitemap is a sitemap.php on my website for the users. I know I can list the ...
1
vote
3answers
34 views

Does the sitemap necessarily need to be in the root folder?

I have created a sitemap index and all the additional sitemaps that I want to upload into webmasters tools. Do I need to keep them in the root folder or can I just create a folder called "sitemaps" ...
0
votes
2answers
37 views

How can I compare sitemaps between two sites (staging and production)

I have rebuilt a site for a client but want to limit the SEO fallout upon deployment of their new website. How can I compare sitemaps from both sites in an effort to make sure that the sites are ...
2
votes
1answer
49 views

Forcing google to rescan sitemaps instantly

When URLs no longer exist on my site, I give a page and issue an HTTP 410 status (the GONE error). I created a script that allows my co-administrator to add and remove pictures from the site and ...
1
vote
2answers
57 views

Google Structured Data and sitemap resubmit?

I've submitted sitemap in google search console a few weeks ago and slowly being indexed. I just implemented Google Structure Data to all the pages. How strong does it for the data to show up in ...
3
votes
2answers
209 views

How should you inform search engines of the location of an XML sitemap?

There are 2 ways to make search engines aware of a sitemap's location that I know of: Include an absolute link to it in robots.txt Submit it to them directly. The relevant URLs are: ...
1
vote
0answers
72 views

Can a sitemap be hosted off-site (e.g., in s3) if you 301 redirect to it?

We compute our sitemap periodically with a chron job and store the results in s3. Can we 301 redirect https://oursite.com/sitemap.xml to the sitemap.xml hosted on s3? I know that the sitemap spec ...
3
votes
1answer
69 views

If I rename a sitemap, does it affect my SEO?

I would like to rename my sitemap: Would it affect my SEO at all? Would I have to resubmit it on GWT? If I have a sitemap-index.xml file, that remains the same, but I change the name of one of the ...
2
votes
1answer
44 views

Redirected site to www and submitted www URLs but Search Console only shows 5 pages indexed

Recently I submitted the www version of my site to Google Search Console and 301 redirected all URLs from non-www to www. I submitted the URLs in my sitemap.xml. The problem is that I've submitted 62 ...
2
votes
1answer
47 views

Does Google prefer HTML sitemaps or XML sitemaps? And does it hurt to implement both?

I just read that Google prefers HTML sitemaps. We have so far, only implemented an XML sitemap. Would it hurt to add an HTML sitemap, and not remove the XML sitemap? Also, can you submit the HTML ...
1
vote
2answers
41 views

Why do some websites hide their sitemaps and exclude mentioning it from the robots.txt file? Does this affect SEO at all?

A lot of websites hide their sitemap.xml file. What I mean by this is that they make the URL something that a person cannot guess and then do not mention them in the robots.txt file. So my question ...
2
votes
1answer
356 views

Google Sitemap for multiple domains of same site

I have a website and its domain has been used for several years. Now I want to use an additional new name for the site, so I got a new domain. There are four valid addresses now: http://old.com, ...
2
votes
1answer
113 views

hreflang tags in sitemaps - your site has no hreflang tags

We are using hreflang tags in our sitemaps and we're currently experiencing a number of issues with errors in WMT suggesting 'no return tags' and 'your site has no hreflang tags' that don't seem ...
1
vote
1answer
29 views

Is it OK that I submitted sitemaps for www and no-www?

I have submitted two sitemaps of the same website. One for www and one for no the www version of my website. Should I delete one of the 2 sitemaps on the URL with redirect? In that case I would leave ...
1
vote
1answer
75 views

Submitting sitemap to Google Search Console: “General HTTP error: Domain name not found”

I'm having a lot of trouble getting google search console to work with my wordpress website. I'm hosting the website myself on VPS. Everytime i try to insert xml sitemap i get General HTTP error: ...
1
vote
1answer
37 views

The Yandex index robot is redirected when requesting this site so the robots.txt file will not be used. What's the fix?

We have a website: www.example.com which redirects to https://www.example.com. Therefore when you type www.example.com/robots.txt you are redirected to https://www.example.com/robots.txt. Another ...
1
vote
2answers
218 views

Google not index my hosted-blog after adding custom domain

I started a blog 1 year ago. And my blog was appearing in Google search results. But, some days ago, I set up a custom domain with GoDaddy and submitted my new site and a fresh sitemap to Google ...
4
votes
2answers
52 views

Possible to check which links in a sitemap are indexed and which ones aren't?

Is it possible to check which links of the sitemap I've submitted are being indexed, and which ones are not? I've submitted my sitemap.xml without errors. When I check the sitemap in google search ...
2
votes
0answers
44 views

Google Webmaster Tools reports URLs blocked by robots.txt on my AVS tube website

I added a tube (with AVS script) in Google Webmaster Tools, but the issue is with the robots.txt file which is blocking everything. Here is my AVS tube robots.txt: User-agent: * Allow: / Disallow: ...
4
votes
1answer
49 views

Moving folder's content to a new domain, what SEO steps to take?

So, we have a functionality on example.com/folder that we decided to make as separate site and we're in need of moving it to a new domain (brand it)... This is what I'm going to make SEO wise ...
2
votes
1answer
16 views

Lower priority for specific pages

I have some documents that are not as important and I would like to have a lower priority for them. If I generate a sitemap I can set the priority for these pages to a lower value, e.g. 0.1. ...
1
vote
1answer
32 views

Robots.txt disallow all except one directory and access to sitemap

robots.txt containts following instructions: User-agent: * Allow: /public/ Disallow: / Sitemap: http://sitedomain.com/sitemapindex.xml it seems that in this case sitemapindex won't be accessible ...
0
votes
2answers
38 views

Sitemap XML title in search results

I have a website that uses an XML sitemap and a robots.txt file. If I do a Google search of: site: mywebsite.co.uk "sitemap" I see the usual Google response, similar to the following image, ...
7
votes
2answers
999 views

Are there specific advantages or disadvantages of an XML sitemap over a TXT based sitemap?

Are there any specific advantages or disadvantages in generating an XML based sitemap in place of a simple txt based sitemap (list of URLs)? I realize that in the XML format I can set priority and ...
2
votes
2answers
239 views

When the page language can be determined by the browser settings, which URLs should go into sitemaps?

At this moment our website both uses the language in the url like "en" and localizes the url itself ("books" in english and "boeken" in dutch). Because of the history of making our website ...
0
votes
1answer
60 views

SEO strategy for dynamic images ( PHP + GD )

I have several images which are generated by using PHP + GD. They consist of a background image, and colored lines drawn on top of the images. As the possible combinations depend on the behaviour of ...