Hot answers tagged

28

You have a misconception of what a sitemap is. The sitemap is used to audit the site crawl by the search engine bot. The sitemap and crawling a site are two different and independent things. Google will continue to crawl your site independently of any sitemap. The sitemap will be used to audit/see if Google is able to properly crawl your site. For example, ...


18

Closetnoc is correct about sitemaps. Don't expect them to limit what URLs Google will crawl and index. In fact sitemaps have little to no influence over SEO. See The Sitemap Paradox Google won't complain about errors from your old URLs if you redirect them. When you change your site's URL structure it is best to redirect all the old URLs to their ...


3

The answer is in the protocol you link to: The location of a Sitemap file determines the set of URLs that can be included in that Sitemap. A Sitemap file located at http://example.com/catalog/sitemap.xml can include any URLs starting with http://example.com/catalog/ but can not include URLs starting with http://example.com/images/. There is not ...


3

Your question actually breaks down into a couple categories, each of which has different considerations: XML Sitemap In your XML sitemap, you can specify both change frequency and priority ranking for the various pages. This helps search engines identify how significant a page on the site is and how often it should be crawled for changes. Avoid listing ...


3

It can have any amount of links, limitless, open end. But i very doubt, whether even the tenth link in it would be visited by Googlebot. You can monitor it with GA and/or your log files. Google, just as any human, doesn't like link lists. They are simply not interesting - not relevant to something. Thats why HTML sitemaps are better to omit.


2

Don't use noindex, it's just not necessary. It seems you have a XSL/XSLT for the sitemap, which doesn't help and is the most likely cause for Google adding the sitemaps in its' index by mistake. EDIT: Unless you specifically restrict indexing of your sitemaps, Google will archive them in its' index. However, if you have followed the rules and guidelines, ...


2

You may not be doing anything wrong at all. "General HTTP error" can mean many things. The Internet/Web is not a guaranteed delivery protocol. Things will go wrong from time to time. This is not uncommon even for Google. If you can access the sitemap yourself, then be patient. Many of us have experienced Google having trouble reaching our resources or ...


2

True, the limit to an XML sitemap is 50MB (uncompressed) and 50,000 URLs. This is also the limit for any type of sitemap file, including HTML; any more than that, and the engine will stop crawling any single file. Also, @Evgeniy is correct that HTML sitemaps are pointless; nowhere in SEO literature does it say that Google or any search engine likes these ...


2

Google was not always like that. After some recent updates this crawling and indexing issue is observed by many webmasters. The only thing you can do is create and submit your sitemap, do some manual crawl request and just wait...weeks...a month. Just wait.


2

The links you provide give both the same error Access Denied You don't have permission to access "http://amd.cdn.example.com/adultswim/episodes/us_geo/" on this server. Reference #18.5f221502.1574092192.186833f Try to make your website public before using Sitemap Generators otherwise they are not able to crawl your website.


2

If your sitemap is being submitted in txt format, then it must have a .txt file extension. https://support.google.com/webmasters/answer/183668?hl=en


2

Search engines don't typically check for a sitemap at a particular URL. You need to do the step of informing search engines about your sitemap. The easiest way is to add it to robots.txt. It is just one line and then all search engines will pick it up. The most common location is to have the Sitemap at the root of your site at /sitemap.xml. If your ...


2

It may takes several days before you can see the results of the submission in GCS, but in case there are validating errors it may take MUCH longer if it all. Try checking if you've get any of these errors and if yes, they should be cleaned up first https://validator.w3.org/


2

I suppose you could set the "last-modified" attribute to the date and time in the future when it will be published. You can read up on the supported attributes here. Search engines don't give much weight to sitemap.xml files compared to the actual structure of your site as visible to a human so your time is almost certainly better spent on something else.


2

It isn't clear what the "General HTTP error" is from the screenshot you posted. I can access your site, robots.txt, and sitemap myself. To resolve that problem you should: Click on "1 instance" and try to see what specific error Googlebot is getting. Use "inspect URL" in Google Search Console with a live test to try and reproduce the problem Examine your ...


2

If you are sure that all your internal links and sitemaps are correct, and that you have no red flag about your sitemaps in the Google Search Console, then there is probably, as clued by GeoffAtkins, at least one badly formatted backlink from a third-party website, coded like this: <a href="http://www.example.com/?page=x&amp;amp;item=5">badly ...


1

You should remove all your <lastmod>, <changefreq>, and <priority> data from your sitemaps. Google says they don't use it. It is just extra bloat making your sitemap files larger with no benefit to you. In fact, sitemaps have almost no SEO benefit. See The Sitemap Paradox. Google will crawl pages listed only in a sitemap, but it ...


1

You may not have added the property to Google Search Console exactly as it is on the site. If you create a URL prefix property (as opposed to a domain property), make you sure you create it with your canonical URLs. I see that your redirect from http to http and also that you redirect to add the www with one of your sites. However, the other site doesn'...


1

Both actions: use optimized image names and create image sitemap are recommended in Google image best practices guideline so you should implement both of them. In addition to that, I'll recommend the following: Use a descriptive alt attribute Optimize images Use title and captions Use next generation image formats


1

Just to begin a question: do you have certain Search Console properties for each language? If not - consider to establish. According to my experience it should be enough to have such setup as you have it - one sitemap with all hreflangs. But such setups are pretty error-prone. There are many factors influencing 100% retrieving of sitemap urls - it is not ...


1

One should distinguish: blocked pages vs. blocked ressources. If pages are blocked by robots, the bot can't come in and read. But, if there are links to blocked page, it could nevertheless be indexed. The SERP result of indexed, but blocked page is ugly - no snippet, only alert "the page is blocked". Getting such alert means, the bot tried to visit the ...


1

It looks, like you submitted this html-sitemap to GSC as if it would be your main sitemap - if not main, then at least having equal importance as the xml-sitemap. Thats why GSC alerts you about noindexed url containing in the sitemap. I would inspect your sitemap setup: which sitemap(s) is/are submitted to GSC? Then i would make sure, that the xml-...


1

"Server response check" in Yandex is returning "Document contains no text". if the whole system of sitemap files really works - meaning actually does being read properly by the search engines? Sure: error-free log-files and alert-free GSC/Yandex Webmaster would mean, there is everything ok with sitemaps. Can this issue be related What issue? Do you ...


1

Is the sitemap good for SEO? It is, though its importance has diminished some, given how good the engine crawlers have gotten. I wrote about this for a related question; you can see it here, if you're interested: Can I delete sitemap.xml from my website, Will it have a negative impact? Is the sitemap good for user experience? Not really. It's more a signal ...


1

You must redirect 301 each old url to its new value. For example If using IIS 7 or higher, the simplest solution is to use the httpRedirect element in your web.config. <httpRedirect enabled="true" exactDestination="true" httpResponseStatus="Permanent"> <add wildcard="/browse/1" destination="/browse/fashion" /> <add wildcard="/browse/...


1

Use one of these links. http://www.bing.com/webmaster/ping.aspx?siteMap={full url to your sitemap) or http://www.bing.com/ping?sitemap=[your sitemap web address] This will give Big a nudge on your sitemap and there is a high chance it will get going within a few hours or a couple of days. I worked well for me


1

I would choose the hierarchical sitemap. If the user can directly click from the homepage the shoes section under the women category, the URL should still reflect that by being example.com/fashion/women/shoes. This is a distinction you have made yourself since you list shoes under the women section and this section is listed under the fashion section. URLs ...


1

I see much more results: Thats because urls are indexed, which shouldn't be indexed. Your collections seem to be redundant - your products seem to appear in multiple collections. I would deindex collections at all. Second thing: i would strongly avoid such construction as you build with sitemaps. Your site is too small - it doesn't need index sitemap and ...


1

It could be that Google was either caching the results for you or the indexing wasn't finished yet as I am able to see your different collections on google:


1

On the Webmaster tools website, you should: Delete the old site (without http) Add the new site (with https) Recheck every configuration Send your new sitemap (it should be generating HTTPS urls at this point). However, prior to doing all this, you should do the following on your site: Make sure every page is redirecting correctly using a 301 code from ...


Only top voted, non community-wiki answers of a minimum length are eligible