93
A Sitemap file helps search engines to discover new and updated URLs on your website. In particular, if your website is fairly large, then this can help them to be able to focus on the new & updated content, instead of having to blindly crawl through everything to see if anything has changed. That can result in new content being found much faster, which ...
28
You have a misconception of what a sitemap is.
The sitemap is used to audit the site crawl by the search engine bot. The sitemap and crawling a site are two different and independent things. Google will continue to crawl your site independently of any sitemap. The sitemap will be used to audit/see if Google is able to properly crawl your site. For example, ...
18
Google does index XML sitemaps (like any XML file). If Google is aware of a URL and it returns a valid response then it's going to pass Google's inclusion rules and could get indexed. Personally, I only submit the sitemap through GWT and include a Sitemap: reference in robots.txt and this is certainly enough to get it indexed.
The recommended method to ...
18
Closetnoc is correct about sitemaps. Don't expect them to limit what URLs Google will crawl and index. In fact sitemaps have little to no influence over SEO. See The Sitemap Paradox
Google won't complain about errors from your old URLs if you redirect them. When you change your site's URL structure it is best to redirect all the old URLs to their ...
answered Oct 25 '19 at 1:27
14
Yes and No. From the sitemap.org official site:-
The location of a Sitemap file determines the set of URLs that can be
included in that Sitemap. A Sitemap file located at
http://example.com/catalog/sitemap.xml can include any URLs starting
with http://example.com/catalog/ but can not include URLs starting
with http://example.com/images/.
So yes, ...
14
In your header you have a canonical link (on line 11, just under <title>).
It looks like this on your page:
<link href="http://escene.ir/component/products/?task=view.12" rel="canonical" />
This element tells Google your preferred URL for a page which has several urls to choose from. This is to prevent you from being penalized for having ...
11
I'm not sure if this changed over the years since this was asked; while in theory you can (as the first answer states), in practice Google at least will give you an error (as seen in their Webmaster Tools):
Incorrect Sitemap index format: Nested Sitemap indexes
The Google help page further states:
A sitemap index file can't list other sitemap index ...
11
We now utilize a crawling method of adding new content to our search index. The subdomain you refer to was just retired over the weekend, and we'll be updating that FAQ page very shortly.
Best regards,
Eric from Ask.com
11
Just checking sitemap.xml is not enough. There are several reasons that that check might fail even though the site has a sitemap.
Did you check sitemap.xml.gz? Google supports gzip of sitemaps. Large sites with large sitemaps are likely to take advantage of this feature.
You can specify the name of the sitemap file in robots.txt. It doesn't have to be ...
answered Jun 4 '13 at 0:20
11
Generally, you have to use a separate sitemap for each host (i.e., different protocol, domain, or subdomain):
From the FAQ "Where do I place my Sitemap?":
All URLs listed in the Sitemap must reside on the same host as the Sitemap. For instance, if the Sitemap is located at http://www.example.com/sitemap.xml, it can't include URLs from http://subdomain....
11
You can make any file dynamic. The best way to do so is not through redirects, but through rewrite rules.
RewriteRule ^robots\.txt$ /robots.php [L]
That way, you power it with a dynamic script, but the URL doesn't change. Most crawlers (including Googlebot) will follow redirects for robots.txt, but some crawlers will get confused if you introduce ...
answered Oct 22 '15 at 9:23
10
If you want to submit your sitemap without a Google Webmaster Tool account, place it in your robots.txt file.
Source
You can specify the location of the Sitemap using a robots.txt file.
To do this, simply add the following line including the full URL to
the sitemap:
Sitemap: http://www.example.com/sitemap.xml
This directive is independent of ...
10
Yes, you can do this, just make sure you specify the file's location in your robots.txt file.
I've had to do this for a site where I had limited access to the file structure, so I know it works - looks like other people have tried too.
10
You can have multiple sitemaps per website, and this is a great example of when that makes sense.
You should make sure you have a Sitemap Index listing each of your sitemaps. It will probably look something like:
<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<sitemap>
...
9
In XML the ampersand has to be escaped, so change it to: &
8
It is not possible to define any hierarchal structure in your XML sitemap. The XML sitemap is a straight forward list of all your pages. Any hierarchal structure to your pages will be determined by Google when it crawls your site.
The hierarchal structure is more relevant to your users. So, your HTML sitemap (if you have one) could be defined in this way. ...
8
First you should address how search engines crawl your site and avoid duplicate content issues by making sure you have the www subdomain (www.example.com) redirecting to your root domain (example.com).
The most efficient way to do this is to create a DNS record - see the first example here on how. You should also use your web server's configuration to ...
8
MrWhite's answer about using X-Robots-Tag appears to be the correct way to do this.
Here is code that can be used in .htaccess or Apache configuration files to do so. (Reference: WebmasterWorld - Sitemaps showing up in SERP - How to prevent this?)
<Files ~ "sitemap.*\.xml(\.gz)?$">
Header append X-Robots-Tag "noindex"
</Files>
Under nginx ...
answered May 21 '14 at 17:10
7
Google's Search Engine Optimization Starter Guide recommends supplying an image sitemap
There's your answer.
Sitemaps help supply search engines with a quick and easy reference to the content on your website. The more information you can give search engines about your site, the better.
Also read http://support.google.com/webmasters/bin/answer.py?hl=en&...
7
Google periodically checks your submitted sitemap.xml file for updates. You only need to submit it once.
See: How often does GWT check dynamic sitemaps?
answered Jul 26 '13 at 21:41
7
The purpose of sitemaps is to tell the search engines about the pages in your website that you want them to crawl and index. If new pages are added to your site that you want crawled and indexed then they should be added to your sitemap. If this is occurring daily then you can add them daily. If this is occurring weekly then you can add them weekly. Search ...
7
The sitemap should reflect the way pages should be accessed, if you want them to be accessed by HTTPS, then yes, that is what you have to do. Otherwise, you are just making the server work a bit extra each time it's crawled.
7
We have another question here that ask why items in the sitemap are not ranked better: The Sitemap Paradox. Google's John Mueller has this to say about common SEO misconceptions regarding sitemaps:
The Sitemap file isn't meant to "fix" crawlability issues. If your site can't be crawled, fix that first.
We don't use Sitemap files for ranking.
...
answered May 29 '14 at 16:23
7
Google will never index HTTPS while the canonicals point to HTTP. I switched my largest site over to HTTPS using the following protocol:
Enabled HTTPS for the site without switching canonicals for about two years. During this time period Google sent all traffic to HTTP.
Switched the canonical version to HTTPS. It has been running that way for about 8 ...
answered Oct 16 '17 at 11:33
7
That isn't possible. You need to map your old URLs to the new with redirects for SEO and user experience.
Google never forgets about old URLs, even after a decade. When you migrate to a new CMS, you need to implement the page level redirects
If there is no equivalent for some particular page you can let it 404 and Google will remove it from the index. ...
answered Jan 22 '19 at 13:52
6
According to the "spec" that w3d mentions, sitemap files cannot reference URLs in alternate directories.
However, this is simply not true for all the search engines. I have put sitemap files in a subdirectory and never had a single error in Google or Bing Webmaster Tools.
So in practice it makes no difference where you put sitemaps.
6
This post is 4 years old, hope the status in not Pending anymore. But this helped me.
After 14 days of pending status, I found this link, to PING bing with your sitemap. It's was a kind of awake call for my sitemap. Few hours later, my site was indexed.
Upload your sitemap (but you already did hence the pending status)
Typ in your browser: http://www.bing....
6
If you're not getting any errors then you can assume Google has parsed it and is aware of the contents. But that doesn't mean they will crawl and/or index those pages. Sitemaps are just another way to tell search engines about your pages. They are not obligated to crawl and index any or all of those pages. The same applies to them finding pages through links ...
6
Your resident SEO expert has valid points, but they're all circumstantial.
Decreased keyword density for the words being targeted in the URL, the
longer the URL the less emphasis is being placed on the keywords being
targeted in the URL
This is an important factor if you have a url like mysite.example/solutions/healthcare/benefits/etc/etc/. But just ...
6
Edit: Google will recrawl it (see comment by John Mueller from Google below).
However, if you want your sitemap reprocessed more quickly by Google, the recommended practice is to resubmit it.
You can resubmit using Webmaster Tools or using an HTTP request.
Using Webmaster Tools:
On the Webmaster Tools Home page, click the site you want.
Under ...
Only top voted, non community-wiki answers of a minimum length are eligible
