Hot answers tagged sitemap
72
A Sitemap file helps search engines to discover new and updated URLs on your website. In particular, if your website is fairly large, then this can help them to be able to focus on the new & updated content, instead of having to blindly crawl through everything to see if anything has changed. That can result in new content being found much faster, which ...
6
If you're not getting any errors then you can assume Google has parsed it and is aware of the contents. But that doesn't mean they will crawl and/or index those pages. Sitemaps are just another way to tell search engines about your pages. They are not obligated to crawl and index any or all of those pages. The same applies to them finding pages through links ...
5
It is not possible to define any hierarchal structure in your XML sitemap. The XML sitemap is a straight forward list of all your pages. Any hierarchal structure to your pages will be determined by Google when it crawls your site.
The hierarchal structure is more relevant to your users. So, your HTML sitemap (if you have one) could be defined in this way. ...
5
Edit: Google will recrawl it (see comment by John Mueller from Google below).
However, if you want your sitemap reprocessed more quickly by Google, the recommended practice is to resubmit it.
You can resubmit using Webmaster Tools or using an HTTP request.
Using Webmaster Tools:
On the Webmaster Tools Home page, click the site you want.
Under ...
4
What you're looking for is at Creating Sitemaps(via your first link):
A Sitemap file can contain no more than 50,000 URLs and must be no larger than 50MB when uncompressed. If your Sitemap is larger than this, break it into several smaller Sitemaps.
...among a bunch of other stuff, but that covers your example above.
4
That's going to block your entire website from being crawled.
No
There is no such thing as securing your robots.txt. If you don't want to keep visitors out of your directory root you need to prevent that using more secure means. Putting a blank index.html file will easily do the trick. If you're running Apache you can also do it easily using htaccess.
4
Your resident SEO expert has valid points, but they're all circumstantial.
Decreased keyword density for the words being targeted in the URL, the
longer the URL the less emphasis is being placed on the keywords being
targeted in the URL
This is an important factor if you have a url like mysite.com/solutions/healthcare/benefits/etc/etc/. But just ...
4
You have 1 page. All the content is loaded as the page loads and all the content is visible. So, there is only 1 page and 1 canonical URL that identifies that page. (This is not an AJAX loaded page where sections are loaded later on request.)
Search engines (ie. Google) index pages, not parts of pages. So, I can't see as there is any benefit (from an SEO ...
3
Sitemap
Sitemaps that you submit to Google are usually in XML format:
See http://www.sitemaps.org/ for details.
They often have a filename of the form: "Sitemap.xml"
And look something like this (source: http://www.sitemaps.org/protocol.html):
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
...
3
If the page is crawlable, then it will be indexed unless you have explicitly marked it as noindex. The sitemap.xml is another way for Google to discover pages on your website as Ilmari Karonen mentioned in the answer. The only thing that will happen is that the page will not be found via the sitemap but by other means.
3
No, leaving a URL out of your sitemap will not prevent Google from indexing it.
To quote Google's sitemaps FAQ:
"Sitemaps provide an extra way for us to understand what pages you have on your site, and can help speed up the discovery of new and updated pages on your site. But neither HTML nor XML Sitemaps replace the normal crawling process."
In fact, ...
3
What you must do is split the sitemap, using the siteindex syntax. You probably have different sections/silos on the site, e.g. blog / forum / e-commerce. Split them by sections/silos, then you may even split them again by category. Make the different sitemaps respect a semantic rule.
Google will love it, and you will love it too in WMT. More info on this : ...
3
Checking for the indexing status of individual URLs is generally futile, especially when you have a website that's a bit larger than "10" pages (depending on how much time you have :)).
One thing that might make sense is to split the website into logical chunks, and to create Sitemap files using those chunks. For each Sitemap file, you can see how many ...
3
Unfortunately, the fact you can access the files doesn't guarantee that Google can. Are you doing any user-agent detection or similar that may be interfering? Try downloading a page with the Fetch as Googlebot tool, and see if it encounters any problems – if that works, that suggests anything else should, too.
Depending on your site, there may be a few days ...
3
There is no fundamental disadvantage to text-only sitemaps aside from what you've touched on. Setting the priority and last-modified date can affect the crawl frequency and on a large site with both frequently and infrequently changing content that can be very important.
XML sitemaps can also carry media information (e.g. video sitemap) and if you have ...
3
Google's John Mueller from the sitemap team had this to say about sitemaps when he answered a question on this site about whether sitemaps help for rankings (they don't):
..for really small, static, easily crawlable sites, using Sitemaps may be unnecessary from Google's point of view once the site has been crawled and indexed.
But he also said that ...
3
You can use various sitemaps to divide yours.
In this article, it mentions:
If you do provide multiple Sitemaps, you should then list each Sitemap
file in a Sitemap index file. Sitemap index files may not list more
than 50,000 Sitemaps and must be no larger than 10MB (10,485,760
bytes) and can be compressed. You can have more than one Sitemap ...
2
Ask.com died as a search engine a long time ago. However, you can still submit a website's sitemap to it, if you wish. However, it will have to be done manually. If you'd like to submit your XML sitemap to Ask.com, simply type in the following in your browser's search bar: http://submissions.ask.com/ping?sitemap=http%3A//www.MYSITE.com/sitemap.xml, where ...
2
For Google and Bing you can open a webmaster tools account and tell them where your sitemap is located, this will also give you access to SERPS information, site analytics and the like.
There is certainly no need to ping it to them every day!
By default both search engines look for sitemap.xml in the root, the only occasion when you would really need to ...
2
That doesn't look correct for images being listed in a sitemap, I would use another sitemap generator. The example below is from Google's Webmaster Central blog post about adding images to your sitemap
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
...
2
As you don't need to provide an XML sitemap to be indexed by Google, then I think it's fair to assume that leaving a page off the XML sitemap won't cause it to be exlcuded either.
On the flipside there is not any evidence that XML sitemaps do anything for your Google ranking. It is merely implied that giving Google a helping hand by providing an XML ...
2
The documented requirements for the loc element are:
This URL must begin with the protocol (such as http) and end with a trailing slash, if your web server requires it. This value must be less than 2,048 characters.
As long as your URLs resolve, they should be fine. Neither is "good" or bad; the issue is whether they're valid. Yes to both.
Note this ...
2
You cannot control crawl frequency except to reduce it in Google Webmaster Tools if you feel you are being crawled too much. Otherwise you have no direct control of increasing crawl rate. changefreq is a "guide" to the search engines for how often a page's content changes. But that doesn't affect how often they will crawl it.
Basic rules for anything ...
2
I will reply as an answer because I cant fit everything in a comment:
You could use a few tools, but I will describe the way I am most comfortable with, PHP + cURL + DOMDocument.
First you need to generate the search queries, so probably reading your sitemap.
$url = 'www.google.com/search?q=' . urlencode("site:".$sitemap_url);
(add more parameters to ...
2
the logo must link to the frontpage
AFAIK there is no SEO benefit of doing this, as long as there is some other link elsewhere on the page that links to the homepage.
However, it is common convention to do so and consequently users often expect this. So it can improve useability, but I don't think this has any SEO benefit specifically.
every page ...
2
Sitemap is essentially an unordered set of URLs, each of them with information about priority, last change, and expected frequency of change. We could imagine the priorities as a kind of linear ordering of the URLs, but that would be the only "layout" possible.
If the site contains too many URLs, it can be split into multiple Sitemaps, linked from one ...
2
At least as far as Google is concerned, the sitemap file will be helping… but not as much as actual links in pages would. It's covered in this Q&A video with Matt Cutts. He's talking about public facing site maps, which I imagine isn't an option for 5 million pages, but the principle is the same: crawlable links on web pages.
I'd definitely consider ...
2
You have an old and outdated architecture, so it's pretty hard to make any actionable recommendation. What I would do :
Remove the external sitemap file on hostedsitmaps.com, an external sitemap is not a sign of quality. Host it on your site.
Redirect with 301 example.com to www.example, or the other way. Choice www.example or .example then stick with it. ...
2
Your images are probably not indexed because you block the bot in your robots.txt file to crawl directory /images.
At first glance I would also add a User-agent: * line above the Allow: / line.
Also add the line
Sitemap: http://www.beyondtime.in
to your robots.txt file (to let more search engines detect it).
Double check your webmastertools (under ...
2
I work in multinational SEO pretty much every day. I'd advise this structure:
Use site.com as the "global" portal.
Use subdomains for territories. (i.e. ch.example.com for Switzerland.)
Use folders for languages. (i.e. ch.example.com/de/ for German in Switzerland, ch.example.com/fr/ for French in Switzerland.)
Use domains as "vanity" for marketing ...
Only top voted, non community-wiki answers of a minimum length are eligible
