Tag Info

Hot answers tagged

10

Google does appear to index XML sitemaps (like any XML file). It seems that if Google is aware of a URL and it returns a valid response then it's likely to get indexed. Personally, I only submit the sitemap through GWT and include a Sitemap: reference in robots.txt and this is certainly enough to get it indexed. I believe the recommended method to prevent ...


6

Most likely some part of your web site generated links like that, and that is how Google started to crawl the URLs. You should check the links in your web pages to see where these incorrect URLs are, and you should fix them. Also, you could change your Apache configuration so that requests for any other virtualhost than example.com or www.example.com would ...


6

We have another question here that ask why items in the sitemap are not ranked better: The Sitemap Paradox. Google's John Mueller has this to say about common SEO misconceptions regarding sitemaps: The Sitemap file isn't meant to "fix" crawlability issues. If your site can't be crawled, fix that first. We don't use Sitemap files for ranking. ...


6

You can have multiple sitemaps per website, and this is a great example of when that makes sense. You should make sure you have a Sitemap Index listing each of your sitemaps. It will probably look something like: <?xml version="1.0" encoding="UTF-8"?> <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <sitemap> ...


5

Sitemaps serve to include, not limit the content Google indexes. If you want to exclude some files, use a robots.txt file as mentioned, or setup redirects. The reason this URL is included is likely that Google found a link pointing to it somewhere else. It could be on your site (which you can fix) or on a third-party site as incoming link. To figure that ...


4

I'm not sure if this changed over the years since this was asked; while in theory you can (as the first answer states), in practice Google at least will give you an error (as seen in their Webmaster Tools): Incorrect Sitemap index format: Nested Sitemap indexes The Google help page further states: A sitemap index file can't list other sitemap index ...


4

Yeah. I have seen this too. You are not alone. I don't see any reason to do this. In fact it is redundant isn't it? And since sitemaps are designed to inform search engine about resources to be indexed and sitemaps are not indexed, it seems rather pointless. It was always a silly thing to do. You will see silly things all over the place especially in ...


4

Those pages are not high quality pages. Who would want to land on page five of unanswered questions on this site? Nobody. These are low quality pages because: They only a list made of of content found elsewhere on your site. If a user does land on this type of page, they have to click to the content. Even if a user does want a list of this type of ...


4

Google looks at things differently than we do. Sorry. That is just the way it is. Sometimes you have to see things from Google's perspective before the data makes sense. The Google Webmaster Tools data lags behind a couple of days and some elements a bit more. As far as the various Index Counts, this explains why the difference. However, there is no direct ...


3

Ok. Found the answer at: Help Google serve the correct language to your visitors We need to have a url tag for each of the url and specify the others as alternate urls.


3

w3d's answer about using X-Robots-Tag appears to be the correct way to do this. Here is code that can be used in .htaccess or Apache configuration files to do so: <Files ~ "sitemap.*\.xml(\.gz)?$"> Header append X-Robots-Tag "noindex" </Files> Reference: WebmasterWorld - Sitemaps showing up in SERP - How to prevent this?


3

do you have a google webmaster tools account? if you create a free account with them and verify that you are the actual site owner then google will allow you to request for removal of a folder or specific urls. my personal experience is that search engines take the liberty of not following instructions but this step would at least remove your pages from ...


3

Here is Yahoo's documentation for their sitemap ping. At the top there is a prominent notification that Yahoo webmaster tools have moved to Bing webmaster tools. That is why the ping times out. It is no longer supported and you should use Bing webmaster tools instead. Here are the instructions for submitting sitemaps to Bing. It includes the ping ...


3

My answers: Search engines bots are web crawlers (like your web browser in some way) and when a 301 redirect is thrown, they are notified. Therefore, they follow the new redirect to reach the new page and know the PageRank must be transferred. If the 301 redirect is up, search engines don't crawl a.html anymore, that's why after a long time, they remove ...


3

No, you don't need to submit a sitemap for the version (here with www) you don't want to use. Remember submitting a sitemap to your Google Webmaster Tools account helps indexing of the site. Therefore, you submit a sitemap only for a site you want to see in Google's index.


3

Google does not treat CSS content the same as that on page Generally Google will only attempt to index content that is actually embedded within the page content associated with an appropriate tag such as <img>. You can however attempt to force Google's hand by adding the path of the background image into a image sitemap. Some Schema markups require ...


3

There are four valid addresses...and they all show the same site; they are synonyms (no re-direction). Prior to considering your sitemap, which is not essential for Google to index your site, you should deal with all the duplicate content issues you have going on here. As covered in this Google Webmaster Tools help doc: Duplicate content generally ...


2

The question focuses a lot on what Google is doing but to me it appears that your fundamental problem not really Google specific at all. Why do these names, which you clearly don't seem to want people to use, even exist in DNS? If it is intentional that these names exist and resolve, why are you serving your actual site when people (and Googlebot) connect ...


2

Google follows not only links made by other content writers, but it also heuristically interprets your javascript and even tries to "simplify" your URLs to strip them off wrappers, such as /index.php?page=news.php => /news.php! One way would be to ban those mangled URLs in your robots.txt, but that would (1) grow your robots.txt and make it messy, and (2) ...


2

Google rejects the sitemap as broken if it has a & character in an URL. It accepts it when you replace & with &amp; BUT: if you later check the list of crawling errors in the Google webmasters tool, it will report this URL of the sitemap file as broken, because it contains &amp; instead of &. Thus the correct solution is to change the ...


2

Submitting a sitemap isn't really an SEO technique at all... A search engine does not require a sitemap in order to crawl and index pages from a website unless it consists of an extremely complex architecture and deep hierarchical level of content. It can be beneficial to submit sitemaps and for images/videos in the likes of Google Webmaster Tools just so ...


2

It's most likely that Google is disliking the fact that you have the videos on the sub domain fruits.example.com rather than the main domain. If you take a look at Google by doing site:http://www.example.com and click videos your see that the only two videos to be indexed are those actually hosted on the site. Example: ...


2

Images not on your site will not be attributed to your site in Google image search. As well, if Dropbox restricts spidering the images with robots.txt, they will not show up in Google image search at all. There is no way out short of moving the images to your server which is what I recommend if you want them indexed by Google. I do not know what kind of ...


2

Google takes several things in account to determine how relevant a page is. The most important factors are: Page content Domain name URL name Let's assume that your content and your domain name didn't change. Even your HTML markup is still the same. OLD example.com/category/subcategory.html NEW example.com/category/filter/cat/subcategory.html By ...


2

Install the Yoast SEO plugin on both installations and then add the generated sitemaps to your Google Webmasters account. Google Webmasters allows you to submit multiple sitemaps, so it should be pretty straight forward.


2

Yes, sitemap index files can refer to mRSS files, but you can also submit them individually if you want. You can't link to mRSS files from normal (non-index) sitemap files though. In addition to the help center page you mentioned, there's also a bit about the differences between these files in an older blog post.


2

No, you can't submit the exact same sitemap (same syntax, same URLs), and expect Google to sort out what kinds of assets you're trying to identify. If you submit a sitemap that you intend to be for your videos, and it doesn't have the required Video sitemap syntax, and contain the required video-specific tags, Google won't see it as a video sitemap. It's ...


2

Not sure if you saw this. Here is a guide from Google describing tactics for your seperated mobile URL situation: Method for mobile with 2 URL's Basically you set the desktop site as canonical, and mobile as alternate, then use tags to point bots to either-or mode using a preferred hierarchy. Do almost the same for sitemap, only you can use rel and ...


2

Yes that link have nothing to see. But you can use following if you want to remove page from google search : Remove page URL from google


2

Sitemaps have almost no effect on rankings. You don't need a sitemap to have your pages rank well. In fact, it has been suggested that using sitemaps is a red herring. You instead should be spending time making sure your site links to all your content. See The Sitemap Paradox. The only time that I can imagine that submitting fresh sitemaps regularly ...



Only top voted, non community-wiki answers of a minimum length are eligible