Tag Info

Hot answers tagged

14

In your header you have a canonical link (on line 11, just under <title>). It looks like this on your page: <link href="http://escene.ir/component/products/?task=view.12" rel="canonical" /> This element tells Google your preferred URL for a page which has several urls to choose from. This is to prevent you from being penalized for having ...


8

Google does appear to index XML sitemaps (like any XML file). It seems that if Google is aware of a URL and it returns a valid response then it's likely to get indexed. Personally, I only submit the sitemap through GWT and include a Sitemap: reference in robots.txt and this is certainly enough to get it indexed. I believe the recommended method to prevent ...


6

The sitemap should reflect the way pages should be accessed, if you want them to be accessed by HTTPS, then yes, that is what you have to do. Otherwise, you are just making the server work a bit extra each time it's crawled.


6

We have another question here that ask why items in the sitemap are not ranked better: The Sitemap Paradox. Google's John Mueller has this to say about common SEO misconceptions regarding sitemaps: The Sitemap file isn't meant to "fix" crawlability issues. If your site can't be crawled, fix that first. We don't use Sitemap files for ranking. ...


6

Most likely some part of your web site generated links like that, and that is how Google started to crawl the URLs. You should check the links in your web pages to see where these incorrect URLs are, and you should fix them. Also, you could change your Apache configuration so that requests for any other virtualhost than example.com or www.example.com would ...


6

You can have multiple sitemaps per website, and this is a great example of when that makes sense. You should make sure you have a Sitemap Index listing each of your sitemaps. It will probably look something like: <?xml version="1.0" encoding="UTF-8"?> <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <sitemap> ...


5

Sitemaps serve to include, not limit the content Google indexes. If you want to exclude some files, use a robots.txt file as mentioned, or setup redirects. The reason this URL is included is likely that Google found a link pointing to it somewhere else. It could be on your site (which you can fix) or on a third-party site as incoming link. To figure that ...


4

http://zhanzhang.baidu.com/sitemap/index is the (sparse) documentation about sitemaps for Baidu. It only describes that it may help to submit your sitemap, and it shows a screenshot of their webmaster tool, listing several sitemaps and their status. However, it seems that they intend this feature only for "high quality" sites, on invitation basis. Here is ...


4

Yeah. I have seen this too. You are not alone. I don't see any reason to do this. In fact it is redundant isn't it? And since sitemaps are designed to inform search engine about resources to be indexed and sitemaps are not indexed, it seems rather pointless. It was always a silly thing to do. You will see silly things all over the place especially in ...


4

Those pages are not high quality pages. Who would want to land on page five of unanswered questions on this site? Nobody. These are low quality pages because: They only a list made of of content found elsewhere on your site. If a user does land on this type of page, they have to click to the content. Even if a user does want a list of this type of ...


4

Google looks at things differently than we do. Sorry. That is just the way it is. Sometimes you have to see things from Google's perspective before the data makes sense. The Google Webmaster Tools data lags behind a couple of days and some elements a bit more. As far as the various Index Counts, this explains why the difference. However, there is no direct ...


3

According to Google, you should list your AJAX URLs in a sitemap exactly as you say you've done: "4. Consider updating your Sitemap to list the new AJAX URLs Crawlers use Sitemaps to complement their discovery crawl. Your Sitemap should include the version of your URLs that you'd prefer to have displayed in search results, so in most cases it would ...


3

You need to put a robots.txt in the root web directory of your subdomain, just like you do for your main domain. You have to think of the subdomain as a separate website in this case.


3

If you have two verified sites (verified in Webmaster Tools) means you can have a single sitemap. If you have multiple websites, you can simplify the process of creating and submitting Sitemaps by creating one or more Sitemaps that includes URLs for all your verified sites, and saving the Sitemap(s) to a single location. All sites must ...


3

You don't have to use GWT to make Google aware of an XML sitemap. It's just one of the ways they make available to webmasters but isn't the only way or a required way. You can use robots.txt to accomplish this as well.


3

You should include to your sitemap all the URLs you want to see in a search engine's index.


3

Sitemaps are for telling search engines where to find your content, not your sitemaps. So you should not include your sitemap URL in your sitemap. (It would be redundant anyway).


3

Baidu has a Webmaster Tools application that is written in Chinese. But all is not lost. Here is a tutorial: http://www.webnots.com/baidu-webmaster-tools.html The tutorial is written in English and translates the options for you. I am not sure how up to date this is, but it does look complete otherwise and may be useful. I do not see that Baidu reads the ...


3

w3d's answer about using X-Robots-Tag appears to be the correct way to do this. Here is code that can be used in .htaccess or Apache configuration files to do so: <Files ~ "sitemap.*\.xml(\.gz)?$"> Header append X-Robots-Tag "noindex" </Files> Reference: WebmasterWorld - Sitemaps showing up in SERP - How to prevent this?


3

Ok. Found the answer at: Help Google serve the correct language to your visitors We need to have a url tag for each of the url and specify the others as alternate urls.


3

do you have a google webmaster tools account? if you create a free account with them and verify that you are the actual site owner then google will allow you to request for removal of a folder or specific urls. my personal experience is that search engines take the liberty of not following instructions but this step would at least remove your pages from ...


3

Here is Yahoo's documentation for their sitemap ping. At the top there is a prominent notification that Yahoo webmaster tools have moved to Bing webmaster tools. That is why the ping times out. It is no longer supported and you should use Bing webmaster tools instead. Here are the instructions for submitting sitemaps to Bing. It includes the ping ...


3

My answers: Search engines bots are web crawlers (like your web browser in some way) and when a 301 redirect is thrown, they are notified. Therefore, they follow the new redirect to reach the new page and know the PageRank must be transferred. If the 301 redirect is up, search engines don't crawl a.html anymore, that's why after a long time, they remove ...


3

No, you don't need to submit a sitemap for the version (here with www) you don't want to use. Remember submitting a sitemap to your Google Webmaster Tools account helps indexing of the site. Therefore, you submit a sitemap only for a site you want to see in Google's index.


3

Google does not treat CSS content the same as that on page Generally Google will only attempt to index content that is actually embedded within the page content associated with an appropriate tag such as <img>. You can however attempt to force Google's hand by adding the path of the background image into a image sitemap. Some Schema markups require ...


2

Just use Sitemap index files: You can provide multiple Sitemap files, but each Sitemap file that you provide must have no more than 50,000 URLs and must be no larger than 10MB (10,485,760 bytes). If you would like, you may compress your Sitemap files using gzip to reduce your bandwidth requirement; however the sitemap file once uncompressed must ...


2

Links that will ordinarily be crawled when your site is crawled, like those in a top navigation or footer, don't necessarily need to be included in a sitemap. As specified by Google: Sitemaps are a way to tell Google about pages on your site we might not otherwise discover...Creating and submitting a Sitemap helps make sure that Google knows about ...


2

No, the sitemap is not cached. consider the file you upload as a config file that a program/system is going to use. The process of recrawling your site may take some time, and adjusting the results according to the new sitemap may take some time, as mentioned, but the file itself is not cached.


2

Search engines recommend your URL's to be no more than 67 characters in length, so you wouldn't want to get carried away with a large tree of categories. Sitemaps provide a weighting to the search engines with regard to the importance/priority you place on a page. Typically the further down the tree, the lower the priority. If all your website's pages were ...


2

If you want to remove the xml sitemaps as file from the Google search pages, you have to use the noindex header. For example, if you create the sitemaps with php, use the following code: header('X-Robots-Tag: noindex', true); If your static xml files, you can send the header from the .htaccess file <Files ~ "\.xml$"> Header set X-Robots-Tag ...



Only top voted, non community-wiki answers of a minimum length are eligible