Hot answers tagged

9

You can make any file dynamic. The best way to do so is not through redirects, but through rewrite rules. RewriteRule ^robots\.txt$ /robots.php [L] That way, you power it with a dynamic script, but the URL doesn't change. Most crawlers (including Googlebot) will follow redirects for robots.txt, but some crawlers will get confused if you introduce ...


5

Yes. Everything you ask for is possible. And here's an example XML sitemap file generated by the Drupal XML sitemap module with a little bit of configuration. Everything is done for you out of the box. http://softkube.com/sitemap.xml If you check the code of the XML file there's a link to an XSL and inside that file you can see the code with references to ...


5

You generally won't want these excess pages in your sitemap. The sitemap should only contain links to URLs you actually indexed & listed by search engines. If you have property/form URLs in your sitemap, you've probably used a 3rd party Sitemap Generator. These just crawl all links on your website indescriminately, whereas something like ...


4

You're better off creating a multilingual sitemap, just to avoid any source of confusion. The format you have shown is correct. In fact, you could even drop the hreflang declarations in your page sections and just use the declaration within the sitemap. The Official Webmaster Tools blog explains the advantages of using the multilingual sitemaps ...


4

Yes, the same way any request can be "dynamic". However, you would not redirect (as in your example code), you should internally rewrite using mod_rewrite. (The same as what Drupal is probably already doing.) For example, in your root .htaccess file: RewriteEngine On RewriteRule ^robots\.txt$ robots.php [L] RewriteEngine should only occur once (although ...


3

This is not an answer, but a caveat for XML sitemap wizard users. I recently did a tech & SEO overview of a site rebuild where an XML sitemap was auto-generated by a wizard. She failed to review it and see that all the URL's were like **new.**site.com/page.asp - the dev server where the site was built! Typing out a txt sitemap has the advantage of ...


3

It seems like you have your HTTP and HTTPS sites redirect to the HTTPS version of your www.example.com. This is good. Just know there is a cost for this initially. In your case, only one sitemap will ever be seen. It is the HTTPS version of your www.example.com site. Any sitemap should reflect both HTTPS and www in the URLs that are listed. Any other URL ...


3

There are four valid addresses...and they all show the same site; they are synonyms (no re-direction). Prior to considering your sitemap, which is not essential for Google to index your site, you should deal with all the duplicate content issues you have going on here. As covered in this Google Webmaster Tools help doc: Duplicate content generally ...


3

There are no SEO benefits to serving a compressed XML sitemap over serving one that is not compressed. The advantage of compression is simply to save bandwidth and the time it takes to download. (If your sitemap is huge.) Note that the limits for the size of the sitemap are the uncompressed size (ie. 50MB uncompressed for Google).


3

First off, you are dealing with a few things in your question. I will try and go through it for you in no particular order. As far as sitemaps go, they have nothing to do with a pages or sites performance generally. It is just a way to tell Google what pages you have. Where sitemaps are necessary is when part of your site cannot be crawled by a search ...


2

I think that what you'll want to do is change the way your system works. If no language is specified in the URL, but you detect the language via the browser settings, then your site should redirect the user to a URL with that language in the path. This is for two reason: 1) Duplicate content. Spiders/Bots will come to test.com/books, test.com/booken, and ...


2

Yes, it's correct to target the countries in the sitemap.xml and point them to the regional url. And yes, you can use the same URL for multiple locations. You might want to simplify it here though, and use /us/ for "en" (all English variations that aren't otherwise specified) and /uk/ for "en-gb" (just English in Great Britain). If possible, use the ...


2

No, the shortened URL should not go in the XML sitemap. The shortened URL presumably redirects to the long (canonical) URL. Only the canonical URL should appear in the XML sitemap. If the canonical URL is an external resource then that wouldn't be in the XML sitemap either. (The XML sitemap identifies resources on your own site that you want to be ...


2

Of course. XML is just XML and can be served as XHTML by using XSL if you wish and styled with CSS.


2

It seems these days that a sitemap in the root named sitemap.xml will get picked up by dang near anyone in time. It all depends on how much control you want. If you do not want just anyone reading your sitemap including scrapper bots, then name it something unique and submit it to Google, Bing, and who ever else you want. Do not put it in your robots.txt ...


2

But going to the Google index->Index status menu it shows 0 page indexed. What is the difference between the sitemap menu and the index status menu? The index status menu lags behind the sitemap menu (about a week, sometimes more), but ultimately, they converge to the same value. If you are in a hurry, then compute the sum from the sitemap menu to have ...


2

Sitemaps have very little effect on SEO. See The Sitemap Paradox. Deleting your old sitemaps shouldn't be an issue. Google will continue to crawl and index the URLs they contain even when the sitemaps are no longer there. In fact, getting Google to forget about URLs is pretty much impossible. If you have new sitemap files that list the URLs you ...


2

Google get tons of request to index many of pages in search result, so simply they ignore many of request, even with high quality of pages that have no reputation. Many of webpages does not have sitemap, but they are indexed on Google search result, just because of do-follow backlinks. For example developer.android.com. there is no any sitemap on root ...


2

Google Sitemap Compression Support It's interesting to read Google's decision to use poorly worded documentation on what sitemaps are or are not supported. Here is a quote from Google Webmaster Tools regarding an error message received when uploading a archived file. Google couldn't process your sitemap because we couldn't uncompress it. This error ...


2

Google will automatically crawl links as it finds them unless it runs into a link labeled as "nofollow" or a page with a robots tag containing "nofollow" in it. If you plan to make a webpage on your site as a sitemap page, then try to make it user friendly by giving some sort of description of what each set of target URLs are about. To make google ...


2

As Closetnoc mentioned in his comment, GZ is just a compressed version of your text sitemap. So my answer to your question Should I include just the .gz file in the robots.txt or both? There is no point in submitting both sitemap.xml as well as sitemap.xml.gz. Most of the search engine bots can read the compressed content. So if you are submitting ...


2

I originally read you question as the number of indexed pages from your sitemap is 1500 and that you are worried. Then I thought about it. Perhaps there are things you need to know. If your site is new, then it will take quite a while for a new site to be indexed. I am not sure how old your site is, but it can take as much as a year for some sites to sink ...


2

You do not want your XML sitemap to include duplicate content. In your case, it sounds like that one product exists at multiple URLs, which is the very definition of duplicate content. So, what I'd say is that you want to have the page mywebsite.com/product1 (the canonical version of the URL) in your XML sitemap, then keep the others out of your XML sitemap. ...


2

The problem is that Google hates pages in the sitemap that do any kind of redirect. So, try something from this few options that crossed my mind: Don't let the page in the sitemap to redirect (the pages must give status code 200 - OK) Change the pages in the sitemap with the landing pages. Exclude redirect pages in sitemap.


2

Please understand that Google will largely ignore sitemaps if a site can be effectively crawled. Google will compare what it can crawl with the sitemap to ensure that it can properly crawl the site. A sitemap is only necessary if a site is extremely large, the content cannot effectively be linked, or if content resides behind a paywall or login. If you ...


2

Google has a help document about sitemaps in which they list the various types: web video image mobile news


2

having a static image's url / path would help your image to appear in image search results. without having an image url it can't be cached properly. yes, i would strongly recommend to use images like images, making use of real image files and implement them with img src alt + title or css.


1

You have to use a separate sitemap for each subdomain. From the FAQ "Where do I place my Sitemap?": All URLs listed in the Sitemap must reside on the same host as the Sitemap. For instance, if the Sitemap is located at http://www.example.com/sitemap.xml, it can't include URLs from http://subdomain.example.com. If the Sitemap is located at ...


1

As per in Matt Cutts blog post, he said: A subdomain can be useful to separate out content that is completely different. Google uses subdomains for distinct products such news.google.com or maps.google.com, for example. This is similar to what Blogger (blogspot) does. John's blog (john.blogspot.com) is totally different from Doe's blog ...


1

Yes but just to add more custimizations like how msny times does it get changed and what is the last update time



Only top voted, non community-wiki answers of a minimum length are eligible