Informing search engine crawlers
Once you have created the Sitemap file and placed it on your
webserver, you need to inform the search engines that support this
protocol of its location. You can do this by:
- submitting it to them via the search engine's submission interface
- specifying the location in your site's robots.txt file
- sending an HTTP request
The search engines can then retrieve your Sitemap and make the URLs
available to their crawlers.
Submitting your Sitemap via the search engine's submission interface
To submit your Sitemap directly to a search engine, which will enable
you to receive status information and any processing errors, refer to
each search engine's documentation.
Specifying the Sitemap location in your robots.txt file
You can specify the location of the Sitemap using a robots.txt file.
To do this, simply add the following line including the full URL to
This directive is independent of the user-agent line, so it doesn't
matter where you place it in your file. If you have a Sitemap index
file, you can include the location of just that file. You don't need
to list each individual Sitemap listed in the index file.
You can specify more than one Sitemap file per robots.txt file.
Submitting your Sitemap via an HTTP request
To submit your Sitemap using an HTTP request (replace
with the URL provided by the search engine), issue
your request to the following URL:
For example, if your Sitemap is located at
http://www.example.com/sitemap.gz, your URL will become:
URL encode everything after the /ping?sitemap=:
You can issue the HTTP request using wget, curl, or another mechanism
of your choosing. A successful request will return an HTTP 200
response code; if you receive a different response, you should
resubmit your request. The HTTP 200 response code only indicates that
the search engine has received your Sitemap, not that the Sitemap
itself or the URLs contained in it were valid. An easy way to do this
is to set up an automated job to generate and submit Sitemaps on a
Note: If you are providing a Sitemap index file, you only need to
issue one HTTP request that includes the location of the Sitemap index
file; you do not need to issue individual requests for each Sitemap
listed in the index.