A site map (or sitemap) is a list of pages of a web site accessible to crawlers or users.

learn more… | top users | synonyms

1
vote
2answers
316 views

How can I allow robots access to my sitemap, but prevent casual users from accessing it?

I am storing my sitemaps in my web folder. I want web crawlers (Googlebot etc) to be able to access the file, but I don't necessarily want all and sundry to have access to it. For example, this site ...
1
vote
1answer
28 views

Can I submit duplicate image, video, and content sitemaps to Google Webmaster Tools?

Can I submit 3 duplicate sitemaps (image, video, content) which have same URLs to Google Webmaster Tools? Image Sitemap 1: <loc>http://mywebsite.com/article1</loc> <video:video> ...
1
vote
1answer
26 views

Can I use sitemap index for media RSS (mRrss) files?

I have a lot of mRSS (media Rss) files. I plan to create sitemap index to contain them. Are sitemaps allowed to contain mRss files? To my knowledge, they are only allowed to contain image, video, ...
1
vote
1answer
79 views

changing URL format afftects SEO reputation and not considering 301 redirection

We had URL http://www.example.com/category/subcategory.html which changed to http://www.example.com/category/filter/cat/subcategory.html We did 301 redirection, updated canonical URL to new one also ...
1
vote
2answers
116 views

Need to encourage google to refresh robots.txt [duplicate]

My old robots.txt is User-agent: * Disallow: / Is blocking me for uploading a new sitemap and also blocking me for fetching robots.txt manually. I don't know what to do.
1
vote
2answers
58 views

HTML Sitemap URL format

When adding a html sitemap to a site, for SEO purposes should the links it contains be locally referenced or is it better to use the full URL? Eg: <ul> <li><a ...
1
vote
1answer
70 views

Why I see my page sitemap XML link in Google?

I am using Yoast SEO plugin for my WordPress website. I am also using Yoast's XML Sitemap. Today I checked my pages in Google and found this indexed by Google: Title: http://example.com/ ...
1
vote
1answer
256 views

Flat vs. Hierarchical Tree Structure (vol. 2)

This relevant question deals with whether a site URL structure is better (SEO-wise) to be flat or hierarchical. How much important is it for a site ranking if the overall structure is flat or ...
1
vote
3answers
73 views

Is it necessary to link to all landing pages from the homepage?

What I am wanting to do is build quite a few landing pages targeting specific keywords from google search. These pages will give a brief explanation of what we do and how we can help them with what ...
1
vote
1answer
53 views

If the size of sitemap is limited: how do I tell to Google all of my URLs?

I heard that the sitemap for Google can not exceed 10 Mb and 50 kURLs. This is not enough for me. What shell I do to pass through this obstacle? Create sitemap for each category?
1
vote
2answers
78 views

Should pages linked to in my nav and footer be included in my sitemap?

I'm more of a back end developer and can't answer this question from my own research. When setting up the links in my sitemap, do I need to include the pages that I link to in my top navigation and ...
1
vote
2answers
58 views

Do search engines rank URLs in sitemaps lower if there are no internal links pointing to them?

I have a site with thousands of pages. The pages are dedicated to be found by the search engines as they are very specific. It is very unlikely that the users will find necessary pages using any ...
1
vote
1answer
1k views

Sitemap for a one-page site

I'm wondering whether it's worth the effort to create a sitemap for a single-page site. By single page I'm not talking about a webapp made with Angular or Backbone, just a presentation site/page ...
1
vote
1answer
184 views

Creating sitemap for Googlebot - how to mark dynamic content / dynamic subpages?

I have a website that is an Internet forum. This forum has many categories, and a single category page that contains a lot of subpages with listed threads. This Internet forum is brand new, and about ...
1
vote
1answer
114 views

Server gzip compression vs gzipped sitemap

My client's SEO guy said that search engines prefer gzipped sitemaps. They keep their sitemap in .xml.gz file. I was wondering if there is any difference for search engines between a gzipped ...
1
vote
1answer
43 views

Should I leave old events in my sitemap?

Should I leave old events in my sitemap that have passed, or remove them? The sitemap is quite huge and I didn't know if that was OK.
1
vote
3answers
3k views

Sitemap submission : Do sitemaps need to be resubmitted when they are updated?

In short: Should I resubmit my sitemap through Google Webmasters Tools every time I replace a sitemap.xml with an existing sitemap.xml (i.e., www.domain.com/sitemap.xml)? Or does it check it ...
1
vote
1answer
76 views

Should ecommerce checkout pages be included in sitemap.xml?

My ecommerce site has a variety of pages specific to the checkout-out process. These include "Cart", "Billing", "Shipping", "Discount", "Finish", and "Complete Order". In general, should I be ...
1
vote
1answer
79 views

Why is this date in my sitemap “invalid” according to Google

My XML sitemap file has <lastmod>2013-07-06T09:39:51-0400</lastmod> formatted with W3C Datetime Format. Google has a warning message in webmaster tools about this for me: Google's ...
1
vote
2answers
70 views

Which pages should be present in the sitemap

So if I have a WordPress blog for example, then out of the following which should be present in the sitemap and which should not be? posts - these will be obviously. pages - should be. categories - ...
1
vote
3answers
85 views

SEO sitemap submission

I always assumed SEO was simply a buzzword that meant "using content that makes sense". Obviously it gets a little bit more complicated than that, using realistic search terms in header tags and ...
1
vote
1answer
96 views

browse-able hierarchy vs. sitemap

The current site navigation is done through a search form. UX wise, I feel it's the best solution we can currently offer our uses. Because this is the case, I have a sitemap that covers all the pages ...
1
vote
2answers
96 views

sitemap for non-CMS based website

I've just finished a new version of my website but now I don't know how to generate a correct sitemap. Now the website is based on PHP files and folder and in blog folder there is Wordpress. This is ...
1
vote
2answers
664 views

Why does google crawl my page only once even I set changefreq:daily?

I have built a new site, submitted my homepage URL to Google, and my sitemap(I have about 6,300 pages.). Now when I type site:mydomain.com, there have 1,992 pages have crawled. But they were all ...
1
vote
2answers
325 views

Creating Google sitemap.xml , is it okay for the images to be wrapped in URL tags?

I'm using a tool to generate the sitemap.xml file for me, it started to crawl my website, got the pages and all images, but when exporting it, I review the XML (to make sure nothing is wrong) and I ...
1
vote
1answer
144 views

article url format for google news sitemap

I have 2 questions about url for google news sitemap. Is it good that I use= url for sitemap witch like ...
1
vote
2answers
124 views

Robots.txt and sitemaps

If i where to block a web page, say 404Error.aspx, in robots,txt (i see no reason for this page to be indexed by google). However, do i still include this page in the xml sitemap or not? Edit: If i ...
1
vote
1answer
566 views

ASP.NET MVC SEO friendly URLs and Sitemap

I've created an ASP.NET MVC 3 application and generated sitemap.xml using some free online tools. It contains the following URLs: http://www.mydomain/ http://www.mydomain/About ...
1
vote
1answer
4k views

How to fix Google 404 not found Crawl Errors?

I was checking on Google webmater tool for my blog site to see if there's any indication on why my blog traffic decreased to half in one day and i saw 43 Not Found crawl errors and 5 in Sitemap Not ...
1
vote
1answer
227 views

Is this Sitemap code correct? [closed]

I'd like to check whether I have any of the code not as it should be in my Sitemap file. Not sure if this is too general a question for here... If it's an ok question, here's the sitemap URL: ...
1
vote
1answer
191 views

I want the links in my HTML sitemap to be crawled but I do not want the sitemap in the SERPs

I have a website that has some pages and then it has a folder that has the majority of the pages. All the pages that exist in this folder could only be accessed by using my website search engine, then ...
1
vote
1answer
112 views

Sitemap file entries for blog and main php site

I'm creating a sitemap manually. It's a PHP site with a WordPress blog. Here's the main site: http://wordfruit.com And here's the blog: http://blog.wordfruit.com Can I put the blog URL in the ...
1
vote
1answer
443 views

Problem with Google indexing URLs in my XML SiteMap

My XML sitemap has a lot of links - for example: nanoc.gigfa.com/post.php?id=1 nanoc.gigfa.com/post.php?id=2 nanoc.gigfa.com/post.php?id=3 nanoc.gigfa.com/post.php?id=4 ...
1
vote
2answers
335 views

When does Google Webmaster Tools re-evaluate re-submitted sitemaps?

When I alter and re-submit a sitemap does GWTs immediately re-evaluate it or do I have to wait 'til GoogleBot crawls our site again? I just re-submitted a problematic sitemap and it now displays ...
1
vote
3answers
30 views

Can canonical tags be placed in the sitemap rather than in each page?

We have canonical tags spread across our web pages. Is it possible to add the canonical tags in our sitemap instead? If yes, how?
1
vote
1answer
60 views

Submitting an Index Sitemap to Bing when the individual sitemaps are hosted on AWS

Been working on the site map recently and now I want to submit it on Bing Webmaster tools. The main site map includes several others, each one for a different country. Like this: sitemapindex ...
1
vote
1answer
29 views

Is it possible to include news into Google News if there is a non-news content under the same section?

We want to include our news into Google News, and now performing research how to properly do this. At the moment we have the same base URL for news and non-news articles (analysis, interviews, etc.), ...
1
vote
1answer
77 views

How to instruct Google to crawl old, non-linked pages?

I am working on a website for a client who wants to launch with at least 2,000 articles, where in terms of links not all 2,000 articles will of course be available on the homepage or elsewhere. But, ...
1
vote
1answer
53 views

Which search engines prefer an HTML sitemap over an XML sitemap?

Is there a search engine out there that prefers a full HTML sitemap over an XML one? On a site with thousands of articles, does it make sense to include a page that has links to every single page?
1
vote
1answer
123 views

What is the meaning of this sitemap error in Google Webmaster Tools?

In my Google Webmaster Tools, I received an error which I am not understanding: When we tested a sample of the URLs from your Sitemap, we found that some of the URLs were unreachable. Please check ...
1
vote
1answer
57 views

Can I submit multiple domain sitemaps in Google Webmaster Tools?

My site is setup in Blogger with a custom domain name. Currently, url.blogspot.com is 301 redirected/forwarded to www.url.mydomainname.com. The naked domain mydomainname.com is returning a 404 status ...
1
vote
1answer
49 views

Google not indexing new forum

We installed a new forum a few months ago now. The URL is: https://www.scirra.com/forum I've 301'd the old topics/threads, as well as included all the new URLs in the sitemap. Yet they still are ...
1
vote
1answer
95 views

Creating more than one sitemap and relating it to a subdomain - will I be penalized?

I'm thinking in creating a subdomain containing all the reviews from my products, for example: http://reviews.mystore.com At this subdomain, I have my sitemap.xml pointing for the reviews page about ...
1
vote
1answer
155 views

Is it a problem for Google if robots.txt blocks URLs in the sitemap?

I had submitted my site's URL to Google Webmaster Tools. I am using OpenCart. I also have robots.txt file which is accessible to Googlebot. Now issue I am facing: Googlebot is blocked from accessing ...
1
vote
2answers
129 views

Do Google sitemap URLs need to be double urlencoded?

Do Google sitemap URLs need to be double urlencoded? I am submitting my sitemaps to Google and Google is having difficulty navigating to URLs that contain urlencoded characters. An example of a ...
1
vote
1answer
55 views

Will there be a conflict between my Sitemap.xml and robots.txt?

I have a robots.txt with the following in it: User-agent: * Disallow: http://tests.compkerworld.com/report_question_error.php?question_id=*&url=* Sitemap: ...
1
vote
1answer
21 views

Sitemap for WordPress extension

I'm using an extension to integrate a WordPress blog into my eCommerce site. WordPress is installed in its own directory with a link called blog on my home page. I used a free extension to do this. ...
1
vote
2answers
55 views

Sitemap structure for network of subdomains

I am working on a project that's a network of 2 domains (domain.gr & domain.com.cy) of subdomains similar to Hubpages [each user gets a profile under a different subdomain & there is a ...
1
vote
2answers
60 views

Should expired or deleted eShop links be removed from an XML sitemap?

If I delete a product should I remove its link from the sitemap.xml also? Is there any penalty from Google for leaving a lot of dead links in my sitemap? What is the most common practice and why? It ...
1
vote
1answer
150 views

Is there any advantage or disadvantage to add subdomain twice in search engines?

I have a domain (example.com) and some subdomains (e.g., sd1.example.com, ...). I added example.com to Google and Bing Webmaster Tools. I added example.com/sitemap and sd1.example.com/sitemap (with a ...