Linked Questions
188 questions linked to/from The Sitemap Paradox
4
votes
4
answers
5k
views
Do you really need robots.txt and sitemap.xml if I want bot to follow every link? [duplicate]
I wonder if I really need robots.txt/*sitemap.xml* files if I want bots to go through every link on my website without any restrictions? And if I do, what should I write in there?
- 311
1
vote
3
answers
362
views
How to make Google index all sitemap.xml records? [duplicate]
I have a sitemap.xml index fine with 3 XML files in it with around 120 pages for indexing.
A year passed and I still have 1/3 of it indexed. All the most important pages for indexing are in 1st ...
- 1,619
0
votes
1
answer
829
views
Would google index a page in sitemap which there is no direct link to it from website? [duplicate]
Let's say I have a listing website with thousands of pages. I don't want to make all those pages(items) crawlable by others, unless they search for those items.
But, still I like to have all those ...
- 163
1
vote
0
answers
514
views
Google will render but not index my SPA site [duplicate]
I have a single-page-app / website built with React JS, where Google will only index my index page. All the other URL's in my sitemap are recognised, but either marked with Discovered - currently not ...
- 111
0
votes
1
answer
446
views
Sitemap.xml doesn't seem to be working properly [duplicate]
I have a sitemap.xml on one of my domains. It contains the following:
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>http://www.domain.com</loc>
...
1
vote
1
answer
185
views
Is it enough to submit an XML file to get Orphan pages indexed and ranked? [duplicate]
We have our sitemap submitted to WMT. But we have a lot of Orphan pages on our website, and it seems that they do not rank or do not appear on search, therefore, hardly have any traffic or no traffic ...
- 119
0
votes
1
answer
152
views
Are internal links to pages more valuable than sitemap.xml links [duplicate]
So part of my site is asynchronously loaded with javascript. This part contains links to products. All these links also exist in my sitemap.xml.
I've tested with Googlebot Fetch and it doesnt see ...
- 101
0
votes
1
answer
30
views
Would google index our page if we have it in sitemap but do not have any links for that on website? [duplicate]
Short
Would google index my web-page if it's not on my website but it's in the sitemap?
Long
We have a few product price pages i.e. for product bwm-x3, following are 3 URLs:
-> bmw-x3-price-in-...
- 538
0
votes
0
answers
10
views
XML Sitemap submitted but few page show indexed via Google site: search [duplicate]
I submitted a sitemap of my website to Google. (japan1900.com)
The sitemap : (https://japan1900.com/en/module/lgsitemaps/sitemap?fc=module&name=product_1_1)
The robots.txt : https://japan1900.com/...
- 1
292
votes
26
answers
62k
views
What are the best ways to improve a site's position in Google?
What are the best ways to improve a site's position in the Google search results?
17
votes
6
answers
12k
views
How to get tens of millions of pages indexed by Google bot?
We are currently developing a site that currently has 8 million unique pages that will grow to about 20 million right away, and eventually to about 50 million or more.
Before you criticize... Yes, it ...
- 1,328
16
votes
3
answers
10k
views
Google Search Console is making up URLs which don't exist in my Sitemap and then complains that these pages have error
I am working on an e-commerce website. I used to have a sitemap with category Ids, something like:
<url>
<loc>https://my-domain.com/home/browse/2/45/139</loc>
<changefreq&...
- 538
28
votes
5
answers
4k
views
Page appears indexed in Google but not findable for any search terms?
(Note that I am going to use screenshots here because I suspect writing about this will change the behavior over time.)
If you do a Google search for
uiviewcontroller best practices
either with ...
- 14k
15
votes
1
answer
10k
views
Do search engines perform JS rendering while crawling?
I have couple of websites recently created (~8 months) and Google is marking them as "Duplicate without user-selected canonical" or "Page with redirect" errors
How I implemented:
...
- 153
11
votes
2
answers
13k
views
Is it necessary to submit sitemap to www and non www?
I have submitted my website to Google Webmaster Tools with both www and with no-www. I had made the domain without www to my preffered domain and have submitted a sitemap.
My question is should I ...
- 211