Linked Questions

2
votes
4answers
1k views

Do you really need robots.txt and sitemap.xml if I want bot to follow every link? [duplicate]

I wonder if I really need robots.txt/*sitemap.xml* files if I want bots to go through every link on my website without any restrictions? And if I do, what should I write in there?
0
votes
3answers
93 views

How to make Google index all sitemap.xml records? [duplicate]

I have a sitemap.xml index fine with 3 XML files in it with around 120 pages for indexing. A year passed and I still have 1/3 of it indexed. All the most important pages for indexing are in 1st ...
0
votes
1answer
61 views

Sitemap.xml doesn't seem to be working properly [duplicate]

I have a sitemap.xml on one of my domains. It contains the following: <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <url> <loc>http://www.domain.com</loc> ...
0
votes
0answers
10 views

Do I need to produce an xml sitemap for a social network? [duplicate]

I'm working on a social network that currently does not have an xml sitemap that has been submitted to Google Webmaster Tools. Will this add much benefit given that it is a social network with often ...
23
votes
5answers
3k views

Page appears indexed in Google but not findable for any search terms?

(Note that I am going to use screenshots here because I suspect writing about this will change the behavior over time.) If you do a Google search for uiviewcontroller best practices either with ...
5
votes
3answers
2k views

How to get tens of millions of pages indexed by Google bot?

We are currently developing a site that currently has 8 million unique pages that will grow to about 20 million right away, and eventually to about 50 million or more. Before you criticize... Yes, it ...
3
votes
4answers
2k views

Google search does not show sub-pages from my website

My website appears in Google search, but only the first page. Of course I have sub-pages linked from the first page, but the sub-pages do not show in Google search. Not in Yahoo, not in Bing. What ...
5
votes
3answers
291 views

Should I include forum posts in the sitemap?

I am launching a medical discussion forum for sharing study materials and discussing other topics. The forum will probably grow fast and reach at least a few hundred topics. Should every discussion ...
7
votes
6answers
222 views

In regards to SEO, is a site map important for a small 3 page website?

In regards to SEO, is a site map important for a small 3 page website? I understand that it is not needed for the convenience of a user visiting the site, but will it help the search engines find it?
7
votes
2answers
138 views

Can a sitemap change result in a drop of visits?

I recently restructured and resubmitted the sitemap for a website of mine in GWT. The old one was built by a script I acquired, which had its flaws but dit the job. After having implemented URL ...
5
votes
3answers
166 views

Is submitting a sitemap an outdated technique for SEO?

I'm not getting more content or recommendations for sitemap practice in SEO. Is submitting a sitemap an outdated practice in SEO or is it necessary to submit a sitemap for SERP?
3
votes
3answers
297 views

Why Facebook profiles are Google-searchable?

Facebook has around 1B user profiles. They can be found by searching in Google. However, I don't think these profiles are linked from anywhere, so how could Google discover them? As far as I know, ...
1
vote
1answer
656 views

What's better for SEO folders or filenames?

I was wondering which of this two possibilities may be better for SEO. Let's take The Sitemap Paradox as an example. What would you reccommend: ...
0
votes
1answer
923 views

How to let Google know about dynamic content?

I'm looking for the best practice to let Google know about a vast number of dynamically created content. Let's say (I mean - dream) that I'm Facebook, and I want to let Google to index all the users' ...
2
votes
3answers
193 views

How to specify priority content for Google to index first when launching a very large website

My team will go online with a portal that will have over 1.000.000 products. And I read here following text: Prioritize which content you have indexed. With that much data, not all of it is ...

15 30 50 per page