0
votes
0answers
12 views

How do services that make subsites (ie. new subdomains) get those sites indexed by Google and other search enginex? [duplicate]

Services like tumblr, github pages, quora blogs, etc. make new subdomains when a user signs up or makes a site. So the subdomains are like: blogname.tumblr.com , pagename.github.io, ...
1
vote
1answer
125 views

Creating sitemap for Googlebot - how to mark dynamic content / dynamic subpages?

I have a website that is an Internet forum. This forum has many categories, and a single category page that contains a lot of subpages with listed threads. This Internet forum is brand new, and about ...
5
votes
2answers
208 views

How frequently does Googlebot fetch sitemaps? Does it depend on PageRank?

How frequently does Google fetch sitemaps? I am now working with a high traffic website that normally has 30 new posts per minute. It currently it provides a sitemap which only includes the 100 ...
1
vote
2answers
236 views

sitemap and crawler

I am still newbie in the entire world of SEO. Let's take a look at my example. I have a website very similar to blog style site. There few patterns for urls, all of the are very simple and allowed to ...
1
vote
1answer
4k views

How to fix Google 404 not found Crawl Errors?

I was checking on Google webmater tool for my blog site to see if there's any indication on why my blog traffic decreased to half in one day and i saw 43 Not Found crawl errors and 5 in Sitemap Not ...
0
votes
1answer
93 views

Specifying crawlers to not crawl Links which are dependent on external API's

I have links on my site which lead to internal pages that depend on first bringing data from an external API. This only takes time the first time they are pushed. Links to pages which exists in the ...