Tag Info

Hot answers tagged

36

This is not uncommon as there is no time frame for Google to index a website. The best thing you can do is to give Google every opportunity to find you. Try all of the following: To submit your site to Google go here. Submit an XML sitemap. You can do that from within your Webmaster Control panel. Here's a drupal add on to make this easy for you. Get ...


13

I would strongly recommend registering your site with Webmaster Tools. There is a crawler access section under site configuration that will tell you when your robots.txt was last downloaded. The tool also provides a lot of detail as to how the crawlers are seeing your site, what is blocked or not working, and where you are appearing in queries on Google. ...


13

There are 2 main ways to prevent search engines from indexing specific pages: A Robots.txt file for your domain. The Meta Robots tag on each page. Robots.txt should be your first stop for URL patterns that match several files. You can see the syntax here and more detailed here. The robots.txt file must be placed in the root folder of your domain, i.e. at ...


10

The main reason for your pages not being indexed is because there are no html links. You're providing javascript links to the other pages and while the #! denotes that it should be a different page - you're not upholding your end of Google's javascript crawling agreement: An agreement between crawler and server In order to make your AJAX application ...


8

I researched the answer to this question this way: using Google since this is the example I have, how Google gets creation dates and modified dates, and date formats that Google recognizes. Please understand that this information does not exist on just a few pages and I had to ferret out the data from very many sources some of which do not seem to apply ...


7

First, I hope you put a proper 301 redirect for any page in your htaccess. If some of your pages require parameters, like example.com/?p=20, then you need to use mod_rewrite with [R=301] instead of the easier mod_alias Redirect directive in your htaccess. Also, until Google Bot comes again and indexes, it will not update the Google index. Second, it will ...


7

Google periodically scan web. Might be your site is not scanned till now. Checklist You have proper robots.txt Some of posts at your site are linking to other posts of your site. sitemap.xml is placed on proper path. And its entry is mentioned in robots.txt. You had successfully submitted your site sitemap to google and other web search engines. I'll ...


7

Use HTTP 410 (Gone) for pages you'd like to stop from being indexed. For changed URLs with the same content use HTTP 301 (Moved Permanently).


7

Check out the URL removal tool in Google Webmaster Tools. I'd also 404 the pages instead of redirecting them to get them removed faster, in the future beyond robots.txt you could drop in the rel="canonical" to make sure Google knows the dev site is just a copy of the main site and is not to be indexed.


6

Wordpress is set to ping Google whenever you write a new post. From http://wordpress.org/support/topic/google-blog-pinging Settings->Writing. At the bottom you'll find a list of "Update Services". The default is to ping the pingomatic server, which then pings a dozen or more other places for you, Google included. However, if you want to ping Google ...


6

If Google has started adding pages again, then you are fine. We had this happen with a few sites that were pushed from development into production and their robots.txt was not changed. In fact, unless you are trying to block pages, I recommend removing robots.txt completely. There's no point in saying allow everything in robots.txt when that is the ...


5

It is best to let them disappear as long as your redirects are 301's. Google will in time see that they are 301(permanent) redirects and will switch over to the new URL. I have done this myself and you should see results within 2-4 weeks, if not faster. If it has been more than a month and your old URLs are still showing up in Google's SERPs then you ...


5

There are various theories as to how Google knows what to crawl. It could be that someone linked to your mobile version. It could be that Google tried random urls and came across the /m version of your site. I'm not aware that they say they won't use URLs from their analytics data. Yes they do follow those rules: ...


5

Simple answer: they don't deserve to be there. They could be thin-content, duplicate content, poor content or any other such reason. It's Google's prerogative to decide what should, and what shouldn't be included in it's index. The purpose of an XML sitemap is to allow Google to discover pages that would be suitable for inclusion in it's index, but for one ...


5

In my experience: with no sitemap submitted, it might take more than 30 days, with sitemap it usually takes a couple of weeks.


5

Do you have any other sites pointing to it? Ironically the fact you've added a link to it from this site will ensure it does get indexed (not 100% but I would put money on) Any way, it is indexed: Google Link Also, your code is poor... You have this code (as an example - this is copied from your site): <img src="/images/arrow_to_login2.png" ...


5

The reason google is not following your Shebang (#!) links is because when the page loads initially they do not exist and they are no where to be found in the source code. In other words with javascript disabled you do not have a single <a> anchor tag in your html source of your page. The only thing that will be indexed is a blank page with copyright. ...


5

The home page of your site is indexed and cached by Google and appears in search results. See this screenshot of a search including a string of text taken from your page: Remember that a "site:" search in Google doesn't necessarily show all the pages that have been indexed, and that the order of pages doesn't reflect the "value" of each page.


5

The URLs you have indexed have both the HTTPS and HTTP protocol in them along with www. These would be reported separately in Webmaster Tools, so if your verified site in Webmaster Tools is HTTP plus www, the sitemap count wont report on the HTTPS links and vice-versa. As you seem to be redirecting to HTTPS, ensure you have an HTTPS version verified and a ...


5

That is normal and expected behaviour and in SEO-terms that's called Duplicate Content. So Website A got indexed before Website B, and since the content is identical, Google sees no value in positioning Website B in its search results. There is no way around the issue than to rewrite Website B's content to be unique to get that website positioned in the ...


5

Will google still crawl to Full_Res_Image and index that in web as well as google images? Yes. If it does crawl to the full res image, how do I then set alt tags on the full res image? You can't but you can still use other indicators of content such as: Use the alt attribute on the cropped image. By being the effective anchor text of that link ...


4

Don't forget the concept of domain aging either. If you slap up a new site on a brand new Domain name, expect Google and the others to de-emphasize it for a time period of at least several months. This doesn't explain flat out omission as you're seeing, but it does explain why it's incredibly uncommon to see a site shoot directly to the top of the search ...


4

First, the Google Webmaster Tools will tell you how many pages are in the index. Google may have dropped pages from its index from multiple reasons, which they will NOT tell you, but the most probable is duplication or very similar content. If you feel that Google is not indexing enough content then you should check the webmaster tools under ...


4

There shouldn't be any noticeable SEO difference between your own domain and a wordpress.com sub-domain in Google. Google ranks pages, not domains. Unless your domain is considered spam, it is a better option to host your blog/site in your domain. In your own domain, you can install plugins to improve SEO by creating a XML sitemap. You can also choose a SEO ...


4

Would generating sitemap xml improve indexing? In theory, but there are no promises, either. ("Google doesn't guarantee that we'll crawl or index all of your URLs." etc.) Sitemaps can be helpful for bots to find all your pages, but they're informational; they aren't required to do anything with them. The search engines may decide to disregard some ...


4

There is no timetable for when Google will crawl and/or index your pages. For new sites it usually is a not very quick process unless you happen to get a high quality link or two pointing to your pages. That always speeds things along but rarely happens. The best thing you can do to speed up the process is to promote your site and seek realted/quality links ...


4

As I see it, you basically have two problems: Google keeps returning the mistakenly created duplicate URLs in search results, confusing users, and Googlebot keeps trying to recrawl the duplicate URLs, slowing down the crawling of your actual pages. The first problem is more serious, since you're actually losing visitors. The best way to solve it would ...


4

You have an old and outdated architecture, so it's pretty hard to make any actionable recommendation. What I would do : Remove the external sitemap file on hostedsitmaps.com, an external sitemap is not a sign of quality. Host it on your site. Redirect with 301 example.com to www.example, or the other way. Choice www.example or .example then stick with it. ...


4

Providing all your links go to correct canonical URLs, then you should set up 301 redirects from the erroneous pages to the canonical URL. This will immediately fix all the links for users following incorrect links in the SERPs, and over time Google will correct the links in the SERPs.



Only top voted, non community-wiki answers of a minimum length are eligible