Tag Info

Hot answers tagged

38

This is not uncommon as there is no time frame for Google to index a website. The best thing you can do is to give Google every opportunity to find you. Try all of the following: Submit your website to Google. Submit an XML sitemap. You can do that from within your Webmaster Control panel. Most content management systems have a sitemap generator of some ...


16

Okay. This is a rather odd one, but not as tragic as you would think. I am not sure what the payoff would be. This makes absolutely no sense to me as to why someone would do this. I cannot see an advantage for the hacker at all. The good news is the visitor is reaching your site. However, there will be a hit from an SEO standpoint so you want to fix this ...


13

There are 2 main ways to prevent search engines from indexing specific pages: A Robots.txt file for your domain. The Meta Robots tag on each page. Robots.txt should be your first stop for URL patterns that match several files. You can see the syntax here and more detailed here. The robots.txt file must be placed in the root folder of your domain, i.e. at ...


13

I would strongly recommend registering your site with Webmaster Tools. There is a crawler access section under site configuration that will tell you when your robots.txt was last downloaded. The tool also provides a lot of detail as to how the crawlers are seeing your site, what is blocked or not working, and where you are appearing in queries on Google. ...


10

The main reason for your pages not being indexed is because there are no html links. You're providing javascript links to the other pages and while the #! denotes that it should be a different page - you're not upholding your end of Google's javascript crawling agreement: An agreement between crawler and server In order to make your AJAX application ...


9

Google periodically scan web. Might be your site is not scanned till now. Checklist Ensure your robots.txt file is not blocking Google. Ensure your pages are linked and are crawler-able by Google's Bot Ensure sitemap.xml is located in the correct path or if using a custom page then you will need to manually submit it too Google and other search engines. ...


9

I researched the answer to this question this way: using Google since this is the example I have, how Google gets creation dates and modified dates, and date formats that Google recognizes. Please understand that this information does not exist on just a few pages and I had to ferret out the data from very many sources some of which do not seem to apply ...


7

First, I hope you put a proper 301 redirect for any page in your htaccess. If some of your pages require parameters, like example.com/?p=20, then you need to use mod_rewrite with [R=301] instead of the easier mod_alias Redirect directive in your htaccess. Also, until Google Bot comes again and indexes, it will not update the Google index. Second, it will ...


7

Use HTTP 410 (Gone) for pages you'd like to stop from being indexed. For changed URLs with the same content use HTTP 301 (Moved Permanently).


7

Check out the URL removal tool in Google Webmaster Tools. I'd also 404 the pages instead of redirecting them to get them removed faster, in the future beyond robots.txt you could drop in the rel="canonical" to make sure Google knows the dev site is just a copy of the main site and is not to be indexed.


7

The URLs you have indexed have both the HTTPS and HTTP protocol in them along with www. These would be reported separately in Webmaster Tools, so if your verified site in Webmaster Tools is HTTP plus www, the sitemap count wont report on the HTTPS links and vice-versa. As you seem to be redirecting to HTTPS, ensure you have an HTTPS version verified and a ...


6

Sadly Google's Webmaster Tools doesn't use REALTIME DATA and is updated in intervals from a few days and sometimes I've seen some data missing for up to 2 weeks. So there's nothing to be worried about, your URLS are indexed as Google updates the rankings more often they do Webmaster Tools. You will also find similar problems with Traffic, and Keyword Data ...


6

Wordpress is set to ping Google whenever you write a new post. From http://wordpress.org/support/topic/google-blog-pinging Settings->Writing. At the bottom you'll find a list of "Update Services". The default is to ping the pingomatic server, which then pings a dozen or more other places for you, Google included. However, if you want to ping Google ...


6

You could do this using the following rewrite rule in .htaccess: RewriteEngine on RewriteCond %{QUERY_STRING} searchkey RewriteRule .* - [G,L] .* means that it will do this for any URL on your site (as long as the condition of having "searchkey" in the query string is met) - means to leave the URL alone and not change it G means "Gone" -- send the 410 ...


6

If Google has started adding pages again, then you are fine. We had this happen with a few sites that were pushed from development into production and their robots.txt was not changed. In fact, unless you are trying to block pages, I recommend removing robots.txt completely. There's no point in saying allow everything in robots.txt when that is the ...


6

To know the age of an URL you can follow this link by replacing www.example.com by the URL you want: https://www.google.com/search?tbs=cdr%3A1%2Ccd_min%3A1%2F1%2F2000&q=site%3Ahttp%3A%2F%2Fwww.example.com&safe=active&gws_rd=ssl For example, here's the result from Google for the Meta site of Stack Overflow: Otherwise, the Wayback machine is ...


5

Don't forget the concept of domain aging either. If you slap up a new site on a brand new Domain name, expect Google and the others to de-emphasize it for a time period of at least several months. This doesn't explain flat out omission as you're seeing, but it does explain why it's incredibly uncommon to see a site shoot directly to the top of the search ...


5

It is best to let them disappear as long as your redirects are 301's. Google will in time see that they are 301(permanent) redirects and will switch over to the new URL. I have done this myself and you should see results within 2-4 weeks, if not faster. If it has been more than a month and your old URLs are still showing up in Google's SERPs then you ...


5

There are various theories as to how Google knows what to crawl. It could be that someone linked to your mobile version. It could be that Google tried random urls and came across the /m version of your site. I'm not aware that they say they won't use URLs from their analytics data. Yes they do follow those rules: ...


5

Simple answer: they don't deserve to be there. They could be thin-content, duplicate content, poor content or any other such reason. It's Google's prerogative to decide what should, and what shouldn't be included in it's index. The purpose of an XML sitemap is to allow Google to discover pages that would be suitable for inclusion in it's index, but for one ...


5

In my experience: with no sitemap submitted, it might take more than 30 days, with sitemap it usually takes a couple of weeks.


5

Do you have any other sites pointing to it? Ironically the fact you've added a link to it from this site will ensure it does get indexed (not 100% but I would put money on) Any way, it is indexed: Google Link Also, your code is poor... You have this code (as an example - this is copied from your site): <img src="/images/arrow_to_login2.png" ...


5

The reason google is not following your Shebang (#!) links is because when the page loads initially they do not exist and they are no where to be found in the source code. In other words with javascript disabled you do not have a single <a> anchor tag in your html source of your page. The only thing that will be indexed is a blank page with copyright. ...


5

The home page of your site is indexed and cached by Google and appears in search results. See this screenshot of a search including a string of text taken from your page: Remember that a "site:" search in Google doesn't necessarily show all the pages that have been indexed, and that the order of pages doesn't reflect the "value" of each page.


5

In my opinion, the best way to solve your problem is to: Make 301 redirects from your old URLs to new ones Google will understand easily if you changed your sitemap.xml, internal links and if there is no more 404 pages. In your case, Google says you must follow these rules.


5

That is normal and expected behaviour and in SEO-terms that's called Duplicate Content. So Website A got indexed before Website B, and since the content is identical, Google sees no value in positioning Website B in its search results. There is no way around the issue than to rewrite Website B's content to be unique to get that website positioned in the ...


5

Will google still crawl to Full_Res_Image and index that in web as well as google images? Yes. If it does crawl to the full res image, how do I then set alt tags on the full res image? You can't but you can still use other indicators of content such as: Use the alt attribute on the cropped image. By being the effective anchor text of that link ...


5

Google is only going to index millions of pages from a site with lots of reputation. I'd estimate that your site would have to have at least PageRank 6 before Google were willing to index that many pages on your site. That is especially true when the pages are only listed in a sitemap (as tillinberlin suggests). User profile pages are often low quality. ...


4

First, the Google Webmaster Tools will tell you how many pages are in the index. Google may have dropped pages from its index from multiple reasons, which they will NOT tell you, but the most probable is duplication or very similar content. If you feel that Google is not indexing enough content then you should check the webmaster tools under ...



Only top voted, non community-wiki answers of a minimum length are eligible