The process of recording where particular words occur. In the web context search engines crawl the web and make a list of which pages have particular words - this is the index.

learn more… | top users | synonyms

2
votes
2answers
42 views

Would Google penalize me for removing indexed pages?

I'm about to remove a large part of a website in order to replace it by a better alternative which, unfortunately, cannot be mapped to older URIs with URL rewriting. The part of the website I want to ...
2
votes
1answer
83 views

Should paginated sites be included in sitemap?

My website is ad listing for boats, boats' engines, boat rentals, etc (www.mynautics.com). There are ads listed on multiple pages. I have finished (not jet seen on link above) pagination: added ...
2
votes
3answers
1k views

Finding Google's indexed pages

How do I find out how many pages Google has indexed on my website? I know I can just Google site:http://foo.com, but I would like to record the data and not have to do it by hand. Is there an API for ...
2
votes
3answers
278 views

After adding *.xls to robots.txt how long will the results continue to show up in google

Is there an estimated amount of time before the document will not appear in google results? We have a public records document. One person raised a concern when they saw their "PUBLIC INFORMATION" on ...
2
votes
2answers
116 views

problem showing my website correctly in search engines

I have a website which i have indexed on google for example (like 15 days ago). some of my pages pass arguments like: http://www.bla.com/products.php?pro=bla&page=view suddently i saw that ...
2
votes
1answer
239 views

Should I strategically use NoFollow on internal links to make our indexing crawl more efficient?

We have a well designed internal linking system that makes certain no page is more than four pages deep, and the indexing process has gone well for us. However, in addition to the hierarchical page ...
2
votes
3answers
148 views

Site not indexed by Google [duplicate]

I typed the following on Google: (1) site: uniquevolve.com (2) site: uniquevolve.com.au Only (2) gets indexed. Is this normal? The strange thing is, my Website is actually hosted on the server with ...
2
votes
1answer
47 views

Google not indexing my Mediawiki's jpg files description pages

My website es.wikineos.com has been running for a few years with the MediaWiki Content Management System like the one they use at Wikipedia. Google seems to be indexing my site correctly at a quick ...
2
votes
2answers
74 views

Can you create a private website without password protection that would not be indexed?

I'd like to create a "private" website that clients can visit by typing in the provided direct URL (on a business card, for example), but is not password protected and would not be indexed by search ...
2
votes
1answer
65 views

Do sites using YouTube video API get indexed by Google?

Suppose I design a small app that heavily depends on Google YouTube API to dynamically fetch hundreds of YouTube video data. My site will be able to respond to users' requests, and generate several ...
2
votes
3answers
117 views

Is it safe to block redirected (but still linked) URLs with robots.txt?

I have a website that has all URLs optimized and 301 redirected from nasty URLs to clean ones. However, everywhere throughout the site the unclean URLs are linked in menus, content, products, etc. ...
2
votes
1answer
152 views

Will Google index images from HTML select tag

If i have code like this: <select> <option value="path/to/image1.jpg">Image 1</option> <option value="path/to/image2.jpg">Image 2</option> <option ...
2
votes
2answers
206 views

Is it better to burn old domain names and 301 to a new one for domains that experienced a high number of errors?

As discussed in this topic, several sites were down due to a server issue for about a month. I have slowly watched some of the more popular sites gain some traffic. I did say slowly. I have also ...
2
votes
1answer
84 views

Are the following assumptions about indexing and ranking correct?

Are the following statements true or false: If a page is crawled by Google, it automatically gets into the index (unless explicitly blocked by robots.txt) An indexed page always gets a ranking (high ...
2
votes
2answers
124 views

Prevent crawler that doesn't honoring robots.txt

I have some problem, when I try to write robots.txt for my site ... I find some issues by search on Google, and tell me about honor and not honoring robots.txt, how I can prevent it, can I perform it ...
2
votes
2answers
110 views

Blogger still visible after moving to WP; Google Indexing issues after moving from Blogger to WP

I recently migrated from Blogger to Wordpress and am having two major transition issues that are really hurting. Despite literally hours of searching and experimenting, I cannot resolve the following: ...
2
votes
1answer
82 views

How to optimize website for SEO when content is accessed via one search field

We are developing a website (sort of information index for companies) where almost all content is accessed via one search field (similar to Google): you type a search phrase, get results and then can ...
2
votes
2answers
296 views

Should I notify 'under construction' websites to crawlers? If yes how?

I am currently developing a website which is going to take some time to finish. I need to move some 'stuff' to production frequently, while maintaining a under construction status. My front page will ...
2
votes
2answers
172 views

Old pages still appearing in Google?

Many pages that have been deleted for a long time are still appearing in Google? When I then click on them, they obviously can't be found. Am I doing something wrong/forgetting something?
2
votes
1answer
242 views

Why do Google FreshBot crawled and indexed pages disappear from SERPs? Will they come back?

Sometimes a page on our site will show up in the Google SERPs that was previously not indexed (yet) due to a news story or blog posting on another site that points to it. From what I understand, the ...
2
votes
1answer
111 views

Is there an alternative to using an iframe so that my content would be visible to search engines?

I have a book-length document that is typeset with the LaTex typesetting system (It's a markup language popular for science and math typesetting.) A custom script runs a program called HyperLatex ...
2
votes
2answers
27 views

How to prevent Google from indexing non-domain URL of website?

My webhost gives you two URLs for your website: the URL on your shared server, which is something like usr283725992783.webhost.com and your domain URL, which is www.example.com Google is indexing ...
2
votes
1answer
119 views

What replaces meta keywords if Google doesn't use them anymore? [duplicate]

As a majority of webmasters already know that Google does not use Keywords anymore to help index a website properly. Which is understandable by people exploiting the system to try to gain a better ...
2
votes
1answer
56 views

Robots and Sitemaps

I have some problems regarding indexing. Google indexed so much in the list. Now I wanted to remove everything from my site. If I have disallow all in robots.txt and send my sitemap.xml. Would ...
2
votes
1answer
21 views

Should I add a site root to the sitemap file?

Is it really necessary to include a root (homedir) of the site to the sitemap? As I can understand the sitemaps main idea - it is not any necessary. Am I right?
2
votes
2answers
515 views

301 redirect from HTTP to HTTPS - how to be sure Google is fetching the correct information?

I'm hoping somebody might be able to provide a bit of advice on an issue I am having. I have one site where we implemented a 301 redirect on the homepage from HTTP to HTTPS. We have links on the ...
2
votes
1answer
55 views

Is this considered to be “Internal Near-Duplicates”?

Some people generate the same pages with a slight variation in keywords, title, country name, region name, etc... This is typically considered to be "Internal Near-Duplicates." I have seen a website ...
2
votes
2answers
68 views

Should I have link rel=next & prev on URLs which have query variables?

For example, I have link rel prev & next set up on these pages of products: site.com?page=2 site.com?page=3 (this is my preferred structure by the way and I'm trying to get all the ugly URLs ...
2
votes
1answer
131 views

JavaScript loaded external content SEO

I wonder what is the best way to have Javascript loaded content indexed by search engines. I know that search engines don't execute Javascript, but I am thinking more of an progressive enchantment. I ...
2
votes
1answer
599 views

What does “all” mean in meta robots tag?

Several websites have the following meta tag aimed at search engines like Google, Bing, etc.: <meta name="robots" content="all" /> What does it do?
2
votes
2answers
282 views

How to have google index a login protected url?

For example, my site: http://abcde.foobar.com is protected by a "single sign on" login page hosted at http://sso.foobar.com, so when a user visits "abcde", they get redirected to the "sso" login page ...
2
votes
1answer
94 views

Why is Google indexing all these other websites?

I run a web hosting company and I have noticed something strange about the way my website and the other websites that I host are indexed in Google. For some reason, other websites that I am hosting ...
2
votes
1answer
130 views

Lasting discrepancy between the site command and indexed pages in Google Webmaster

I have recently uploaded a blog on blogger. The pages of my blog are accessible for crawling and indexing and I can see that some crawling has started two weeks ago. I have noticed that the ...
2
votes
1answer
253 views

301 redirect to different page on different domain?

I want to make a 301 redirect to a different page on a different domain. The page is on the same topic, but is different content. Also, the traffic to the original page comes from the alt text on an ...
2
votes
1answer
337 views

how to set 301 redirect in wordpress .htacces file

In continuation of my previously asked question preserve-search-engine-index-while-shifting-to-new-domain. The migration is being done in the word-press and we will have same server with same IP.Since ...
2
votes
1answer
65 views

Preserve search engine rankings while shifting to new domain name

We have to shift our web-site to a new domain due to some unavoidable circumstances. The portal has been up and running there since 2009. It has good index in various search engines and we are ...
2
votes
0answers
29 views

How come a site is indexed but not indexed? [closed]

An example URL: islamqa.info/ar The site is indexed in Google Webmaster Tools as per this image But When I use google site: operator, I get Your search - site:islamqa.info/ar - did not match any ...
2
votes
2answers
47 views

Google indexing pages very slowly [duplicate]

Is there anything I can do to speed up the time it takes to index my pages? It's currently indexing them on it's own time I believe which is every 2 - 3 days and when working in music and media I need ...
2
votes
1answer
36 views

Adding Disqus to wordpress effect SEO [duplicate]

I am planning to add Disqus to my WordPress site. Now will my comments will be indexed by the Google as earlier or do I have to add any extra plugins or sitemap for comments.
2
votes
3answers
251 views

Why is Google Webmaster Tools account showing fewer indexed pages than a site: search?

When I have check my website index pages it is showing only 252 pages are indexed but when I checked through Google Search by typing site:http://www.example.com then is showing more then 900 pages are ...
2
votes
1answer
104 views

google removed links from the index [duplicate]

I asked google to crawl a site through webmaster tool and include it to the index. How actually it was done: Firstly, i picked some urls manually. Approx. after half of a day those pages were ...
2
votes
1answer
806 views

How to get google and bing to index original image, not thumbnail?

I've read this post Make google index the actual image not the thumbnail but my case is a little different. I have over a thousand images, in many different directories, with many different names, ...
2
votes
0answers
690 views

google webmaster showing 6 pages submitted 0 indexed, yet i can see them all there when i search in google? [closed]

I have a small 'brochure' type site with 6 pages, i can see them all the pages showing up in google when i search for my site. But in webmaster tools under the sitemaps section it says 6 submitted, ...
1
vote
2answers
361 views

Why won't title attributes get indexed in Google?

When I search for Ride On + my site's name, I see that it's indexed. But when I search for Green Horse + my site's name, I don't see my site appearing in the results anywhere! Here's my code: ...
1
vote
2answers
2k views

Why are new pages not being indexed and old pages stay in the index?

I currently have a site that was recently restructured, causing much of its content to be reposted, creating new URL's for each page. To avoid duplicates, all of the existing pages were added to the ...
1
vote
3answers
776 views

How to suppress PHPSESSID in URL for Googlebot?

I use cookie based sessions, and they work for normal interaction with our site. However, when Googlebot comes crawling our PHP framework, Yii, needs to append ?PHPSESSID to each URL, which doesn't ...
1
vote
2answers
66 views

Does Google provide different Index status for HTTP and HTTPS?

Will Google provide different index statuses for HTTP and HTTPS? Examples: http://www.example.com and https://www.example.com Or it will provide same results for both? If it doesn't provide the ...
1
vote
3answers
194 views

Is Google still giving weightage for old domain, i.e at least 6 month old?

Is Google still giving weightage for old domain, i.e at least 6 months old? I mean I have the best content which is useful for users. Do I still have to wait for my domain name to be at least 6 ...
1
vote
3answers
885 views

SEO: Duplicate content caused by pagination/tag/index pages [duplicate]

Possible Duplicate: What is duplicate content and how can I avoid being penalized for it on my site? I read that I should use a NoIndex tag for transitional pages like index, pagination, or ...
1
vote
2answers
40 views

sitemap.xml approach on non-indexed site

On a site that has a robots.txt file that disallows all crawling... User-agent: * Disallow: / ...does it matter what the sitemap.xml content is? Is it better, or does it make a difference at all, ...