The process of recording where particular words occur. In the web context search engines crawl the web and make a list of which pages have particular words - this is the index.

learn more… | top users | synonyms

4
votes
2answers
330 views

sitemap.xml for huge online store

I'm running online store with 30,000+ products (having unique urls) which are updated every day. I want google to index my products faster. Now crawler index ~500 pages/day (reports from google ...
4
votes
2answers
1k views

Understanding Google Webmasters Tools Sitemaps Information

So, I'm an 'avid' user of google webmaster tools and analytics. I'm building an e-commerce site, all with original product descriptions and a base of 'useful' industry-relevant content. I generated ...
4
votes
3answers
228 views

Why is the number of pages on my website indexed by Googlebot decreasing?

I am monitoring the number of pages indexed by Googlebot for one of my websites . I noticed that since last week, the number of pages has dropped by about 60 or so. Now to be fair, I have not added ...
4
votes
2answers
274 views

Google Webmaster Tools reports fake 404 errors

I have a website where Google Webmaster Tools reports 15,000 links as 404 errors. However, all links return a 200 when I visit them. The problem is, that eventhough I can visit these pages and return ...
4
votes
2answers
162 views

Index a 301 Redirect to help a user locate a page

On a site I'm currently working on, there is a 301 Redirect to redirect a nice short URL for users to type in when trying to find a particular piece of content - e.g. www.mydomain.com/apply -> ...
4
votes
2answers
287 views

Best approach to have Google index content primarily generated by Javascript

I'm developing an app where customers can drop in a Javascript tag onto their website, and it will generate content. What is the best approach to have Google index this content and associate it with ...
3
votes
4answers
289 views

Should I use 'robots.txt' or 'sitemap.xml' to get dynamically created files indexed in search engines?

I've structure my video site like this: / (root) index.html -/videos -/video-id1 (dynamically generated subdirectory when user uploads a video) /index.html (dynamically generated file ...
3
votes
4answers
246 views

How sitemap.xml influences indexation?

If a page is missing in sitemap.xml (but crawable through HTML hyperlinks), may this cause it not to be indexed? (I mean it would be indexed without sitemap, but not indexed with sitemap because it is ...
3
votes
2answers
3k views

Why the number of indexed pages decreased for a “site:” search

Recently I created a website with hundreds of thousands of unique pages. A week ago, Google had indexed about 250,000 pages. It has now decreased to 90k URLs (using a site:domain.com query). What ...
3
votes
3answers
76 views

How to prevent a search engines from indexing a section of a page?

I have many pages with lots of text in it. But I will always have two sections of text and I want to prevent one section from appearing in search results, the other section must be indexed. <p ...
3
votes
3answers
1k views

Why are URLs with my domain name at the end showing up in analytics?

I have a multi-language WordPress site that has been live since the last 3 months and today when I checked Analytics, I saw a lot of weird URLs: ...
3
votes
1answer
1k views

Search Engine Indexing of a Database-Driven Website

Suppose I have a MySQL whose rows are dog breeds and whose columns are properties of the breeds (e.g. coat color). The purpose of the site is to allow users to query the database with questions like ...
3
votes
3answers
290 views

Changed my blog's URL: what next?

I started working on a blog from last month. My blogs URL was quite big (for ex. http://my-old-long-url-blog.blogspot.com). I used it because I thought having 2-3 keywords in my URL will help me (I am ...
3
votes
6answers
152 views

Stopping Google index some web pages I have

I have some web pages that google has indexed. Now I added a robots.txt file to exclude them but should I also add a “noindex” tag and if so then how can I best add this?
3
votes
2answers
464 views

How can I get Google to not index archive pages in Blogger?

I'm using Blogger with a custom domain. When I use the Google search site:example.com, the top results are in the form Example: 2010-01-10. These are archive pages that Blogger generates. My relevant ...
3
votes
4answers
3k views

How can I get a list of all indexed pages for my domain?

We're in the process of migrating an outdated e-commerce cart to Magento. Part of this process will entail configuring proper 301 redirects. I'm aiming to get a list of all indexed pages to throw ...
3
votes
3answers
828 views

How to let Google know that content on the page has just changed?

I'd never thought of such a problem in the whole SEO spectrum before I considered launching my own website. Imagine that I have a website which is updated once per month and Google spider knows ...
3
votes
1answer
991 views

Indexing a multilingual website

I am trying to create a multilingual website using the Django framework, all default content is English. I have added a native translation (Dutch). https://mydomain.com/ redirects to the user request ...
3
votes
3answers
2k views

Google webmaster Index Status. Total Indexed=0

I previously changed my domain from www.visualstudiolearn.blogspot.com to www.visualstudiolearn.com. I had around 300 posts with the previous domain name and most of them where showing up on Google. ...
3
votes
5answers
1k views

How quickly does Bing index URLs submitted through Webmaster Tools?

In Bing's Webmaster Tools, there is a Submit URLs page that allows you to "Submit URLs that are currently not in the Bing index or that should be reindexed due to recent changes." Does anyone know ...
3
votes
1answer
321 views

Is there a way to bulk submit URLs to Fetch as Google?

Google is taking a while to re-index changes I have made to thousands of pages, so I want to use Fetch as Google to give it a prompt. However, you can only enter 1 URL at a time as standard. Is there ...
3
votes
7answers
80 views

Will a missing <head> tag prevent Google from indexing a WordPress site?

I was recently trying to configure a Google Webmasters Tools account for a client. I spent almost an hour wondering why I couldn't get the site to validate, when I found that the opening tag was ...
3
votes
3answers
45 views

Indexed webpages stopped rising, while Submited keep rising normally

Last months I have noticed a strange behaviour in Google's Webmaster Tools. Although I provide a valid Sitemap for my website, which is normally processed and without any issues from Google, while ...
3
votes
1answer
260 views

How to allow Google to index protected content?

I'm working on my site and as it require users to log in it will be hard to Google to index the sites, because to see 90% of the content you have to be online. So I've made a script which looks for ...
3
votes
2answers
134 views

Traffic fall after a server problem

I have a website from which I analyse the traffic with Google analytics. Day after day the traffic (mainly from Google SE) incresed until I get a problem with my server. For one day the server has ...
3
votes
1answer
103 views

How can I force a certain URL for Google to use in search results?

Each page on my site can be accessed by a shortened URL and normal "long" URLs. The shortened URL redirects to the long URL and the long URL includes an optional title that's not parsed (much like on ...
3
votes
2answers
61 views

How can I get Google to index content that is written into the page with JavaScript?

I have a website that focuses on charts and graphs. The graphs on the site are drawn using JavaScript, but contain keywords that I would like to have indexed. For example, here is a screenshot of ...
3
votes
1answer
62 views

Odd/faulty page links being served to Googlebot

Yahoo recently bought Tumblr, where I write fairly long form scientific pieces. I noticed recently that a new script had been added to the end of pages, and it seems to be interacting badly with the ...
3
votes
1answer
1k views

Google Webmaster Tools Index Status 0 for one year

I've read this topic with answers but they do not address why zero for one year? but other areas in Google Webmaster Tools report 400 URL indexed. Google Webmaster Tools Index Status Only 2 Indexed. ...
3
votes
1answer
132 views

Indexing dynamic loading page

Does Google index dynamic loading page? For example index.html <div id="dynamic_message"></div> index.html after few seconds <div id="dynamic_message">Hello Google, do you see ...
3
votes
2answers
1k views

Can I force Bing to reindex our site?

Our website is ranking quite well (first page) on Google for quite a lot of keywords. However, our website is not on the first 10 pages of Bing for most of these keywords. I notice our Google ...
3
votes
2answers
318 views

How to have google index a login protected url?

For example, my site: http://abcde.foobar.com is protected by a "single sign on" login page hosted at http://sso.foobar.com, so when a user visits "abcde", they get redirected to the "sso" login page ...
3
votes
1answer
478 views

Webmaster Tools show the “Total Indexed” number to be zero

I recently switched my domain from yannbane.blogspot.com to www.yannbane.com, and added my new domain to my Webmaster Tools account. After a few days, I went there and checked the data, and I was a ...
3
votes
2answers
214 views

Using .htaccess for proper indexing/SEO of timthumb images

I don't know a whole lot about the intricacies of SEO so I was wondering if someone here could provide some insight to this problem. When you use the php script timthumb on a website, it means images ...
3
votes
2answers
257 views

Google De-Index many pages at once?

On one of our websites, Google has been indexing something it wasn't supposed to. We fixed the problem so it shouldn't happen anymore, but are interested in requesting that Google de-index these ...
2
votes
4answers
270 views

Why doesn't Google remove my noindex and 404 pages?

I've added 404 error codes and no-index meta tag to pages quite some time ago. However, when I check in Google Webmaster Tools, looking at the graph of "removed" pages from their index, I see very low ...
2
votes
1answer
2k views

how to make sure an individual website is not indexed by search engines?

I'm setting up a personal website with a blog that I would like to be indexed by search engines. But the blog will link to CV with personal information that shouldn't be indexed. Can it be reliably ...
2
votes
2answers
70 views

New domain and submission to search engines

I have registered a new domain with a hosting company. They offer the feature that for each new domain there is an associated placeholder page. Actually it is a "Site not configured page" with some ...
2
votes
3answers
81 views

Multiple site pages not accesible via main site nav- SEO implications?

I run a site that has some of the site pages accessible through the site's main navigation. Then, there are 50+ "state" pages that are accessible to a user if they click on a specific state on a ...
2
votes
1answer
869 views

Google is not indexing URLs in my XML SiteMap?

I have used multiple sitemap.xml for my product pages and my category pages and problem is google only not index my whole site map's link. /sitemap_main.xml -- URLs submitted:114 // ...
2
votes
2answers
99 views

How to explain visitors that we don't control Google's results?

I have a message boards. Once in a while, some people contact us to remove anything related to them. We comply and delete all the posts and all traces. Some of those visitors still complain that they ...
2
votes
1answer
86 views

Is it bad to have both www and the naked domain indexed in Google? Should both be added to webmaster tools?

Currently when searching on Google, the www.example.com version will appear as opposed to example.com. I realize that isn't a big deal, however it made me wonder if I should do some of the same ...
2
votes
3answers
131 views

Will the already indexed URLs be removed if I use a robots.txt for my site?

Many URLs from my site have been added to Google's index, but a lot of them are outdated and they will never result in a 404 error and take the user to my site's homepage. I have submitted a new ...
2
votes
2answers
104 views

No description for any page on the website is available in Google despite robots.txt allowing crawling

I seem to have the weirdest issue with Search Engine Optimization, and I asked the IT folks at my university, I asked people on Joomla forums and I am trying to sort this issue out using Google ...
2
votes
4answers
542 views

Google crawl speed — how fast can it go?

I have a huge website with 5 million pages. Currently Google indexes about 10,000 pages per day. This is very slow, I still have lots of pages that I can't get indexed. Does anyone know what is the ...
2
votes
3answers
335 views

“X-Robots-Tag: noindex” on an HTTP 301 response

I understand that a resource with X-Robots-Tag: noindex forces some search engines, including Google, not to index the resource further. I also understand that an HTTP 301 response causes search ...
2
votes
2answers
1k views

Difference between canonical and base tag

I am a little bit confused regarding the usage of canonical tags and base tags. I have placed canonical tag (rel="canonical") on my website because of my website opening with no-www. I have reviewed ...
2
votes
1answer
1k views

Google Webmaster tools showing sitemap URL's indexed but not showing up in search results

I have submitted simple text sitemap to Google Webmaster tools and after few days it was crawled, after few more most links were indexed according to GWT but it is not showing those links in google. ...
2
votes
2answers
366 views

I've changed my URL schema. How do I tell Google to index the new schema and forget the old one?

I had a site where the urls were constructed like this /index.php/Topic /index.php/AnotherTopic These were indexed in google, and search results returned that pointed to these. However, I've ...
2
votes
2answers
994 views

What does Google do with indexed pages returning 403?

I guess that Google removes already indexed pages that are now returning the HTTP error 403. Unfortunately I can't find any definitive statement on that matter. Do you know the answer?