Questions tagged [search-engine-indexing]

The process of recording where particular words occur. In the web context search engines crawl the web and make a list of which pages have particular words - this is the index.

Filter by
Sorted by
Tagged with
36
votes
5answers
3k views

Why isn't my website in Google search results? [duplicate]

It's been sometime and Google has yet to index my site. How can I get Google to index my website?
29
votes
3answers
4k views

Why aren't search engines indexing my content?

This is a general, community wiki catch-all question and answer pair intended to address any questions concerning the reasons a site or specific site contents do not appear in search engine ...
27
votes
5answers
4k views

Page appears indexed in Google but not findable for any search terms?

(Note that I am going to use screenshots here because I suspect writing about this will change the behavior over time.) If you do a Google search for uiviewcontroller best practices either with ...
26
votes
6answers
63k views

How to prevent a PDF file from being indexed by search engines?

I have a link to PDF document on a public web-page. How do I prevent search engines from indexing this link and PDF document? The only idea I thought of is to use CAPTCHA. However, I wonder if ...
23
votes
4answers
10k views

Prevent XML sitemaps from showing up in Google search results

How do I prevent my XML sitemap files from showing up in Google search results like this result of a site: search query: I don't understand why Google would choose to show sitemap files in search ...
21
votes
2answers
8k views

Do search engines index text contained in SVG elements?

Example: <svg><text>Hello World</text></svg>?
20
votes
13answers
3k views

How do I get images containing text to be indexed by search engines?

I am building a small portfolio website for a friend, and being in advertising and all, she hates the standard fonts and wants everything written in a custom font. As CSS font-embedding is not really ...
19
votes
3answers
23k views

Google is still crawling and indexing my old, dummy, test pages which now are 404 not found

I have set up my site with sample pages and data (lorem ipsum, etc..) and Google has crawled these pages. I deleted all these pages and actually added real content but in webmaster tools, i still get ...
16
votes
3answers
23k views

How do I find when an URL was first indexed by Google?

How do I find out when a particular URL was first indexed by Google? I'd prefer a solution that works even for competitors' URLs that are not owned by me.
15
votes
4answers
2k views

How can I get Google to index my PDF documents?

We are having troubles getting Google to index the PDF files in our site. There are about 50 PDF and range in size for 20 KB to a little under two megs. They are not protected, can be read ...
14
votes
2answers
5k views

Indexing for Google scholar, which tags to use?

In the "Inclusion Guidelines for Webmasters" for Google Scholar is written under Indexing Guidelines: Google Scholar supports Highwire Press tags (e.g., citation_title), Eprints tags (e.g., eprints....
13
votes
3answers
4k views

Number of indexed pages with “site:” search less than reported in Google Webmaster Tools

In Google webmaster tools it shows that I have 309 pages indexed, but when I search using "site:site.com" appear about 180 only (On the first page appears "about 608 results," something near the total ...
13
votes
3answers
3k views

How does Google find a domain with no links to it?

I recently registered a new domain, pointed it to my existing server, and set up a minimal page just saying "test" and nothing else. I just discovered tonight that the page is already indexed in ...
13
votes
5answers
1k views

Is there any way to have search engines not index a certain section of a page?

I have Blogger setup with the Blog Archive panel shown on every page so that users can easily get to old posts. The problem is that it lists the title of each page in this panel, meaning that the ...
12
votes
7answers
8k views

How to get tens of millions of pages indexed by Google bot?

We are currently developing a site that currently has 8 million unique pages that will grow to about 20 million right away, and eventually to about 50 million or more. Before you criticize... Yes, it ...
12
votes
3answers
4k views

Can you use googleon and googleoff comments to prevent Googlebot from indexing part of a page?

I've seen code like for preventing Google from indexing part of a page: <p>This is a paragraph that will be indexed by Google.</p> <!--googleoff: index--> <p>This is a ...
11
votes
8answers
9k views

How long did it take for your new website to be indexed by search engines [duplicate]

Possible Duplicate: Why isn't my website in Google search results? I have added my website to the webmaster tools of some major search engines already, and supplied the sitemap for the ...
10
votes
2answers
2k views

Does Google submit HTML forms?

I have a web page, say http://domain/purchase and in this page, I have a web form. User, on submitting this form (which has validation, both client-side and server side and won't be validated until ...
8
votes
6answers
4k views

What is Google's policy about separate content on same URLs in mobile and desktop versions?

I'm currently developing a mobile version of my site where users' devices are automatically identified and then displayed either mobile or desktop view of the same URL. For usability purposes, I ...
8
votes
5answers
2k views

Is it possible to 'trigger' indexing on a site?

We're launching a redeveloped site soon. Google has already indexed the site that we're going to replace, but the links that google indexed are no longer applicable, because the new site has a totally ...
8
votes
5answers
1k views

Is it worthwhile to pay for boosting SEO for a new wesite?

I have come across a couple of websites today that claim to boost your website index in a matter of days. There are 1000s of reviews and all of them seem quite positive showing off results and stuff. ...
8
votes
2answers
323 views

Is using URL rewritting instead of multiple parameters the only way to make sure all pages are indexed by Google?

If I keep the pattern (index.php?module=test&param=1) will pages be less likely to be indexed then if I use the pattern (test_param_1.html) ?
7
votes
2answers
11k views

Is there a way to bulk submit URLs to Fetch as Google?

Google is taking a while to re-index changes I have made to thousands of pages, so I want to use Fetch as Google to give it a prompt. However, you can only enter 1 URL at a time as standard. Is there ...
7
votes
3answers
1k views

“X-Robots-Tag: noindex” on an HTTP 301 response

I understand that a resource with X-Robots-Tag: noindex forces some search engines, including Google, not to index the resource further. I also understand that an HTTP 301 response causes search ...
7
votes
1answer
1k views

Best way to let Google index page with tabs - SEO

Let' say I have a /pets page with 4 tabs on it: All Dogs Cats Owls The All tab displays all the pets on page load. Each of the 3 other tabs holds a list of dogs/cats/owls, respectively - loaded ...
7
votes
3answers
87 views

Is there a way to show Google and other robots a URL to original-sized image without anchor?

Let's assume I have an image. <img src="book_thumb.jpg"> And I want it to be indexed in Google and other search engines, but in its normal size, not as a thumbnail. So I just want Google to ...
7
votes
1answer
122 views

How to tell which URLs have fallen out of search engine index

I've had a few pages trimmed from google's indexed number on my sitemap in webmaster tools. How can I tell which pages have been dropped?
7
votes
3answers
4k views

Can a sitemap index file reference sitemap files under different domains?

My company owns several domain names. We also host websites for our users at their custom domains. Our domains: company.com product1.com product2.com Users's domains: user1.com user2.com ...
7
votes
5answers
266 views

Is it positive or negative for the average website to allow search engines to index user comments?

I've been trying to answer this question myself by researching and searching on google but I haven't been able to come with a final answer. I have some hypothesis I would like share anyway: The ...
7
votes
1answer
1k views

Index status shows significantly less pages than search results? *Not a New Site

I am trying to understand how within Google Webmaster Tools (Google Index>Index Status) I can have a figure of 2,000 total indexed pages (and falling daily) yet when I search Google using the (site:) ...
7
votes
3answers
1k views

For webpages using jQuery, will the Googlebot index them on “page load” or after the “page ready” state?

Will the Googlebot index a webpage on "page load" or after the "page ready" state. The reason I'm asking is because I'm trying to change my h1 and h2 tags using jQuery on page Load.
6
votes
5answers
6k views

Huge drop off in Google “indexed” pages after switching to HTTPS, should I switch back to HTTP?

I did a full redesign of my website to make it responsive (bootstrap), added localization, and also switched the whole website to use HTTPS. My website use to default to http://www.domain.com/ and ...
6
votes
3answers
1k views

How do I prevent Google from indexing text that is generated by JavaScript?

Google is executing the JavaScript on my password creator website and indexing the random passwords that get generated by the JavaScript. The random text that is generated could look spammy to Google ...
6
votes
2answers
2k views

Why doesn't searching for the unique name of my website bring the site up in a top position on Google?

I have a website with a unique name, already indexed and with a perfect SEO. Although there are no websites with the same unique name, when I'm searching on Google I found my homepage listed on the ...
6
votes
2answers
6k views

Why the number of indexed pages decreased for a “site:” search

Recently I created a website with hundreds of thousands of unique pages. A week ago, Google had indexed about 250,000 pages. It has now decreased to 90k URLs (using a site:domain.com query). What ...
6
votes
3answers
3k views

How long does it take for Google Webmasters to index site after submiting sitemap? [duplicate]

Possible Duplicate: Why isn't my website in Google search results? I have submitted my website's sitemap to webmaster's tools. The status on the sitemap says OK and it shows that 12 URLs ...
6
votes
1answer
786 views

How to tell bots to forget a website and reindex it from scratch

I was wondering if there was anything I could add to robots.txt or sitemap to tell bots to completely forget everything they know about a website and index it from scratch? Context: After replacing a ...
6
votes
5answers
7k views

Why are URLs with my domain name at the end showing up in analytics?

I have a multi-language WordPress site that has been live since the last 3 months and today when I checked Analytics, I saw a lot of weird URLs: /menswear-fashion-autumn-winter-2013-2014/www.domain....
6
votes
2answers
391 views

Is content inside IE conditional comments indexed by search engines?

Although I'm aware of the various methods to show content only to users of a specific IE version, I was wondering if using conditional comments would have an effect on the way my site is indexed. If I ...
6
votes
4answers
809 views

Will JavaScript (document.write) content be indexed by search engines?

I have an external JS file embedding some content inline in a web page. Basically it just does a document.write("my content") via a script tag and loaded externally with the src attribute. Any ...
6
votes
2answers
138 views

Preserve search engine rankings while shifting to new domain name

We have to shift our web-site to a new domain due to some unavoidable circumstances. The portal has been up and running there since 2009. It has good index in various search engines and we are ...
6
votes
1answer
664 views

Why doesn't Google index the translated versions of my page?

I have a website in Persian language and translated some of my popular posts into English and added this header in <head> section in both of the pages (not entire site): <link rel="alternate"...
6
votes
1answer
60 views

UK/USA Exact Copy English Causing Error “Duplicate submitted url not set as canonical”

starting around February 25th, 2019 we have been having pages excluded from indexing for our UK website with the error "Duplicate, submitted URL not selected as canonical". We have not had this issue ...
6
votes
3answers
676 views

Get search engines to index hash-based URLs

I have a client who has website with multiple pages. Now, each page has tabbed content which is accessible through #. For example: example.com/page1#content1 example.com/page1#content2 example.com/...
5
votes
5answers
1k views

How can I fix Google indexing URLs that are not part of my sitemap?

Using site:example.com in Google is returning many results with the following format: https://www.w.example.com/services/edison/16mm-to-2k Obviously this is not what I submitted and is not part of my ...
5
votes
6answers
280 views

How do you prevent web pages from appearing in the Google search index?

I have some web pages that Google has indexed. I have now added a robots.txt file to exclude them, but should I also add a noindex tag? If so, then how do I do so?
5
votes
5answers
246 views

What measures can be taken to make sure Google is aware of the existence of a newly created page?

Consider a website with a large number of pages. New pages are published regularly. When publishing a new page the website operator wants to get the newly created paged crawled/indexed in Google as ...
5
votes
1answer
585 views

Will Google penalize me for duplicate content on GitHub from my knowledge base?

I want to have a copy of each of our documentation articles in GitHub so that other users can improve, edit and add to the document. The accepted changes will be made live in our knowledge base forum. ...
5
votes
2answers
858 views

Prevent Googlebot from seeing or indexing common keywords such as “read full story”

I have a site that uses a lot of the same anchor text. For example, "read full story". When I do an SEO audit for my site, I find that these are my most frequent keywords, but I am not trying to ...
5
votes
3answers
391 views

Do WordPress websites get indexed quicker by search engines than a regular website?

I registered a couple of domains with the names of categories of products we sell. I then installed WordPress in one of those domains and played around with it for a bit, and left it alone for about a ...