Tag Info

Hot answers tagged

20

There are a number of reasons your content may not appear in search engine results, however, it is important to note that a search engine's index may contain pages that it doesn't display in its results page. How to tell if your content is actually indexed It may actually be difficult to tell if your content is indexed. Search for all the documents from ...


17

To prevent your PDF file (or any non HTML file) from being listed in search results, the only way is to use the HTTP X-Robots-Tag response header, e.g.: X-Robots-Tag: noindex robots.txt does not prevent your page from being listed in search results. What it does is stop the bot from crawling your page, but if a third party links to your PDF file from ...


16

Some potential strategies: Google Webmaster Tools allows you to request an increased crawl rate. Try doing that if you haven't already. Take another look at your navigation architecture to see if you can't improve access to more of your content. Look at it from a user's perspective: If it's hard for a user to find a specific piece of information, it may be ...


16

EMBED THE FONTS! No, but really, Embeddable fonts work on all current browsers (FF, Chrome, Safari, Opera) and IE5.5+ (yes, it's been working in IE since the 90s.) Get your TTF upload it here: http://www.kirsle.net/wizards/ttf2eot.cgi It'll give you the code and 2 files back (a TTF and then an EOT[M$ web font]) back. Copy, paste, upload, done. Win win! ...


12

There's not much you can do about this. There's a teensy bit you can do to help, but the problem is endemic to Google's index. There isn't just one Google "index;" it's sharded. Pages with low PageRank appear in very few shards. Using site:stackoverflow.com forces your query onto a shard that has a larger fraction of your URLs indexed. That explains the ...


11

This is an XY problem. You want to prevent indexing on your site and you know that 404s are not indexed, so you want to prevent indexing 'using' 404s. This is the wrong way to go. There are many proper ways to prevent indexing such as using robots.txt, meta tags or authentication.


10

Google does appear to index XML sitemaps (like any XML file). It seems that if Google is aware of a URL and it returns a valid response then it's likely to get indexed. Personally, I only submit the sitemap through GWT and include a Sitemap: reference in robots.txt and this is certainly enough to get it indexed. I believe the recommended method to prevent ...


10

That's a very bad idea and your site will suffer in the organic search rankings. For one, Google does have image recognition abilities and so your assumption that the bot can't "view" the image is wrong. And two, the algorithm does render pages to decide whether they are not mobile friendly and if your pages are delivering different content to users and to ...


9

Yes You can now use Google search to find SVG documents. SVG is an open, XML-based format for vector graphics with support for interactive elements. We’re big fans of open standards, and our mission is to organize the world’s information, so indexing SVG is a natural step. We index SVG content whether it is in a standalone file or embedded ...


9

There are at least 3 ways: Links to your site. Using Google Webmaster Tools (now called Search Console) Registrar dumps, triggers, and other options. Google will find many new sites quickly from some registrars. For example, Google found one domain name I registered using GoDaddy, indexed it, and began sending search results within 20 minutes of ...


9

A similar question was asked on Stack Overflow back in January. John Mueller was kind enough to respond with the following: "googleon" and "googleoff" are only supported by the Google Search Appliance (when you host your own search results, usually for your own internal website). They are not supported by Google's web-search at all. As far as ...


9

Okay. First things first. Do not mark your 404 as being fixed. You are actually prolonging the issue. Google will try and fetch a page that returns a 404 several times before giving up. This is because the 404 error indicates a temporary situation where a 410 error says the page is gone. So every time you mark a 404 as being fixed, you are in effect telling ...


9

If this is a test site that shouldn't be indexed at all, there are a couple of steps you can take that tell search engines not to index your site more effectively than returning 404 headers. robots.txt Include a robots.txt at the site's root including: User-agent: * Disallow: / X-Robots-Tag Include the following to your .htaccess to add an X-Robots-Tag ...


8

The best course of action is to use canonical URLs. This avoids a situation where you are penalized for duplicate content. When it comes to desktop vs mobile websites, most sites will have something like this on their mobile website: Example for: http://m.mywebsite.com/page.html <link rel="canonical" href="http://mywebsite.com/page.html" /> The ...


7

No, URL rewriting has no positive influence on crawling or indexing; oftentimes it can even have a negative effect. For instance, many websites embed all parameters as well as plain text (eg the "title") in the URL when rewriting them, creating pages with many possible URLs (we often even see session-IDs rewritten in URLs...). We did a blog post on this at ...


7

I do not believe Google says. It used to be true that test_param_1.html was better than index.php?module=test&param=1 but I do not believe that is the case anymore for Google. However, if your 2 examples pages are very similar in content then you will want to use a rel canonical on them so Google knows which one is the master. Otherwise your pagerank ...


7

Eprints tags are used in the EPrints academic repository software. BE Press tags are used in The Berkley Electronic Press SelectedWorks product. Neither offers documentation on their proprietary meta tags that I could find. You can, however, view the source of any HTML page generated by an Eprints or SelectedWorks server to see their metadata tags in use. ...


7

Google has been able to crawl HTML forms since 2008. Having chosen the values for each input, we generate and then try to crawl URLs that correspond to a possible query a user may have made. If we ascertain that the web page resulting from our query is valid, interesting, and includes content not in our index, we may include it in our index much as we ...


7

If it's a HTML document, include the following meta tag in it: <meta name="robots" content="noindex" /> This will tell Google and every other search engine that understands the tag (which should be pretty much all of them, since it's been around since 1996) not to index the page. For non-HTML documents, you can configure your webserver to send the ...


7

Technically speaking http://www.new.com/tag/relationships and http://www.new.com/tag/relationships/ are two different pages just like http://www.example.com/ and http://www.example.com/index.html are two different pages even though they pull up the same page. To make sure the search engines understand that http://www.new.com/tag/relationships and ...


7

Wordpress is set to ping Google whenever you write a new post. From http://wordpress.org/support/topic/google-blog-pinging Settings->Writing. At the bottom you'll find a list of "Update Services". The default is to ping the pingomatic server, which then pings a dozen or more other places for you, Google included. However, if you want to ping Google ...


7

Robots.txt just tells search engines not to crawl your pages. It does not tell them to not index your pages. So if your pages have links from other websites the search engines will know they exist. And because off-page factors affect rank, sometimes greatly, your pages can rank well for long-tail search terms without ever being crawled. To actually prevent ...


7

In my experience, mobile visitors want the same content as your desktop visitors do. I worked for a travel website with lots of information about hotels and restaurants. The site is generally known for hotels, but we thought that mobile users would be much more interested in restaurant content because they we looking for something when they were out. ...


7

you should use both: crawling encouragement using a sitemap.xml – A well structured sitemap.xml can help search engine spiders to find your content quick and directly. And a sitemap.xml gives you the option, of curating what you want to have crawled, how often you consider crawling useful and you can even put a focus on content you find important – the ...


7

No SEO campaign is perfect. So please throw this notion out of the window. Since search engines change strategies and updates come on the average of several a day, it is impossible to have a perfect SEO score at anytime. However, you can have a highly competitive score regardless of the changes if done well. Without knowing your website name and any terms ...


6

Supposedly you can add the class robots-nocontent to elements on your page, like this: <div class="robots-nocontent"> <p>Ignore this stuff.</p> </div> Yahoo respects this, though I don't know if other search engines respect this. It appears Google is not supporting this at this time. I suspect if you load your content via ...


6

Create a Webmasters Account from Google, here http://www.google.com/webmasters/. Once you have an account you can attach it to the website in question by putting a file into the root folder of your site. Google will then recognize that you as the owner of the Webmaster account also own/control the site in question. Once you have done this Google will ...


6

Unless she is selling her own fonts on this website, I think you both could work more on CSS embedding (yeah, I read this is not an option for you now, but I insist). If the case is a heavier graphic work upon fonts (gradient colors, twisted alignment, glossing, embossing, engraving...), making it really impossible to render thorugh CSS options as she ...


6

There are multiple ways to do this (combining them is obviously a sure way to accomplish this): 1) Use robots.txt to block the files from search engines crawlers: User-agent: * Disallow: /pdfs/ # Block the /pdfs/directory. Disallow: *.pdf # Block pdf files. Non-standard but works for major search engines. 2) Use rel="nofollow" on links to those PDFs ...


6

Because you've created a sitemap you can register with Google webmaster tools if you haven't already, and then submit it. (Site configuration -> sitemaps.) This doesn't guarantee anything, but increases your chances.



Only top voted, non community-wiki answers of a minimum length are eligible