New answers tagged

1

What I would suggest is to create a brand new set of URLs for search result pages and then for anyone requesting the old URLs, produce an error page with an HTTP 410 status code. Also, make your search pages only accessible via the POST request method. Google won't crawl pages that are requested via POST if it is requested as a result from filling out a ...


1

Google supports Noindex: in robots.txt as an experimental feature. It sounds like this would be the perfect case for using it: User-Agent: * Disallow: /search/ Noindex: /search/


1

Google won't crawl the disallowed path specified in robots.txt but you can't control through robots.txt the references to your site search results from other sites. While Google won't crawl or index the content blocked by robots.txt, we might still find and index a disallowed URL from other places on the web. As a result, the URL address and, ...


1

I recommend your option of using the URL Parameter Tool in search console. For each of your tracking parameters, add it to the tool and set it to "Doesn't effect page content (ex. tracks usage)." When you tell Googlebot that these are passive parameters, Googlebot will usually crawl just one URL with a specific parameter value. If the parameter can be ...


0

There is no way to tell Googlebot to forget about something it has crawled. Your only recourse is to: Fix the problem with your HTML. Redirect any faulty URLs that were caused by the problem. Wait until Googlebot has recrawled all the pages with faulty HTML and all the bad links that those pages might have generated. NOARCHIVE prevents Google from ...


1

But, without Googlebot being able to crawl the live URLs on pages, the way they're appearing on pages, it won't be able to follow the links and give them any weight, authority and relevance to these pages. Is that right? In theory yes, but my personal interpretation of having backlinks to pages that are not accessible to search engines is that they do play ...


0

411 Pixels Wide. This was changed as of April 16, 2016. It will likely change again.


0

Does your webpages deserve to be indexed? Do them give something useful to an user? If not, Google won't penalize your website, it will completely ignore those pages, meaning they won't be indexed at all.


0

It does this by following : checking sitemap for new links on existing indexed pages, once they gets re-indexed, all links on that page are crawled/indexed. So you can put new links on existing pages for the visibility how fast that happen depends upon crawling rate of your site. It might take few hours to few days before Google start indexing these new ...


5

Google, conceptually, uses an HTML DOM parser. What this does is break any web page HTML down into its basic structure and each HTML tag is given an ID. This ID represents the order of the HTML tags from beginning to end, any dependency between HTML elements such as a li tag is dependent upon a ul tag, any parent-child relationship between HTML elements ...


1

If you have a new page, you probably have a link to it. Google visits your page often and will catch the link. If it's a matter of new content, that may take longer because I don't think Google examines content as often as it examines links. This is why they recommend a sitemap. When you have a significant change to a page, you can update your sitemap and, ...


2

"Cloaking" in this situation would be fine. When the user agent contains bot|crawl|slurp|spider you should not use session id parameters or check for cookies. You are delivering the same content to users and bots. Google won't have a problem with this particular cloak. I use a similar technique for deciding whether or not to use Data URI for the images ...


2

When Google determines that your site is having problems, it will slow down the rate at which it crawls. If your site is less responsive, or is giving Google 500 errors, Google will make fewer request per day to your site as a whole. Googlebot will never give up on any particular URL. Google will come back and try that URL again. How soon it comes ...


1

There is no easy way to measure this as there doesn't appear to be a hard limit and Google is so complex it's difficult to measure. However, the more intermittent the service the lower an URL will rank as it accrues a performance penalty. This would lead me to guess that it's based on a curve rather than a binary 'indexed' or 'not indexed'. The only thing ...


1

Use Hreflang codes to indicate to Google which site it should index for which region. You'll find it then indexes all those links and presents them according to the engine the user uses to access the site. This largely replaces the need to identify a users location - even using a good service, it's going to be wrong if a user is behind a VPN or is part of ...


1

This article suggests that Google will cache content loaded by an onscroll event: http://dinbror.dk/blog/lazy-load-images-seo-problem/ Admittedly that is for images, but I can't see why it wouldn't work for other content. I would test though.


0

Titles are supposed to be unique for each page. That's what titles are for! To differentiate one page and its contents from another. Does Wikipedia concern itself with different titles on its millions of pages? No.


2

I'm head of SEO for a website with a similar structure to your own: The with large amounts of content coming from the same template but being filled out by Event details. Google will choose whether a page is worth indexing or not. If it deems a page too sparse on content, it will drop it from the index. There are penalties for bad linking and bad content. ...


2

Google would be happy to index billions of very unique web pages, but it hates indexing multiple URLs that point to exactly the same content. For your match pages, you need to make sure the information is unique enough. If one page has this content: This football event just happened between (insert team name here) and (insert other team name here). Here ...


0

Using your example, www.example.com/dashboard versus www.example.com/dashboard?tab=something_public versus www.example.com/dashboard?tab=something_else_public, regardless of the resulting HTML, Google will see these as separate pages. The reason for this is simple. The URL is one of two major keys within the index. As long as each URL is unique, it is a ...


2

The AdsBot crawler with user agent AdsBot-Google (+http://www.google.com/adsbot.html), is part of AdWords not AdSense. It is supposedly used for checking landing page content to determine quality score. This bot appears to also visit pages that are not ad landing pages, including pages with no ads on them, no ads to them, and no links to them. Based on the ...



Top 50 recent answers are included