Tag Info

New answers tagged

1

Is there a way you can take advantage of iframes? Google is able to just look at the source of an iframe, but it can't crawl an iframe directly into the parent page, as far as I know. So, the iframe source should be noindex.


5

robots.txt can block JavaScript files from Googlebot. http://www.robotstxt.org/ has more information about how to construct a robots.txt file. You could put your JavaScript that shows the password into an external JavaScript file (called showlists.js): $(document).ready(function(){ showLists(); }); Call that JavaScript file in the page head: ...


1

I created a cloaking function in JavaScript: function isBot(){ return /bot|crawl|slurp|spider/i.test(navigator.userAgent) } Then I use that function to either show the passwords onload, or to show a message saying why no passwords were generated: if (isBot()){ $('#isbot').show(); } else { showLists(); } Now when I use the "Fetch and Render" ...


2

I suspect you are simply misunderstanding the "renders the pages" phrase. Googlebot stills views pages as raw text, it doesn't see pages as "text rendered into pixels". Googlebot is running the javascript/css/whatever it finds to discover(aka "render") text that was previously not visible to it in the raw html text source (ie because it was pulled in via ...


-1

Remove the old sitemap and submit new one. For fast indexing, use bulk ping services for for all URLs. If you want to remove duplicate URLs from current site map, then use Microsoft Excel to this job.


1

I can agree that it definitely is NOT google at all. This webpage http://www.whois.com/whois/185.62.188.98 tells me its from a system out in ukraine called blazingfast. Before downright blocking the IP, you have to figure out what types of people you want to welcome to your site. It's quite possible that someone on that network simply has a hijacked ...


2

No. It is not Google. Google would not POST /xmlrpc.php. This may be a vulnerability probe or an attack against an existing vulnerability that may or may not exist on your site. 185.62.188.98 is hosted-by.blazingfast.io. When you see something like hosted-by as a sub-domain, it is a webhost, obviously, but also an anonymous address block where the server ...


1

Yeah. That is the wrong approach. Just replace your sitemap. Keep life Simple! Honest and legitimate bots won't fully spider your site just because you updated or replaced your sitemap. They are smarter than that. The decision to visit a page for a search engine is based on metrics in their database and not an updated sitemap. I update my sitemap often. ...


1

I spoke with John Mueller regarding robots.txt, URL parameters, canonicals, and noindexation. Using 'URL parameters' in GWT is a strong suggestion to Googlebot, but not absolute. Googlebot still spot checks the URLs, so depending on the number of URLs the spot check might be fairly visible. Also, since we had 'URL parameters' set to not crawl search ...


0

This may not directly address your link count warning, but there is something you said: The facets are using rel="canonical" to the non faceted version and set to 'No URLs' via URL parameters Some of the canonicals are indeed using facets. Have you thought about changing your URL parameter tactic? You can "teach" Gbot how to use the facets in GWT > ...


2

This is not an issue that needs to be resolved. Any site that has a large number of pages on it gets this message. Google tells you this in case you accidentally published URLs, especially ones with duplicate content. As long as you mean to publish your URLs and are handling any duplicate ones appropriately, this is not an warning that you need to pay ...


1

It is only supported by Google Search Appliance and is not officially documented by Google to be used for Google indexing so most likely will not work on Google. However, Quora seems to use Googleon/Googleoff on their site for the purpose you want to use it for.


1

Do you have any .htaccess redirects other than the default WordPress redirects? Have you moved from another website recently or changed URL structure? No harm running a quick crawl of your site with Xenu or Screaming Frog. Sometimes you find weird stuff in Webmaster Tools usually from other people's websites, inbound links to pages that don't exist etc. It ...


1

You could use Javascript to modify the head, but as search engines don't generally run Javascript that wouldn't work. What you probably want to do is PHP output buffering as described in the accepted answer to this question. By the time you get to the end of the page you know what needs to go in the head. Then you can output the head and release everything ...



Top 50 recent answers are included