Hot answers tagged

4

Google and Bing do not have a function such as Yandex to tell them what is the original source of the content. Google will generally go on the first version that was indexed and Bing is possibly the same. Google can get it wrong though, so if you see sites out ranking yours with your content, you can file a DMCA take down request: Digital Millennium ...


2

Here you get ALL user agents of Yandex bot (this is the Yandex own page, so you shouldn't worry about whether this information's true). Specially about your question's subject: yes, this is a mobile bot. Beside of it, there is another one mobile bot by Yandex. Its user agent is: Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 ...


2

Use a robots.txt crawl delay as described in http://help.yandex.com/search/?id=1112639 Example: User-agent: Yandex Crawl-delay: 2 # specifies a 2 second timeout Before you start banning this bot, you should first verify that your logs are actually Yandex and not someone else who is spoofing the user agent to look like they are yandex. A tactic used by ...


2

Certain browsers will always search for a favicon and will carry on doing so until you place the file. There is absolutely no reason why you should need to put a 307 or any other redirect method for that matter simply to get rid of a 404 error. 404's does not imply your site is broken It's important to note that errors 404 does not indicate that something ...


2

Browsers tend to cache favicons for longer than normal files, so trying to use it as your counter would probably make the counter quite unreliable. Even if it worked this would slow down your page unnecessarily, since you're adding a redirect into the mix. Just stick with the standard code, and if the favicon errors are a problem use one of the options bybe ...


2

I actually ran into this same problem a few months back on a site which forces SSL. I contacted Yandex support directly about this so I'll share their response: Hello, Please accept my apologies for the delayed reply. You should specify https domain as the main mirror in your robots.txt file and then wait for a couple of weeks ( ...


1

location = /favicon.ico { return 307 http://mc.yandex.ru/watch/XXX; } seems to work; however, it would seem like only certain browsers request a favicon.ico by default, so, not all visitors would be counted. Also, it has a limitation of not letting you know which pages are popular on your site: when a redirect is made, it is likely that the ...


1

There could only be two explanations for this behavior: The crawler found a link to that malformed URL, either on your site or on some other site. The Yandex crawler has a bug. If you aren't seeing that URL crawled by other bots or visited by real users, then I suspect it is a bug with the Yandex crawler. As for why Yandex would have that particular ...


1

You can set a main mirror in Yandex. I think you need something like this: Host: myhost.ru # uses User-agent: * Disallow: /cgi-bin User-agent: Yandex Disallow: /cgi-bin Host: www.myhost.ru # is not used or even: # domen.myhost.com is the main mirror # www.domen.myhost.com, is the correct use of # the host Host directive:User-Agent: * Disallow:Host: ...


1

In response to your key question point, can Googlebot be restricted from indexing certain portions of a page only the answer is not at this time. The only supported methods at this time for controlling indexing apply to full pages such as adding rel="nofollow" to certain links and using your robots.txt file to deny access to certain resources. As for what ...


1

Yes, the more search engines you allow to your site, the higher impression count you'll likely get with adsense. Now if the websites main focus is on images and the english text mostly describes those images, then the images speak for themselves for the international users, so the remote search engines can definitely raise your impression count there. It's ...



Only top voted, non community-wiki answers of a minimum length are eligible