Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

Just checking GWT today and it's suddenly showing 1,379 crawl errors (404) but for the most non-sensical URL's that don't exist, for example...

www.example.com/services/newsletters/services/contact

The only changes I've made recently have been setting the preferred domain to 'www' and updating the .htaccess file to redirect to 'www' as Google was showing some results with 'www' and some without.

This is my .htaccess file

<IfModule mod_rewrite.c>
    RewriteEngine On

    # Trailing slash enforcement

    RewriteBase /
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_URI} !(.*)/$
    RewriteRule ^(.*)$ http://www.example.com/$1/ [L,R=301]

    # Returns WWW

    RewriteCond %{HTTP_HOST} ^example.com
    RewriteRule (.*) http://www.example.com/$1 [R=301,L] 

    # Removes index.php

    RewriteCond $1 !\.(gif|jpe?g|png)$ [NC]
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteRule ^(.*)$ /index.php/$1 [L]

    #Redirects old static .html pages to EE CMS

    Redirect 301 /services.html /services 
    Redirect 301 /take-away.html /services/take-away
    Redirect 301 /blog.html /blog
    Redirect 301 /about.html /about
    Redirect 301 /pin-up-studio.html /pin-up-studio
    Redirect 301 /press.html /press
    Redirect 301 /links.html /links
    Redirect 301 /newsletter.html /newsletter
    Redirect 301 /events.html /events
    Redirect 301 /contact.html /contact
    Redirect 301 /terms-and-conditions.html /terms-and-conditions
    Redirect 301 /parties.html /services/party-package
    Redirect 301 /gallery.html /gallery
    Redirect 301 /bridal.html /services/bridal-styling
    Redirect 301 /404.html /404

</IfModule>

Any ideas what I'd done wrong?

EDIT

So another 379 crawl errors have appeared in the last day. When I check the 'linked from' status it says it is being linked from the website itelf. An example is this URL is not found...

http://www.example.com/services/contact.html/bridal-styling/bridal-styling/services/party-package/take-away/contact/pop-up-parlour/bridal-styling 

...and lists these 2 URL's as the 'linked from'

http://example.com/services/contact.html/bridal-styling/bridal-styling/services/party-package/take-away/contact/pop-up-parlour

http://www.example.com/services/contact.html/bridal-styling/bridal-styling/services/party-package/take-away/contact/pop-up-parlour/

As I said before the only changes I've made recently have been setting the preferred domain to 'www' and updating the .htaccess file to redirect to 'www' as Google was showing some results with 'www' and some without.

My .htaccess file is as above.

It seems the bots seem to have got stuck in a loop when crawling the site? Not sure why it's getting hung up on the www and non www as I've specifically stated in GWT to use www.

share|improve this question

1 Answer 1

up vote 3 down vote accepted

Click each of the URLs listed in turn, and check the "Linked from" tab, it will tell you how Google found those URLs.

It's likely to be a bad link from another site. I've had problems with spammy link directories/search engines listing truncated URLs that don't exist, e.g. http://example.com/pag...

share|improve this answer
    
As well it could be some Black Hat SEO to try and take down your site a few notches. –  Christopher Aug 14 '12 at 8:58
    
@Christopher doing what exactly? Linking to pages that don't exist doesn't affect your ranking. –  DisgruntledGoat Aug 14 '12 at 9:11
    
Those URLs seem arbitrarily generated... just looks too fishy to be a legitimate mistake. –  Christopher Aug 14 '12 at 9:15
    
@Christopher sure, but what exactly would they be trying to achieve? Like I said, linking to pages that don't exist doesn't affect your ranking. –  DisgruntledGoat Aug 14 '12 at 9:18
    
In my experience, if it doesn't seem legitimate, it's not. It could have been a half-assed attempt at a Black Hat bot, or trying some kind of new social engineering hack in the wake of the Apple/Amazon snafu. Just because you haven't seen the exact same symptoms before doesn't mean it's harmless, especially if you have user information you're trying to protect. You don't know of holes in your security until someone jumps through it, keeping a protective mindset helps to prevent social engineering. –  Christopher Aug 14 '12 at 11:11

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.