3

In Google Webmaster Tools I've recently noticed several concatenated URLs appearing in the Crawl Errors. On investigation it seems Google is picking up these links from a page that shows a user's posts from the forum. I take the raw (non-HTML) text, truncate it then add "..." after it, but if a URL gets cut off I end up with something like this:

Here's a good link you might find interesting: http://example.com/page...

In other words, just some plain text that happens to look like a URL. GWT's Crawl Errors now shows http://example.com/page.. (note only 2 dots) as an error.

So, Google is obviously parsing and following unlinked URLs, but how do they treat them? Do they treat them like "nofollow", despite the fact they are following them - i.e. not counting as a "vote" or regular link?

Also, is there any solution to avoid truncating URLs?

Improve this question
1
  • A possible way to test it might be to make one such "link" to a site you control and see if it shows up as an incoming link or not.
    – John Conde ♦
    Commented Oct 27, 2011 at 15:12

2 Answers 2

Reset to default
2

Google picks up links wherever it can find them. Sometimes those links are good (and lead to new, previously unknown pages), sometimes they're bad, that's fine. You generally don't need to do anything special in that regard, unless you're seeing a lot of user-traffic to those URLs you don't really need to redirect them either. If these truncated URLs are found on your site, one thing you could do is to just show the host name instead of the truncated URL, but even that isn't really necessary from Google's point of view.

Given that we find links in all kinds of places and are primarily using them to discover unknown URLs, I wouldn't worry about whether or not they pass PageRank. I'll double-check on this to make sure that we're using them appropriately just to be sure :).

(On a tangent ...) One thing this also tells you is that Google is interested in finding and indexing more content from your website, so if there are parts that aren't being indexed optimally, you might want to make sure that they're easily findable through normal links on your site. A good way to check for the indexing status of parts of your website is to split the URLs into separate Sitemap files based on your site's logical structure. Google shows the number of submitted & indexed URLs per Sitemap file in Webmaster Tools, which might help you to pin-point potential areas that could be reviewed.

In general, finding legitimate 404s on a website (when the links are bad / truncated) is not a problem and does not count "against" a website. We published a blog post on this subject recently.

Improve this answer
3
  • Thanks for the insights. The truncated URLs are parsed as actual links on the posts themselves, but not on this "summary" page. AFAIK Google is indexing all my pages just fine. Main reason this is annoying is because it diminishes the usefulness of GWT's Crawl Errors section having these bogus "links" in there.
    – DisgruntledGoat
    Commented Oct 31, 2011 at 11:55
  • Yeah, I can see how that might make the crawl errors section messy. I don't see a simple fix for that part, but I'll forward your feedback to the team.
    – John Mueller
    Commented Oct 31, 2011 at 14:09
  • Thanks! If it's not too much trouble, could you also mention this issue with 404s? (I have seen a lot of posts regarding this in various places so they may already be aware of it.)
    – DisgruntledGoat
    Commented Oct 31, 2011 at 14:50
0

Having links with href values that end up nowhere usually results in a 404 error of a non existing page. Broken links are bad for your SEO. I don't understand why you would actually want to truncate the URLs?

Improve this answer
3
  • They aren't links, they are unlinked URLs as I thought I made clear. For example http://www.google.com <- that is not a link.
    – DisgruntledGoat
    Commented Oct 27, 2011 at 14:45
  • I've must have read your post 4 times before answering and I still didn't get it. Now I do understand. Its the first time i've ever heard of search engines following plain text URLs. Because of the volatility of the link length, using a redirection script for broken links wont do the trick. Do you really need to show the URL's in the plain text?
    – Luuk Barten
    Commented Oct 27, 2011 at 15:04
  • From a technical point of view it's much easier to truncate a string than figure out exactly where all the links are and/or risk having unclosed HTML tags if I did link them. But maybe I can regex them out of the snippet.
    – DisgruntledGoat
    Commented Oct 28, 2011 at 0:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.