Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I recently had to reconstruct the way my website runs. Along with that I have to change the way my urls are formatted. My website is not terribly popular (hoping changes will fix that) but I do have 500+ pages indexed on just Google.

My question is should I/can I do anything about removing these dead pages from Google and other search engines? What should I do about backlinks?

share|improve this question
add comment

migrated from stackoverflow.com May 20 '11 at 6:55

This question came from our site for professional and enthusiast programmers.

2 Answers

In an ideal world, you'd set up a rule that sends 301 Moved Permanently (sorry, it's a text doc, so you'll have to search for the linked phrase) responses when somebody asks for one of the old pages. Depending on your situation/constraints, that may or may not be feasible, but it's the "best" in terms of trying to keep your existing Google juice and making people who've bookmarked/linked to your pages happy.

As far as Google-specific things go, you may want to check out their webmaster tools to see if they have anything targeted at this specific use case.

share|improve this answer
add comment

The best option is always to leave the URLs unchanged. URLs should never reveal the technology that's running the website; it should merely identify the content. This question on StackOverflow, for example, includes a question ID and a "slug" describing the title, but no indication of what technology might be used to retrieve it.

However, sometimes you have no choice, and the URLs simply have to change to compensate for earlier mistakes. In this case, the best option is to leave behind 301 redirects from the old to the new address. With Apache's mod_asis, you can create an index.asis file which is the "as-is" response to the HTTP request — i.e., complete with headers. And thus you can do

Status: 301
Location: http://www.example.com/new/page/

The 301 status code is important, since it tells crawlers the page has "Moved Permanently", and most will then update their links accordingly.

If you're doing this in bulk, mod_rewrite may be better suited. You can create a rewrite rule that performs the same sort of redirect for all pages matching an old-style URL into the corresponding new-style URL. This, of course, demands that there be some pattern to the URLs (or a script that can somehow map old to new).

Either way, the redirects can then remain in place permanently, so even severely antiquated links may continue to work.

Anything short of that leaves open the potential to lose a visitor, since you cannot be sure where old links to your site may lurk. One might, for example, be in my personal bookmarks, which are stored only on my local computer — and you'd never know until I click it to revisit your site some day. So any effort to specifically find and fix old links is necessarily incomplete.

Having said all that, if you absolutely cannot keep the existing URLs and you absolutely cannot use 301 redirects... then yes, you should have Google specifically crawl your site again, which will automatically detect the dead links and discover the new ones.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.