Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I have a site with over 300,000 products. Many of these products have non-alphanumeric characters in their names. I have encoded these, rendering thousands of links that look like this:

product/SPARKLE+POWER+INTERNATIONAL/FSP180-50NI%28V%29/316925

note: this equates to: products/[manufacturer]/[model#]/[site specific ID]

However, Google webmaster tools thinks that the url should be reviewed like this:

SPARKLE+POWER+INTERNATIONAL/FSP180-50NI(V)/316925

Which returns a 404 error

To simplify things, I want to change all non-alphanumeric characters to '~'. My site navigation will still work, but I am afraid that I will lose a lot of search ranking and search page links because of the url changes. What is the best way to keep this from happening?

Also, is their a way to make Google webmaster tools understand urlencoded URL elements (double encoding?)

share|improve this question
add comment

1 Answer 1

I understand your dilemma, you sated it well, but what I do not know is where you are in the process. I will work on the assumption that nearly half of your pages are getting a 404 error. I also do not know what systems and applications you have, your skill level, etc.

For the pages that are not getting a 404 error, you can quickly change whatever scheme you are using to update pages. I am assuming you are using a 301 redirect. I would work on this first to stop the hemorrhaging by quickly fixing the 301 redirect (or whatever mechanism) to the proper encoding scheme you have.

For the pages that are getting a 404 error, the harm is already done but it's not too late. I would quickly create the pages that Google thinks you have, and do a 301 redirect to the proper encoding scheme you have.

If it is too difficult to figure out what pages have 404 errors, then I would quickly create all new pages that Google thinks you have and redirect them to your proper encoding scheme and create these pages as well. This may actually be less work and faster to complete. If you know a scripting language, then this may be really easy.

We have all done these kinds of things. I have done them and I have more than 300k pages. It may be work, but I think this problem is simple even though it seems like the end of the world. It will take Google some time to process all of the pages. It may take months. Meanwhile, the system should work okay for your users if done right. Once all of the pages appear in the index correctly and redirect traffic seems to settle down, then you can remove any redirect. At this point, you are done.

share|improve this answer
    
Agreed; set up redirects for everything ASAP. I handled a similar situation by using a script as the 404 handler -- from there I could look up / calculate the actual new URL and redirect appropriately (this sometimes makes it a bit easier to setup). Also: log the referrer, so when a user views an old URL, you have the source-page & can fix it when it's within your own site. –  John Mueller Mar 16 at 13:15
    
Google is showing around 14,000 pages that are not working, which is better than half. I am using PHP and Codeigniter, so I can change all of the links at once without breaking my site. However, because the character that Google is mis-reading a slash, writing a script to handle the redirects is really hard. Besides, I don't want Google to index the these mistakes at all. Sounds like I just need to fix the url and hope Google indexes that improved URLs as fast as possible. –  Hoytman Mar 17 at 18:50
    
Yeah. Sometimes it is a cut bait and fish scenario. Good Luck! –  closetnoc Mar 17 at 18:59
    
Do you think that double encoding the URLS within the site map would fix the problem with the slashes? (here%2Fthere becomes here%252Fthere)? –  Hoytman Mar 17 at 19:02
    
I would assume that if the double encoding works in the browser, then it should work in the sitemap. Good idea BTW!! I forgot about double-encoding. This may speed things up since Google seems to like sitemaps for larger sites. Just make sure it has a new creation date stamp (within the file system). That may help and certainly would not hurt. –  closetnoc Mar 17 at 19:07
show 2 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.