Handling duplicate content caused by 'new' smart-url structure

Question

I used to have an URL structure like this:

mydomain.com/search.php?key=value 

which I've since changed to:

mydomain.com/key/value.

I've changed all the internal links in my site, but Google still holds results which point to the old URL structure, which makes me concerned that when crawling the new links, they'll be considered duplicate content.

Current thought is to add a wildcard Disallow in robots.txt to the old URL structure.

Disallow:/search.php?key=*

Will that resolve the duplicate content issue I might be having?

Answer 1

Do a 301 redirect from the old URL to the new URL. Google will then replace the old URLs with the new in their index. It won't happen quickly, but it will happen.

Answer 2

Disallow:/search.php?key=*

Will that resolve the duplicate content issue I might be having?

That will only tell robots to not crawl any URL's containing mydomain.com/mysearch.php?key=, but it does not tell search engines to remove these URL's from previously indexed search results.

The first thing to check is if the older URL structure is still being served or not. If you only want the new URL structure being served, then make sure that your web server/PHP returns requests for URL's containing the parameter key=value as either 404 or 410 errors, so they will no longer be indexed. See this for more on that: Removing an outdated page or link

Alternatively, so that the previously indexed links with the old URL structure still work when people click on them in your SERP, you can do a 301 redirect using PHP to parse the value parameter from the old URL structure to redirect them to the /value path of the new URL structure.

If it's still desirable to serve URL's with the mydomain.com/search.php?key=value structure too, then you should add a canonical URL to the pages with the new mydomain.com/key/value URL structure to avoid duplicate content issues.

Lastly, make sure that URL's with the older URL structure no longer appear in your sitemap and submit it to Google.