What steps do I have to take in order to make sure that I keep my search engine rankings after I redesign or update my site?
From practical experience we've found that beyond the usual 301 redirecting these following actions have resulted in a shorter (and on one occasion non-existent) Search Engine fluctuation:
Time the migration well away from your domain name expiry/renewal, so there is little ambiguity over it being a different site
Set all far-future HTTP expiry headers to the date/time that the website is to be changed, relative timestamps require a little fiddling to become absolute when they are close enough
Migrate slowly, backporting as much as you can so the site runs in parallel - Google has a learning process to "trust" your site - if all paths are suddenly 301's it sometimes backs off indexing only to come back a day or two later, during which time your ranking plumets
Have your new site structure up early (and in parallel) using sitemaps to usher search engines towards the correct URL paths - when it gets to the point that you're now using 301 messages to enforce the change, there'll be additional trust between the Search Engines and yourself
Remember that if you find that there will be duplicate URL paths, use the
Link: <http://www.example.com/my/new/page>; rel="canonical"
HTTP Header to signal your correct destination (there is a META tag version too).
If your linking structure/position is changing drastically the best advice is to ask "is it in the users best interest?"
If your URLs are changing be sure to do a 301 redirect so the search engines know that the old URL has moved to a new location and all incoming links for the old URL should be transferred to the new URL. A sample 301 redirect in .htaccess would look like this:
redirect 301 /old.html http://www.example.com/new.html
That's about all you can control when you redesign a web site. Because your HTML will be changing your rankings may change, too, as semantic markup very much affects your rankings. The best you can do is make sure you properly markup your content (use heading tags, etc) take advantage of every internal linking opportunity.
The biggest thing is to make sure that you know what all your current URLs are and then redirect them to the new URLs using a 301 redirect when you redesign. You may find that your rankings take a slight dip temporarily, but don't panic, that can sometimes happen as Google readjusts itself to your site. Wait it out a few weeks before doing anything drastic.
If you are able, you could also change your anchor href's to match the new URLs.