The way that you're talking about changing your site, if anything Google would support.
By improving your website's UX, you are increasing the likelihood that potential customers of Google's clients' will have a pleasant Ad experience.
Adsense might even give you a cookie.
I couldn't help myself...sorry
As long as doing so doesn't mess with the actual Adsense code, or modify content in a way that violates the Adsense publisher policy, you're fine.
Don't tamper with the AdSense code.
Once you've generated your code, we ask that you do not alter any portion of the code or change the layout, behavior, targeting, or delivery of ads for any reason, unless specifically authorized to do so by Google.
Full list: Adsense Policies: Beginners Guide
Despite the rather absolute verbiage of the above guideline, there are some things you can do. For example, in the spirit of improving your UX, you may also be interested in implementing Responsive Ad Units.