Actually, responsive web design is really easy to implement in this day and age. Gone are the days when you needed to design two different sites, or compromise on your overall layout to make everyone happy.
With media queries in your css code, you can easily design things so that your visitors with desktop displays are still seeing the exact same site they've been seeing, while changing the layout for your mobile visitors so that they can also enjoy your content.
Here's a great (free) tutorial on the subject if you'd like to learn more about responsive design.
I was hesitant at first as well with my commercial sites, but the truth is mobile is now a huge chunk of the market and you MUST accommodate those users in some form or fashion or risk losing market share and visitors.