Looking at your setup...
Identical desktop and mobile URLs, no subdomain or sub-folder:
Desktop URL
site-name.com
<link rel="canonical" href="index.php?/...">
Mobile URL
site-name.com
<link rel="canonical" href="index.php?/...">
I see an issue. you're confusing the heck out of google. you need to create an association between desktop and mobile site, not indicate that neither is original.
Here's what you do
What you need on your desktop pages is the following between <head> and </head>
<link rel="alternate" media="only screen and (max-width: 111px)" href="http://mobilesite.example.com">
...but change the 111 in 111px to the maximum number of pixels the remote device connecting your site should have for you to recommend the mobile site. Change http://mobilesite.example.com to the full URL to the mobile equivalent of the page you put the above HTML code on.
Then on the mobile version, you use:
Of course change http://desktop.example.com to the full URL of the desktop equivalent site.
Once you do that, configure your server so mobile devices automatically are redirected to the mobile version of your site, then you can test the desktop version of the page in page-speed insights.
Mobile Usability
Touch elements too close 307
Content not sized to viewport 306
Small font size 170
Viewport not configured 170
If you're getting a host of errors like this, then include this between <head> and </head> on all mobile pages:
<meta name="viewport" content="width=device-width,initial-scale=1">
That will set the window to the actual mobile device size. then when you get errors about things that are too wide, you'll know the width limit.
How do I tell googlebot-mobile to leave the desktop content alone?
Earlier, you mentioned "...Which is a link on the mobile layout to force-switch visitors to the desktop version.".
Don't make links to incompatible pages too easy for robots with limited capabilities to access
The problem is that the link is on at least one mobile page that anyone including googlebot can access.
Since google doesn't touch post-based data, what I'd suggest is to create a special warning page telling users who insist on the desktop version what they really are getting into, and if they want to continue, then they can select a button that will take them in.
This is an example of how to make the button:
<form action="/path/to/url-to-desktop-switcher" method="POST">
<input type="hidden" name="mydata" value="1">
<input type="submit" value="switch">
</form>
The reason why I suggest this is because when user clicks the button, the data is sent to the server via POST method, not the GET method, and bots normally don't click form buttons.
I added a hidden form item so that data can be passed into the script when the button is selected. The good news is that the button can be styled, but the bad news is that the text in the button can't have too many characters or it will be too large for the mobile screen.
