Google measures core web vitals based on real-world metrics (like when users visit pages in their Chrome browsers). But what if a user browses from a location where network connection is poor? Does Google adjust for the network connection speed?
Google's Search Advocate, John Mueller, stated last April that users' connection speed does have an affect on your field data. In that video, he says:
If 90% of your users are coming from locations that are slow and essentially 90% of your users have this sub-optimal experience with your site, then that’s what will be taken into account.
That statement lines up with Google's documentation on core web vitals thresholds, which states that the 75th percentile value of all page views will be used to classify the site's overall performance.
There are issues inherent in this setup, which the person in the video brings up when discussing it with Mueller.
The rollout of core web vitals ranking still seems to be an evolving situation, and so it is possible that Google will take a more nuanced approach to last-mile latency. So, hopefully this answer does not age well.