The CrUX Dashboard can show you historical data for your origin-level performance. It's based on the Chrome UX Report (CrUX) dataset, which is also what PageSpeed Insights uses. The dashboard includes coarse distributions of user demographics like effective connection types and device form factors.
For finer-grained insights about which types of users are experiencing slow performance, you can use the public CrUX dataset on BigQuery. There are some sample queries in the CrUX Cookbook repository. For example, you can look up the 75th percentile LCP value for 4G mobile users in the US:
SELECT
`chrome-ux-report`.experimental.PERCENTILE(ARRAY_AGG(bin), 75) AS p75_lcp
FROM
`chrome-ux-report.experimental.country`,
UNNEST(largest_contentful_paint.histogram.bin) AS bin
WHERE
yyyymm = 202001 AND
country_code = 'us' AND
origin = 'https://www.example.com' AND
effective_connection_type.name = '4G' AND
form_factor.name != 'desktop'
Source
Using these tools you should have a better understanding of the user demographics on your site and how they correlate to performance.