Should I care about those non-user friendly links already in the index
and hope that Google will correct itself soon?
This depends on what you want search engine users to see in your search results: the clean URLs:
example.com/1234 or URLs with parameters:
Clean URLs are generally more understandable and recognizable to search engine users, hence they're more likely to click on them. Since you went through the trouble of creating them, they would likely be your preferred version to index for your search results.
You can indicate to Google which of the two types of URLs to index is the preferred one, and hence avoid duplicate content issues, through canonicalization, in which you'd add a rel="canonical" link to the
<head> section of the non-preferred URL's page pointing to the preferred URL.
If I should - how do I remove those wrong links and replace with correct ones?
To encourage search engines to drop the URLs with parameters in favor of the clean ones, you can 301 redirect the URLs containing parameters to the clean URLs using your web server (e.g., with an Apache .htaccess file), which will let search engine bots know they've permanently moved. Over time they'll enventually be dropped and replaced with the clean ones.
You should also add the clean URLs to your sitemap, and remove any URLs with parameters from it. Then you can resubmit your sitemap to Google. You might also use the Fetch as Google tool to trigger crawling for your site and the clean URLs.
Finally, you should also block the URLs with parameters from being crawled again by disallowing them in your robots.txt as covered here. For example:
/ indicates the root directory, and
* is a wildcard for anything up to
?id=, followed by anything after it.