If your listing page is supposed to list all these different products, then just shuffling their order of appearance is not going to affect your SEO score.
edit: to add more.
Do you use links to indicate the paginated content? Webmasters use rel="next" or "prev" to indicate that a url shows a specific page of some paginated content. If so, then a product that appeared in page 2 gets indexed there by search engines and when a user searches for that product he might be redirected to said page number, only to find that the product is not there because you shuffled it.
Even then, since we are talking about a listing page, this is a page whose content is supposed to change often. Even if you did not shuffle your products, adding and/or removing them, listing new ones or stop selling others, would change the content of the page. This is normal behavior for listing pages.
If you do use link to mark each page in your pagination, consider that constantly shuffling your products may produce scenarios where search engine result links direct the user to a cached search result for your site where what he was searching for was in page 5, but you shuffled it to page 2. This may indeed cause some confusion.
If you do not use pagination mark up, or you shuffle your products once in a while and not constantly, I believe it should not affect your SEO score.
In any case, why shuffle the products? You could have filters on the page for users to find what they need and let search engines cache the page. If you need to advertise new arrivals or whatever you could use other approaches to show your content (even random ones) without messing with your listing page.