Should totally disabling breadcrumbs have any serious impact on SEO?
I would say no, but I don't know if any of the major search engines would want or require you to have a breadcrumb or breadcrumbs
div in your website as an SEO standard.
"If any of the major search engines would want or require you to have... breadcrumbs... ?" The answer to this is 'no'. There is no requirement to have breadcrumbs. Therefore there's unlikely to be a serious effect on SEO. There'll certainly not be any sanction.
As many of the answers and comments say - the degree of SEO effect depends on how strong your site is currently on all manner of SEO factors. I have been able to improve a sites listing in SERPs by just implementing breadcrumbs. However, the site was incredibly weak in all other aspects.
It can help the search engines understand your site structure better and this can help with listing.
The most important thing here though is that visitors to your site tend to benefit from breadcrumbs. If you remove them, the behaviour on your site may change. Visitors may not find their way to things as much as they do now... I have a theory this could feasibly contribute to increased bounces... just a theory though.