Is there any way that too many HTML comments in a document can effect search engine rankings?
<!-- This is a comment -->
Do the spiders ignore all of them?
They are completely ignored so no, it doesn't affect rankings. The only problem might be if you have such a massive amount of text in comments, that it increases the load time significantly. Users might go elsewhere.
If you're using a server side language, you could always put some comments inside comment tags in that language. I do this for comments I don't want outputted, or for situations where I'm taking down a section and don't want people to be able to view it like so...
<?php // php comment example /* <strong>DON'T WANT ANYONE TO SEE THIS</strong> */ ?>
Your HTML page will be smaller, but your PHP process will be interpretting more... not sure which is more efficient, so for me it comes down to need.
This is a debate within the SEO community but I think this can be an issue if overdone. While things like comments and scripts make no sense to a search engine spider, if over done they can still affect the code to content ratio which can dilute the effect of any optimization as the content which does affect rankings starts to be interfered with by the noise of scripts and comments that don't.
One of the worst examples of this I have seen is with the __VIEWSTATE field in ASP.NET with some CMSs that leave a massive block of encrypted gobbledygook in the middle of the markup to maintain state with the server.
As a rule I wouldn't worry about code to content ratio too much but definitely adhere to best practice (ask yourself what would Chuck Norris do and don't over do scripts/comments) and remember that the limit of the Google cache is about 101kb per page and this should be considered the absolute limit. Spiders will give up on anything larger than this.
There is a good discussion in this SEOMoz forum thread.