I'm trying to find real-world HTML pages that are actually valid (in the sense that they are displayed by modern browsers), but are not compliant with HTML specification and are considered invalid. Any thoughts, links etc. are very welcome.

P. S. I've tried using http://validator.w3.org/, and according to it many pages I've tried (randomly) contain tens to hundreds of errors. Are there really so many broken pages on the Web, or is this validator unrealistically strict?

P. P. S. The reason for this question is I'm writing a simple HTML parser, and I want to test it on extreme real-world pages to ensure it withstands most common errors.

  • This is a strange request, or maybe it's not clearly explained. The vast majority of pages on the web displays acceptably and most do not comply with the html specification.
    – Osvaldo
    Oct 29 '11 at 20:14
  • OK, so it is normal. That's one question answered. I'm searching for extreme cases of "normal" invalidity. Perhaps, some kinds of errors that only recently received support in modern browsers? And another question: which are the most common errors, which errors should any parser definitely support and ignore/correct? Oct 29 '11 at 20:24

[There's an actual suggestion at the end of this, but there are also a bunch of problems with your entire question that I think have to be worked through, so bear with me.]

There's no such thing as "unrealistically strict" here. The validator is not intended to be a liberal parser; it's a validator(obviously) and "valid" has a fixed technical definition. It tests against the rules set out by the specs and pages that don't follow those rules are invalid, end of discussion. Strictness is the entire point. It's also why, at least until recently with HTML5, we specified a DTD: the rules change sometimes and so it was necessary to point out which set your page was following. See my answer to a previous question for some notes on that.

So, yes. There really are that many messed up pages on the web, as far as validation is concerned.

Whether those pages display is a separate issue altogether; you're mixing concepts in your question(which is why your first paragraph actually doesn't make sense). They generally do display, because browsers tend to have extremely liberal parsers that bend over backwards to make do with anything thrown at them. That's because it's generally not considered fair to penalize the user for bad code on the site builder's part.

Beyond a few pretty trivial ones like case differences between matching tags(<p>...</P>), I don't personally think a list of "common" errors is going to be very helpful, or even really exists. That list is going to be different depending upon who you're looking at, and if you're planning on building a simple parser, sorry but it's doomed if you plan on pointing it at an arbitrary set of "extreme real-world pages." HTML coders–at varying skill levels–don't, for example, make the same errors as content producers. The writers for a site I manage occasionally have to insert bits of markup and make mistakes that I can't figure out how they even managed to type.

If your parser really has to be simple and custom, you would do better to tailor what it handles to what's common in the data you actually plan on feeding it. If you do have to parse arbitrary content from around the web, you should probably just use one of the existing parsers. Either way, though, the best source of errors to look for would probably be for you to examine one of those parser's test suite and crib from them. They've obviously already done the research. (But at that point, I'd again lean toward just going ahead and using that parser rather than writing your own.)

  • Thank you for your detailed response. Unfortunately, I really need to write a parser that's able to analyze any page from the Web. Why? It seems like there are NONE available, except for monstrous WebKit and Gecko, of which I can't make heads or tails (and those are not parsers but complete engines). Thank you for your idea of using the test suit, I really didn't think of it. Oct 30 '11 at 10:31
  • @VioletGiraffe: You want to write a parser, but not a "complete engine", because "there are NONE available". But what is the W3C Validator? The W3C Validator will analyze any page from the web and inform you where it breaks the standards and a few hints as to why. Which is just about what any 'parser' can do. If a page does not follow the standards (ie. is not valid) it is entirely subjective how a 'liberal parser' or 'engine' should handle it. Browsers vary considerably in this respect.
    – MrWhite
    Oct 30 '11 at 12:50
  • If you were to write a 'handle anything' parser how would you choose to handle non-standard markup? Bearing in mind that, since it is non-standard there is no right or wrong way to handle it.
    – MrWhite
    Oct 30 '11 at 12:52
  • By saying "complete engine" I mean Javascript interpreter core, css interpreter, renderer... Stuff like that. As for your comment on how to handle non-compliant markup - that could well be my original question, if I were to think a while instead of posting question right away :) oh, and by saying "none available" I mean there are no good (read - resilient) free parsers in C++. Yes, I'm quite a dreamer... Oct 30 '11 at 20:37

Not the answer you're looking for? Browse other questions tagged or ask your own question.