5

It happened several times that an automatically generated RSS feed broke due to the presence of HTML entities. This makes sense since RSS is not HTML but the issue is that it is generated from fragments of HTML articles.

As a simplified example, say we extract the first sentence of an HTML article to generate part of an RSS feed and that sentence contains and é (é). Now this appears in the RSS feed and therefore the entire feed is no longer valid. Not only does it it fail validation but most readers abort reading the RSS as they do not recover from the failure.

What is the right thing to do when creating RSS from HTML that includes HTML Entities?

There are a few approach that some to mind such as resolving the entity to a charset but would that be universal? Would an RSS in Unicode instead of ASCII work with readers? Or could the HTML entity definition somehow be imported into RSS? Other ideas are welcome. It should not matter but in this case the generated back-end is PHP.

3

I think a solution suggested by Tomalak should work universally. Ref: https://stackoverflow.com/questions/3805050/xml-parser-error-entity-not-defined

His solution was to convert character entities to corresponding numerical entities.

You could HTML-parse the text and have it re-escaped with the respective numeric entities only (like:   ). In any case — simply using un-sanitized user input is a bad idea.

All of the numeric entities are allowed in XML, only the named ones known from HTML do not work (with the exception of &, ", <, >, ').

I created a simple xml document and tested the solution. In first document I used é and in the second I used the corresponding numeric entity é. The first one returned error while second one was valid.

I. Invalid

<?xml version="1.0" encoding="UTF-8" ?>
<rss version="2.0">

<channel>
  <title>Test XML Title</title>
  <description>&eacute;</description>
</channel>

</rss>

II Valid

<?xml version="1.0" encoding="UTF-8" ?>
<rss version="2.0">

<channel>
  <title>Test XML Title</title>
  <description>&#233;</description>
</channel>

</rss>

References:

  1. https://www.w3.org/TR/html4/sgml/entities.html

  2. https://stackoverflow.com/questions/11176752/converting-named-html-entities-to-numeric-html-entities

1
  • Fantastic. The answer from the last link worked exactly as intended.
    – Itai
    Jun 13 at 12:47
2

RSS (and Atom) are based on XML, so the issue you are having is that the & in the HTML entities isn't valid.

Typically you can solve this by double encoding the entities so that the & becomes &amp;: &amp;eacute;.

Most RSS readers will then unencode the first encoding, returning it to the original which is then handled by the HTML viewer.

Alternately, you could update your site and feed to declare that they are using UTF-8 and just go with an é everywhere (you'll still need to encode any actual & characters you use either in copy or url query stings).

4
  • Would it not simply complain that &amp in &amp;eacute; is an unknown entity and this invalid? I'll check it out but unless I am missing something, I expect the same error as before just on a different character.
    – Itai
    Jun 10 at 23:50
  • 1
    &amp; is a valid entity in XML. The eacute; after it would not be part of the entity to the XML parser. Jun 11 at 8:20
  • I'll admit that I've got a bunch of warnings for my feeds, but you can see various examples of standard and double-encoded HTML entities in doodle.uk/blogs/rss - things like &gt; are fine, and then there's quotes &amp;quot;, and the feed is valid Jun 11 at 9:16
  • Yes. You are correct about the &amp; - It is a pretty clever trick and indeed does get rid of warnings. The issue that I am still seeing is that some readers that are not embedded in a webpage just leave the HTML entity name, possibly because they are not interpreting HTML entities.
    – Itai
    Jun 11 at 21:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.