This has come up at several ends.

FeedAPI node stores feed item titles and descriptions without decoding them. On presentation, they are HTML encoded through check_plain().

The result are HTML escape strings (e. g. &) in the output, most notably in titles.

This patch uses html_entity_decode() to decode HTML characters before storage.

Comments

alex_b’s picture

StatusFileSize
new2.16 KB

This patch includes decoding the title element of the feed itself.

Büke Beyond’s picture

Thank you for the patch!

This is a critical issue. I have encountered it very often.

The node title is saved as a plain string, not as html.  If you enter '&' in the node title manually, that is how it gets serialized into the db, as a plain string.  Drupal upon rendering this plain title string, it will html entity encode it (as '&') for the browser.

If you manually save a node title as '&', Drupal will render that as '&' for the browser, and on the browser you will literally see '&' displayed exactly as you entered it.

Things are different for the body. If you enter '&' into the body, Drupal will leave that alone during rendering, and the browser will show a '&'.

So yes, for the title, FeedAPI definitely must html_entity_decode() the source title.  Probably even strip the tags; html_entity_decode(strip_tags($Feed_Title), ENT_QUOTES);
alex_b’s picture

Status: Needs review » Needs work

Code needs work.

- html_entity_decode() should specify ENT_QUOTES and 'UTF-8' as parameters
- decoding needs to be moved to parsing stage

aron novak’s picture

Status: Needs work » Fixed

Please test the status now. I committed the changes for both parsers.

Anonymous’s picture

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.

alex_b’s picture

Status: Closed (fixed) » Active

Aron - see #3 - we need to work on this as the proposed patch isn't the right way to go.

If we move the encoding handling to the parsing stage all following stages shouldn't need to worry about html_entity_decode().

alex_b’s picture

Hmm. I'm seeing now that you actually didn't commit this patch but rolled your own. Could you please clarify?

aron novak’s picture

Status: Active » Fixed

Yeah, i committed the patch what follows the good way.
- html_entity_decode() should specify ENT_QUOTES and 'UTF-8' as parameters
Done. But at parser_simplepie, it's complicated, because under php5 version, no way to define "UTF-8" param.
- decoding needs to be moved to parsing stage
See _parser_common_syndication_title() and _parser_simplepie_title()

Büke Beyond’s picture

Title: HTML escape strings in output - encode HTML before storing. » Not fixed

I have been using hook_nodeapi ($op == 'presave') to decode and fix the titles.

I just tried the latest build feedapi.module,v 1.23.2.119.2.29 2008/10/27, and bypassed the hook, with the following feed:
http://www.clipsyndicate.com/rss/feed/1778

The escaped ampersand codes are back on the titles!

So this issue is definitely not fixed. I have to go back to relying on the custom hook. For other users, this may not be as viable.

Büke Beyond’s picture

Status: Fixed » Active
Büke Beyond’s picture

Title: Not fixed » Title Decoding - Not fixed
Büke Beyond’s picture

Title: Title Decoding - Not fixed » Title Decoding - Found Problem
Status: Active » Needs review

$Id: feedapi.module,v 1.23.2.119.2.29 2008/10/27 10:50:26 aronnovak Exp $
Line: 1019

$feed->items[$i]->title = filter_xss($feed->items[$i]->title, $allowed);

Upto that point the title is decoded correctly, as recently fixed in the parser modules.
But, filter_xss on the title re-encodes the ampersands, etc, and reintroduces the problem.

Since the parsers are now responsible for removing all the title html tags and decoding the entities, this line 1019 should now be removed.

aron novak’s picture

Status: Needs review » Fixed

Thank you for the debugging, the problem was exactly what you've identified. I chose different way to fix it but i tested and now it's fixed.

toemaz’s picture

Would there be a possibility to backport this for the 5.x release?

fumbling’s picture

Hi Alex - I'm on FeedAPI 6.x-1.4. To fix this issue, which I'm experiencing as well, should I install the feedapi_decode_html.patch you attach to your 10/2 post, or a more updated one? Wasn't quite sure from the thread here. Thanks either way.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.

kevinwalsh’s picture

Component: Code » Code (general)

Indeed, can this solution be backported for 5.x?