By merlin99 on

I have just installed Drupal 6.5 on a machine and enabled the Aggregator and have a strange problem. It appears that all of the character entities in the feed items are being stripped out somehow. This clearly leaves all of the HTML tags incredibly broken. For example, this is what a feed item from drupal.org looks like:

div style=float: right; padding: 0.8em; background-color: #0174BB; font-size: 1.2em; margin: 0 0 0.3em 0.3em; text-align: center;a href=http://ftp.drupal.org/files/projects/drupal-6.5.tar.gz style=color: white; text-decoration: underline; line-height: 1.5em;Download Drupal 6.5/abr /a href=http://ftp.drupal.org/files/projects/drupal-5.11.tar.gz style=color: white; text-decoration: underline; line-height: 1.5em;Download Drupal 5.11/a/div p style=position: inherit;Drupal 6.5 and Drupal 5.11, maintenance releases fixing problems reported using the bug tracking system, as well as strongcritical security vulnerabilities/strong, are now available for download./p p style=position: inherit;stronga href=/upgrade/Upgrading/a your existing Drupal 5 and 6 sites is strongly recommended./strong There are no new features in these releases. For more information about the Drupal 6.x release series, consult the a href=http://drupal.org/drupal-6.0Drupal 6.0 release announcement/a, more information on the 5.x releases can be found in a href=http://drupal.org/drupal-5.0Drupal 5.0 release announcement/a./p pa href=http://drupal.org/drupal-6.5read more/a/p

I am really confused. Does anyone know what is going on and how to fix it?

⋅ Categories: Drupal 6.x

Comments

merlin99’s picture

merlin99 commented

I managed to reproduce the problem directly with a php program using the xml parser. I download an RSS feed and ran this program on it directly:

function element_start($parser, $name, $attributes) {
  print "START ELEMENT ".$name."\n";
}

function element_end($parser, $name) {
  print "END ELEMENT ".$name."\n";
}

function element_data($parser,$data) {
  print "ELEMENT DATA: ".$data."\n";
}

/**
 * Test code
 */
$fp = fopen("php://stdin","r");
$data = stream_get_contents($fp);
$xml_parser = xml_parser_create("iso-8859-1");

xml_set_element_handler($xml_parser, 'element_start', 'element_end');
xml_set_character_data_handler($xml_parser, 'element_data');

print $data."\n";
xml_parse($xml_parser,$data,1);
xml_parser_free($xml_parser);

The debug shows that the 'element_data' callback is getting the data in chunks with the character entities missing between them. I still have no idea why, but at least I am narrowing down the problem. For example, the following input:

<hello>this&lt;is&gt;a test.</hello>

Produced the following output:

<hello>this&lt;is&gt;a test.</hello>

START ELEMENT HELLO
ELEMENT DATA: this
ELEMENT DATA: is
ELEMENT DATA: a test.
END ELEMENT HELLO
merlin99’s picture

merlin99 commented

This issue as described by php bug #45996 is that the way php uses the expat style parsing routines does not work correctly with the more recent versions of libxml2. The suggested workaround in to build php with libexpat instead of libxml2.