htmlspecialchars() error with views created by AF

I have exactly the same error using Drupal 6.24 and Advanced Forum 6.x-2.0-alpha4+30-dev.

Referrer: http://example.tld/forum/unanswered
Message: htmlspecialchars(): Invalid multibyte sequence in argument in file /includes/bootstrap.inc in line 860.

This error occurs only on AD views pages.

I'm afraid this is very difficult to debug without the corresponding multibyte / unicode content.

Would anyone experiencing the error be willing to (privately) share a database dump with me so that I can have a closer look?

I've been looking at this problem for days and I didn't find the solution for it. I'm running Drupal 6.24 and advanced forum 6.x-2.0-alpha4. The website is in spanish language.

I have the error "htmlspecialchars(): Invalid multibyte sequence in argument in file /includes/bootstrap.inc in line 860. " when the forum is displayed.

I've realised that the errors come from some of the views of the forum. For instance, "forum/active". It shows a table with the header, topics, answers, views, ... And then a couple of errors occur per topic displayed. Not sure if the problem comes from the trimming of the topic header.

I did look at the code searching for any substr or similar but nothing suspicious. I wonder if the problem is in the advanced_forum.js. I can't tell cos I'm not good at javascript.

Any help with this will appreciated.

There is a lot of info in this issue. I don't have time to go through it all but maybe something there can help. #837322: htmlspecialchars - Invalid multibyte sequence in argument

Would anyone experiencing the error be willing to (privately) share a database dump with me so that I can have a closer look?

I can give you a full DB dump of my site, which has two error with AF: aforementioned "warning: htmlspecialchars(): Invalid multibyte sequence in argument in /www/istmat/soft/drupal/includes/bootstrap.inc on line 860" error and unknown HTTP 500 error (without any error messages in logs of web-server and php-fpm). All errors are reproducable on the same URLs. Dump of DB is about a 30 Mb in tgz.

Thanks RedRat, could you PM me with a link to your db dump on drop(box|send) or similar please?

If possible it might help if you could also provide the output of drush pm-list (if not, I should be able to get the info from the db dump).

This has been an interesting one. Thank you to RedRat for providing me with a db dump with Russian content where the problem was reproducible.

As far as I can see the problem is actually in views, and only arises (visibly) in versions of PHP >= 5.2.5 and < 5.4.0, and with certain mulitbyte content.

The reason that the error may not be apparent in PHP < 5.2.5 is the version_compare in check_plain. If the PHP version is earlier than 5.2.5 check_plain does a check for valid UTF-8, and if this check is failed, the text is never passed to htmlspecialchars.

It appears that despite what it says in the check_plain comments, in PHP 5.3 (I've tested with 5.3.2 and RedRat tells me he's running 5.3.10) it's possible to pass htmlspecialchars invalid multibyte strings which cause it to emit a warning and return nothing (effectively an empty string from the check_plain point of view).

This specific problem arises because in advanced_forum_topic_list.view AF specifies that the [teaser] token for a topic should be used as the alt (which ends up as the title attribute) for the link which is constructed from the topic's title:

  'title' => array(
    'label' => 'Topic',
    'alter' => array(
      'alter_text' => 1,
      'text' => '<span class="forum-topic-title">[title]</span> ',
      'make_link' => 1,
      'path' => 'node/[nid]',
      'link_class' => 'topic-title',
      'alt' => '[teaser]',
      'prefix' => '', 
      'suffix' => '', 
      'target' => '', 
      'help' => '', 
      'trim' => 0,
      'max_length' => '', 
      'word_boundary' => 1,
      'ellipsis' => 1,
      'strip_tags' => 0,
      'html' => 0,
    ),  
    'empty' => '', 
    'hide_empty' => 0,
    'empty_zero' => 0,
    'link_to_node' => 0,
    'exclude' => 0,
    'id' => 'title',
    'table' => 'node',
    'field' => 'title',
    'relationship' => 'none',
  ), 

When views is rendering the topic title as a link, it does this in views_handler_field.inc

    $alt = strtr($alter['alt'], $tokens);
    // Set the title attribute of the link only if it improves accessibility
    if ($alt && $alt != $text) {
      $options['attributes']['title'] = html_entity_decode($alt, ENT_QUOTES);
    }

The problem here is the optional third parameter for html_entity_decode, encoding:

If omitted, the default value for this argument is ISO-8859-1 in versions of PHP prior to 5.4.0, and UTF-8 from PHP 5.4.0 onwards.

If the title attribute code in views_handler_field is changed to:

  $options['attributes']['title'] = html_entity_decode($alt, ENT_QUOTES, 'UTF-8');

...the htmlspecialchars warning goes away, and the teaser is rendered correctly as the title attribute of the topic title link.

This seems to be a problem in both views-6.x-2.x and the 6.x-3.x branches.

However, there is already an issue open for views which would fix this: #1373588: Use decode_entities() instead of html_entity_decode() ... as replacing html_entity_decode with decode_entities also solves the problem.

In the meantime, AF users could either patch that specific line of their views module, or we could perhaps provide our own handler which overrides the problematic render_as_link() method. I don't have time to do the latter right now though.

I'll comment in the views issue to mention this problem in AF.

Setting to back to active while we decide whether to work around this in AF or wait for views to fix it.

I am very glad you're on that because I never would have figured that out!

I have added a 'UTF-8' parameter to the html_entity_decode function in the handlers/views_handler_field.inc file and htmlspecialchars() error gone. Thanks a lot to you, mcdruid, for your excellent investigation!

I've added an extra views field handler for node title which overrides the render_as_link function and corrects the problem with html_entity_decode.

This mean it's not necessary to hack the views module.

If the views module fixes the issue, we can remove advanced_forum_handler_field_node_title

Committed to 6.x-2.x

http://drupalcode.org/project/advanced_forum.git/commit/8e97798

Well, that did not work out great for me. All my views on my site broke. None of the titles linked to nodes no longer worked and it took me really long time to connect that to my updated Advanced Forum. Once I backed one version and did a database backup my views worked again. Any idea why? Was it just for me?

Sorry about that - I'd missed the fact that views already has:

/**
 * Field handler to provide simple renderer that allows linking to a node.
 */
class views_handler_field_node extends views_handler_field {

...and that my new handler should extend that class rather than views_handler_field.

This meant that the 'Link this field to its node' functionality provided by views_handler_field_node wouldn't work. AF itself doesn't use that option, which is why I didn't notice I'd broken it.

The attached patch fixes this, and has been committed to 6.x-2.x

However, unless I'm missing something, you didn't need to rollback your database - your data should not have been changed, just the (broken) presentation of node titles as links.