I recently started setting up a drupal 7 website on Azure cloud using a web role and SQL azure backend. I was very much pleased until I hit a character set brickwall.

I have a virtual machine with Windows Server 2008 R2, IIS 7.5, SQL Server 2008 R2 as a development server, trying changes before I upload them to the cloud. I have some extra tables in drupal database to support my custom module for the application. Among others, at some point I need to run some code like that:

$r = db_select('my_creators', 'c')
    ->fields('c')
    ->condition('my_url', $creator_url)
    ->execute()
    ->fetchAll(PDO::FETCH_ASSOC);

$creator = $r[0];

foreach ($creator as $k => $vv)
    $s.= t('@key -->> @value<br>', array('@key' => $k, '@value' => $v));

print $s;

I have cut out a lot of extra code, but this is the code having the problem, when $creator_url has a value with some special character (e.g. André_Breton). This runs perfectly on my development machine, fetching the record and displaying the data so I was very surprised when I uploaded to the cloud and got this error

PDOException: SQLSTATE[IMSSP]: An error occurred translating string for input param 7 to UCS-2: No mapping for the Unicode character exists in the target multi-byte code page. in dblog_watchdog() (line 157 of E:\sitesroot\0\modules\dblog\dblog.module).

The underlying databases are identical, actually I data-sync the sql azure database to the development sql server database daily. All character data fields are nvarchar.

I cross-checked all versions of IIS, php, php extensions, php.ini and all are identical.

I managed to kind of work around this by adding

Database::getConnection()->setAttribute(PDO::SQLSRV_ATTR_ENCODING, PDO::SQLSRV_ENCODING_SYSTEM);

$creator_url = iconv('UTF-8', 'UCS-2', $creator_url);

before the select, and then for the print


foreach ($creator as $k => $vv) {
    $v=iconv('ISO8859-1', 'UTF-8', $vv);

    $s.= t('@key -->> @value<br>', array('@key' => $k, '@value' => $v));
}

Using this workaround the select goes through, the correct record is selected and displayed, but then I get several warnings

Warning: htmlspecialchars(): Invalid multibyte sequence in argument in check_plain() (line 1572 ofE:\sitesroot\0\includes\bootstrap.inc).
Warning: htmlspecialchars(): Invalid multibyte sequence in argument in check_plain() (line 1572 ofE:\sitesroot\0\includes\bootstrap.inc).

which makes sense since usually drupal functions expect all strings to be UTF-8.

I am hitting my head against a wall for a couple of days now, any help will be deeply appreciated.

Thank you!

Comments

diehard6996’s picture

You just have to add few lines of code at two specific drupal distribution files. This works for Drupal 7 for sure.

First the file modules\dblog\dblog.module, go to function dblog_watchdog(array $log_entry) and add the following lines right at the beginning (in Drupal 7.15 this is line 140):

  if (!mb_detect_encoding($log_entry['message'], 'UTF-8', true))
          $log_entry['message'] = mb_convert_encoding($log_entry['message'], 'UTF-8', 'ISO-8859-1');
  if (!mb_detect_encoding($log_entry['request_uri'], 'UTF-8', true))
          $log_entry['request_uri'] = mb_convert_encoding($log_entry['request_uri'], 'UTF-8', 'ISO-8859-1');
  if (!mb_detect_encoding($log_entry['referer'], 'UTF-8', true))
          $log_entry['referer'] = mb_convert_encoding($log_entry['referer'], 'UTF-8', 'ISO-8859-1');

Then at file includes\bootstrap.inc, function check_plain($text) which at Drupal 7.15 is at line 1571 add the following lines right at the beginning again:

    if (!mb_detect_encoding($text, 'UTF-8', true))
        $text = mb_convert_encoding($text, 'UTF-8', 'ISO-8859-1');

Many credits to Robert Johnson at http://social.msdn.microsoft.com/Forums/en-US/sqldriverforphp/thread/6a1...

david_garcia’s picture

Check this out for a driver level fix:

https://drupal.org/node/2168191

If you are getting this error, you are probably going to come accross it soon in the most unexpected of the situations and modules other than DBLOG.

jordanrussellsmith’s picture

Thanks for providing this solution. This error was driving me crazy!

Is there any way to make these changes without having to modify core files? I need to make a similar change to includes\session.inc.