Problem:

MigrateSourceMSSQL class does not support "CharacterSet" key in the configuration array. The migration is not able to import data containing any multibyte characters.

Function sqlsrv_fetch_object() returns false for such rows with multibyte characters.

The migration does not process all rows from SQL queries and gives no explanation of why some rows are unprocessed.

Solution:

- Extend $connectionInfo array with "CharacterSet" key.
- Add 'character_set' => "UTF-8" to the database configuration array.

A patch is attached.

CommentFileSizeAuthor
#7 interdiff_2-7.txt976 bytesmartin_klima
#7 migrate-mssql_multibyte_support_3097408-7.patch1.29 KBmartin_klima
#3 migrate-mssql_multibyte_support_3097408-2.patch1.12 KBmartin_klima
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

martin_klima created an issue. See original summary.

martin_klima’s picture

martin_klima
he/his
Primary language Czech
CreditAttribution: martin_klima at Czech Drupal Association commented
Issue summary: View changes
martin_klima’s picture

martin_klima
he/his
Primary language Czech
CreditAttribution: martin_klima at Czech Drupal Association commented
FileSize
migrate-mssql_multibyte_support_3097408-2.patch1.12 KB
martin_klima’s picture

martin_klima
he/his
Primary language Czech
CreditAttribution: martin_klima at Czech Drupal Association commented
Issue summary: View changes
martin_klima’s picture

martin_klima
he/his
Primary language Czech
CreditAttribution: martin_klima at Czech Drupal Association commented
Status: Active » Needs review
klausi’s picture

klausi
he/him||they/them
Primary language German
Location 🇦🇹 Vienna
CreditAttribution: klausi at jobiqo - job board technology commented
Status: Needs review » Needs work 
+++ b/plugins/sources/sqlsrv.inc
@@ -95,7 +96,7 @@ class MigrateSourceMSSQL extends MigrateSource {
+      $connectionInfo = array( "Database"=>$this->configuration['database'], "UID"=>$this->configuration['username'], "PWD"=>$this->configuration['password'], "CharacterSet"=>$this->configuration['character_set']);

you say in the docs that this new parameter is optional, but you don't check it with isset() here?

This will cause PHP warnings when people upgrade the module, we should check the character_set key before adding it to the $connectionInfo.

And can you split this over multiple lines? Very hard to read in this long line.

martin_klima’s picture

martin_klima
he/his
Primary language Czech
CreditAttribution: martin_klima at Czech Drupal Association commented
Status: Needs work » Needs review
FileSize
migrate-mssql_multibyte_support_3097408-7.patch1.29 KB
interdiff_2-7.txt976 bytes

Thank you for your review @klausi. I fixed the issue.

klausi’s picture

klausi
he/him||they/them
Primary language German
Location 🇦🇹 Vienna
CreditAttribution: klausi at jobiqo - job board technology commented
Status: Needs review » Reviewed & tested by the community

Looks good, thanks!

pifagor’s picture

pifagor
Location 🇺🇦 Rivne
CreditAttribution: pifagor at Drupal Ukraine Community, GOLEMS GABB for Drupal Ukraine Community, GOLEMS GABB commented

  • pifagor committed 8daa1f9 on 7.x-2.x authored by martin_klima 
    Issue #3097408 by martin_klima, klausi, pifagor: UTF-8 support: Add...
pifagor’s picture

pifagor
Location 🇺🇦 Rivne
CreditAttribution: pifagor at Drupal Ukraine Community, GOLEMS GABB for Drupal Ukraine Community, GOLEMS GABB commented
Status: Reviewed & tested by the community » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

joseph.olstad’s picture

joseph.olstad
Primary language French
CreditAttribution: joseph.olstad commented

https://www.drupal.org/project/migrate/releases/7.x-2.12