By joel_guesclin on

Upgrading from 4.6 to 4.7, I have discovered a problem with "special" characters such as right and left apostrophes for example. On searching the forums, I doscover that this seems to be a known problem: see here and here and here.

There seem to be several proposed ways of dealing with the problem:

  • use some tool to convert all the "funny characters" manually: this solution is barely acceptable in English, unacceptable in any language with accented characters like French or Swedish, and simply impossible for Bengali or Japanese (I have all of these cases)
  • after the conversion manually change the default character setting for each table to utf-8
  • unload and reload the database to convert to utf-8 before (I think) the conversion, so that it is ok afterwards

I am always a bit nervous about things like this, not wanting to find myself with nasties further down the road. I would very much welcome an "official" ruling about how to resolve the problem.

A final point to add to this: I must be able to unload and reload the database, which ever solution is adopted, since I must upgrade on a copy of the live database, not the live database itself.

⋅ Categories: Drupal 4.7.x

Comments

cog.rusty’s picture

cog.rusty commented

I don't think this has to do with the Drupal 4.6 to 4.7 upgrade. It is rather about
(a) MySQL versions, 4.0 or older vs 4.1 or newer and
(b) the default character sets and collations of the database and the tables, even when moving between same MySQL version. Drupal always writes utf8 but MySQL isn't always aware of this.

There are different solutions to different problems. Sometimes some tables are set up with the wrong collation. This happens often with "Fantastico" installations.

starbow’s picture

starbow commented

Definitely not official, but I just posted how I dealt with the problem after-the-fact: http://drupal.org/node/108052#comment-192990