By David Latapie on

Hi,

I'm new to Drupal. I installed 4.7 RC 1 on MySQL with PHP5. I tried to enter a very simple content with the full HTML option (I do not even consider another option - for me, this is full HTML or nothing).

Here is the address blog.empyree.org/drupal/

The text is the following:

Ceci est la page d?accueil. C?est un test, on verra ce que ça donne.

And it should be the following:

Ceci est la page d’accueil. C?est un test, on verra ce que ça donne.

Err, actually, even this very site doesn't display the correct character. Well, it is supposed to be a typographic apostrophe (right single quotation mark).

Since all the pages are UTF-8, I guess the problem comes from the editor. Any idea how to fix it?

⋅ Categories: Drupal 4.7.x

Comments

Torenware’s picture

Torenware commented

David,

I used to be in the International Client group at Netscape, back in the day. There are a couple answers to your question.

There are several steps that your text goes through before you see it in the blog's browser page:

  1. First, you have to get the text to Drupal one way or another, either by typing it in an HTML textarea, or by cutting and pasting from another application like a word processor, or by using something like Blogger. Any of these can go wrong, but here's what to watch for:
    • The web page should set its encoding correctly. Drupal.org does this right, setting its encoding to UTF-8
    • You need to make sure that you are not setting your browser to ignore the page encoding. On Firefox 1.5 in the US/English localization, when I'm looking at this page, View->Character Encoding says I'm using Unicode (UTF-8), which will work right.
    • When you cut and paste, either your word processor has to be a little bit smart (and know to paste in your text converted to UTF-8 from the Latin-1 encoding you are probably using in your word processing program), or you need to save your document as UTF-8, and then paste in from the converted document.
    • If you use Blogger or something like that, you'll need to read up how to set things up correctly, but the idea is the same.
  2. The database the site uses has to be set up correctly to store and recover strings in UTF-8. Since that's the standard set-up for Drupal 4.7, that's done right for you if the people running the web site use MySQL 4.1 or later with Drupal 4.7. I'm pretty sure that's the case here with Drupal.org.
  3. In rare cases, people who write modules need to be a little careful when they do certain things with text like try to compare it, change the capitalization, or try to figure out "how many characters" to print or truncate. People still get these things wrong sometimes, but I'm guessing that the blogging pieces of Drupal are pretty well tested in languages other than English, and probably get these things right.
  4. If you do these things right, and you use the right software, your accented French display should display just fine on Drupal.org, as should Japanese or Chinese, which I've seen displayed on Drupal.org. It will even work in Hebrew or Arabic in most modern browsers as well.

That's a little long, but that's how it works. And it does work pretty well most of the time.

Rob Thorne
Torenware Networks
http://www.torenware.com

Rob Thorne
Torenware Networks

Steven’s picture

Steven commented

Most likely you used phpMyAdmin to set up your database. There is a known issue with phpMyAdmin removing the character set definitions from the .mysql file. As a work around, you can remove the comment markers /*!40100 and */ from that file before you upload it to phpMyAdmin.

--
If you have a problem, please search before posting a question.

David Latapie’s picture

David Latapie commented

Hello to both of you,

  • I run Mac OS X 10.4, which means native UTF-8 everywhere.
  • My text editor is BBEdit (world class word editor, this is the one Eric Meyer uses, go figure).
  • I personally helped debugging the UTF-8 part of BBEdit. I use UTF-8 w/o BOM, as BOM poses problems with some browsers (back in the days before I bloged, that is in 2002)
  • I’m using standards-compliant browsers such as Safari 2, Opera 9 and Firefox 2
  • Character encoding is UTF-8 everywhere, with all browsers, so no problem here

All these information are not for parading, just so that you know that human or browser problems had been reduced to the minimum.

Now for the interesting part:

  • I successfully entered this text on WordPress and ExpressionEngine with the browser I'm currently using.
  • Previewing the text show me the right content, with every character the way it should be
  • I saved the text both with "PHP code" input format or "full HTML" input format (by the way, the latter doesn't deserve its name, as it autoinsert <br / >; is it asking too much to simply enter text with no intervention from the machine?)

So, it seems the problem comes from saving. Not only from saving but from saving on Drupal. I'm using 4.7 RC 1 with no mod at all. Plus, the database is shared with other CMS like WordPress and ExpressionEngine, which do not have any problem regarding the encoding. Finally, when I installed Drupal, I had to open the text editor and copy and paste the content of database.mysql, since my version of PHPMyAdmin does not have this function for calling an external file. I selected all and pasted in the "SQL" tab. So the bug told by Steven should not have occured with me…

Oh, I forgot: I'm using MySQL 4.1 and PHP5. Could it make a difference?

I'm quite lost on this thing…

smilodon’s picture

smilodon commented

About the databse and utf8, read this:
http://www.phpn.org/item/16586_PHP_MySQL_4_1_and_UTF_8.html
A hint how to improve the situation.

Im in this mess also now. I made a backup with CP, and if i restore it, then the unusual characters are all messed up... I dont remember the old database setting so... things are crappy...

vrykolakas’s picture

vrykolakas commented

 

David Latapie’s picture

David Latapie commented

Hello,
Since I posted here, I deleted Drupal and very recently reinstalled it, in version 5.1. So far, no more encoding problem. Problem fixed!

davidw’s picture

davidw commented

subscribe