Tested in Chrome and Firefox.

My page and DB is UTF-8
My language is Spanish.

I got the thing preetty nailed out. Even without actually editting anything, the editor is putting random special characters after my correctly inputed special characters.

for example:

í should be í but is making to the DB like this í­ I don't know why.

I've tried solutions like setting my encoding to UTF8 in .htaccess and so on, but the page is OK, even special chars in the HTML are rendered ok (with a browser not forced to windows cp1525).

Also:
Whenever I edit something I know it's right in the DB (phpMyAdmin shows good marked HTML) and the editr shows wrongly and thus exitnding the problem wih each editing...

Disabling CKEditor for the "filtered HTML" and this problem doesn't happen at all.
I saw lots of threads in the CKEditor forum with CKEditor having problems with UTF8 encoded data, at this point I'm at the verge of choosing another editor.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

SebaSOFT’s picture

BTW using the CKEditor 3.6.1

dczepierga’s picture

Hi,

Pls try to change value of "HTML Entities" in CKEditor profile -> Advanced settings section. It should help.

Greetings

dczepierga’s picture

Status: Active » Postponed (maintainer needs more info)
mkesicki’s picture

Status: Postponed (maintainer needs more info) » Needs work

This issue looks similar to #1266406: NLS Problem

mkesicki’s picture

Status: Needs work » Postponed (maintainer needs more info)

@SebaSOFT can you check character encoding in your browser ?
Please also character encoding on your site ?
I also use UTF-8 in database and don't have any problems with characters (in polish language).

SebaSOFT’s picture

I deactivated "HTML Entities" (which I don't want disabled BTW) and it saved the special char to the DB correctly, then whenever I want to edit random chars are added even to the textarea underneath.

also BTW

Page encoding UTF8
Browser encoding UTF8 (autodetected)
MySQL encoding utf8_general_ci

dczepierga’s picture

@SebaSOFT, could u attach here file with the result of phpinfo() function?

Greetings

donSchoe’s picture

Status: Postponed (maintainer needs more info) » Needs work
FileSize
9.12 KB

Hi,
I'm re-opening this issue as I have the same problem with malformed character encoding. (I'm using german language but it doesn't matter as any HTML special character gets malformed.)

Example

  • Typing an ä gets correctly converted by the CKEditor to <p>&auml;</p>.
  • If I save this and want to edit this again, the following happens: <p>&auml;</p> gets malformed to <p>&Atilde;&curren;</p>. (Looks like this: ä)
  • Saving and editing malforms this again to: <p>&Atilde;&fnof;&Acirc;&curren;</p>. (Looks like this: ä)
  • And so on, ...

I've checked everything mentioned above

  • Drupal 7.8: Drupal uses UTF-8 for encoding all its data;
  • CKEditor-Module: 7.x-1.3;
  • CKEditor: 3.6.1;
  • MySQL-Database 5.0.77: "UTF-8 Unicode (utf8)";
  • PHP-Version 5.2.17: see attached phpinfo();
  • Firefox 6.0: "Unicode (UTF-8)"; (Updated to 8.0 later, same problem persists.)
  • CKEditor advanced settings "HTML Entities": neither "Yes" nor "No" doesn't change the problem.

Please help us, to track down this problem, it's really annoying to manage content this way. :)
-- donSchoe

dczepierga’s picture

Status: Needs work » Postponed (maintainer needs more info)

Could u try to update your CKEditor module to last DEV version and in CKEditor profile in "Security" section change value of "Use Drupal text format filters" to "Disabled"?
After change this pls try again to test it and write here your result.

Greetings

goood’s picture

I have the same problem as donSchoe.
New DEV version doesn't fix this problem.

But this problem appears only in non IE browsers. In IE 8 it is OK.

On my localhost works everything fine even in Firefox 7.0 or Chrome ...

CKEditor: 3.6.2
CKEditor-Module: 7.x-1.3 (last DEV version checked too)

Server configuration:
PHP 5.3.8
Apache 2.2.3
MySQL 5.0.77

localhost:
PHP 5.3.1
Apache 2.2.14
MySQL: 5.1.41

mkesicki’s picture

@goood m @donSchoe guys are you sure that your connection with database is UTF-8 ? Is your data correct saved in database ?

donSchoe’s picture

Status: Postponed (maintainer needs more info) » Active
FileSize
30.13 KB
48.24 KB
46.09 KB

Hi, as I already mentioned in comment #9, I double-checked everything. See attached screenshots.

donSchoe’s picture

@#11

But this problem appears only in non IE browsers. In IE 8 it is OK.

It seems not to be browser-related as this problem persists with the following browsers:

  • Microsoft Internet Explorer 9.0.3 (9.0.8112.16421)
  • Google Chrome 14 (14.0.835.202)
  • Mozilla Firefox 8.0
  • Opera 11.52 (1100)
  • Safari 5.1.1 (7534.51.22)
mkesicki’s picture

I think that this problem is related with bad encoding of database connection or browser.
Do you use any custom module or theme ? What modules do you have installed.
Please check http://dev.ckeditor.com/ticket/7564 - someone has similar problem to this one. This problem was caused by user's php code.

Please remember that you have utf8_unicode_ci codding set in you PhpMyAdmin it doesn't mean that your drupal also connect to database with this encoding. Please read about database connection configuration: http://api.drupal.org/api/drupal/sites--default--default.settings.php/7/... (reading settings file for drupal6 can also be helpful) and check database settings in your installation.

goood’s picture

I don't understand, but now it is ok. I didn't make any changes in settings.php...
I will watch it and check if it will be ok or not and try to check what happens...

edit:
I'am sorry for mistaking. It is not ok. I tried to add
'collation' => 'utf8_general_ci', into database settings in settings.php, and it had no effect.

When I edit some article, textarea with text is ok (encoding is ok), but when CK editor appears, encoding is wrong

Here are my extra modules:
webform
captcha
menu_block
admin_menu
features
devel
job_scheduler
advanced_help
pathauto
date
ctools
l10n_update
custom_search
views
transliteration
backup_migrate
conditional_fields
token
devel_themer
ckeditor
taxonomy_menu
feeds_xpathparser
feeds
better_exposed_filters

mkesicki’s picture

Remember that when you change database connection encoding after you have data saved in database it doesn't help.

'collation' => 'utf8_general_ci',

I think this is drupal default collation.
Can I see your site somewhere ? You can send me login data (if necessary ) in e-mail .

alwor’s picture

I have the exact same problem as #9 after moving to a new server. Works fine on the old server.
I just tried to reinstall the ckeditor module (new version 7.x-1.4 as of yesterday) and update the editor to 3.6.2. But to no avail.
All charactersets I find are utf8_general_ci, and have set 'collation' => 'utf8_general_ci' in settings.php.

Tried toggling rich editor off as default, and the text displays correctly in the plain text editor on edit, but if I toggle to the rich text editor I get the weird characters again. Then, if toggle back to the plain text editor, it keeps the weird characters.

Any ideas are welcome.

mkesicki’s picture

Status: Active » Postponed (maintainer needs more info)

Please try disable all filters (drupal and CKEditor). Check if this change anything.

alwor’s picture

Aah, thank you. For me it's the security filter - if the rich text editor is disabled initially and the Security settings are set to " Run security filters only when CKEditor is set to start automatically", then the problem will not occur.

OK, so that's a good workaround, but what to do to fix the security settings and have the rich text editor on by default?

mkesicki’s picture

@alwor
what filters and in what order do you have enabled ?
Guys can you check yours filters settings.
Does this http://drupal.org/node/1283918#comment-5169538 change anything for you ?

goood’s picture

The same think as alwor, but not depends on Security settings. It doesn't matter whether is in "Always run security filters for CKEditor" or "Run security filters only when CKEditor is set to start automatically" mode. Critical for me is if CkEditor is enabled initially. If not, problem will not occur.

What filters do you mean exactly?

mkesicki’s picture

@goood , so if you have CKEditor disabled on start and than you start it (switch to rich text editor) everything works ok, but when CKEditor starts on start , you have problems with characters encoding ?
And in your case it has nothing common with "Security settings" ?

goood’s picture

In my case it has nothing common with Security settings on that page /admin/config/content/ckeditor/edit/Advanced

I tried this two scenarios (both with CKeditor disabled at start):
1.
a. click new content
b. write text (switch to ckeditor and back works fine)
c. switch to plain text
d. save
e. edit - text is ok

2.
a. click new content
b. write text (switch to ckeditor and back works fine)
c. switch to CKEditor
d. save - in view mode everything (encoding) seems ok
e. edit - text is wrong! - because CKeditor is started initially due to last article state before save

mkesicki’s picture

Title: UTF-8 issues » [D7] UTF-8 issues
Status: Postponed (maintainer needs more info) » Fixed

I commited changes to GIT. Please test it and write if everything works as expected.

goood’s picture

For me it works fine!
I tested it with both of Security settings and with enabled and disabled "Use Drupal text format filters ". Everything is ok.
Great work michal!
Thank you!

mkesicki’s picture

Status: Fixed » Closed (fixed)
donSchoe’s picture

thanks, it's working now. all i did was an update to 7.x-1.6.