Field locales_source.source is not suitable for long texts and huge config objects

I have the same issue.

The size of locale_sources' source field is too small to hold the data you're inserting. Either don't insert so much data, or make the source field take more data:

       'source' => array(
         'type' => 'text',
         'mysql_type' => 'blob',
         'size' => 'big',
         'not null' => TRUE,
         'description' => 'The original string in English.',
       ),

function locale_update_7006() {
  db_change_field('locales_source', 'source', 'source', array(
    'type' => 'text',
    'mysql_type' => 'blob',
    'size' => 'big',
    'not null' => TRUE,
    'description' => 'The original string in English.',
  ));
}

The lack of patch is because it's more seeing if this helps the Drupal 7 folk first. The fix would need to get into Drupal 8 before backporting to 7.

This is in response to comment #2 only:

I believe that the potential negative performance impact of increasing the size of the blob outweighs the benefit. The purpose of the Locale module is to support translations in the interface in which case 16kb would appear to be more then sufficient. I would not recommend making the change suggested in comment #2.

Hi all,

I had this issue, and not even #2 could help me.
I had a very large string in watchdog, and dblog page failed to load because the string was forced to be saved into locales_source.source.

Not having enough time for debugging how to eliminate the large string (deleting from watchdog did not do it) I had no choice.
My only remained solution was to patch the core (I know, never ever even think of it) temporary:

if(strlen($string) > 50000) {
    return $string;
}

/modules/locale/locale.module line 671

After some time I will remove this and see if survived the large string, but now it saved me a lot of debugging time.
I Hope this helps someone someday.

This still happens (of course. since it hasn't been fixed/backported yet).
During development of a module, which logs some events to watchdog I made a mistake and a complete view including 25 full nodes was var_dumped into the watchdog message. Since I was on a multilingual site and currently in another language interface than English, the "Show Recent Log Posts" page fails with the above error. It works fine when switching to English.

I am experiencing this issue now, but @kardave's temporary fix doesn't seem to work any more... Does anyone know how to fix it, also even if only temporary?

See this #2057615: Increase size of source field to hold long URLs

This is because mysql limit for blob field is 64k
Faced with that in d8 where config object is bigger than that

BTW looks the same limit we have for translation

The attached patch is port of #4 to Drupal 8. All it does is store an empty string in the 'locales_source' and 'locales_target' table if the source string is greater than 64KB.

This is only a temporary solution. The most immediate side-effect I am seeing is when translating config the source string no longer appears because it is empty.

Here is the patch and I am setting it to needs review just get the testbot to review it.

The watchdog is the most likely place where too long strings get passed to t(), but it any code may make this “mistake”. The translation system should guard itself for this types of errors.

I see two options: 1. Truncate the string; 2. Throw an exception.

I agree that increasing the table size is not the solution. Exceptional cases like these cannot justify any performance this change may have.

I just ran into this issue on 8.9.13.

#14 applied and works.

Rerolled the patch in #14, thanks!

1. +++ b/core/modules/locale/src/StringDatabaseStorage.php
   @@ -463,6 +464,14 @@ protected function dbStringSelect(array $conditions, array $options = []) {
   +    if (Unicode::strlen($string->getString()) > 64000) {
   

   There's no such function in D9, see https://www.drupal.org/node/2850048

2. +++ b/core/modules/locale/src/StringDatabaseStorage.php
   @@ -463,6 +464,14 @@ protected function dbStringSelect(array $conditions, array $options = []) {
   +      $string->setString('');
   

   This event needs to be logged in watchdog (warning at least) so users can catch this bugs

Addressed #28.1, thanks.

Note: in my tests I did not have this issue with PostgreSQL, however I came across it with MySQL v8.0.28

so far I was able to mitigate my use case by:
#2844452: Export configuration YAML strings as multiline
#3271926: mysql 1406 Data too long for column 'source' at row 1 insert into locales_source

With those two related items that @joseph.olstad referenced now fixed, is this still an issue

Curiously enough, for some reason I noticed that webforms configuration that is edited by ckeditor generates \r\n even on the recent core release 9.3.16.

Postgresql has a larger blob, doesn't have this issue but with MySQL variants I had issues importing large webform config yml , not sure why ckeditor would be outputing \r\n or where to turn this bad ckeditor behavior off as it affects the processing of translations that eventually go into the locales table.

Sorry not much in dept here but I did check out the blob sizes for MySQL:

TINYBLOB   : L < 2^8  =           256 Bytes
BLOB       : L < 2^16 =        65,536 Bytes
MEDIUMBLOB : L < 2^24 =    16,777,216 Bytes
LONGBLOB   : L < 2^32 = 4,294,967,296 Bytes

MySQL seems to blow up on 64K limits , the blob type is used in a few places.

Drupal running PostgreSQL 12 that I also use doesn't have this issue because it's blob field has different /higher limits.

As a workaround I manually edited my exported YML, replaced \r\n with \n and everything imported correctly with MySQL, however today someone brought up to me a large webform that is unable to save a webform translation in the ui after a certain limit. Seems like two different but similar issues going on, haven't done a deep dive on this yet.

Wonder if this should be reopened?

I've got this issue in Drupal 9 and Drupal 10.
Having the body field or content fields with both editorial content as well as full HTML on larger pages causes this local field validation to fail every single time.

I have to patch each release so that the 10,000+ records and updates will actually complete in my builds.

Most recent Patch attached.
This commits to both Drupal 9.5.10 and Drupal 10.1.2

Maybe increasing size of blob for MySQL could help here as both Sqlite and Pgsql has limit for blobs 1GB but Mysql only 64k

It would be worthwhile looking into increasing the blob size. With that said, 64kb for a locale string is very large.

When I manually replaced \r\n with \n and everything imported correctly with MySQL.
The issue is that the ckeditor outputs \r\n, I'm guessing that a filter needs to be modified in core for the locale processing that treats \r\n the same as \n.

Ideally the common web interface tools should be handled and understood by the locale system.

It seems strange that we'd be translating strings of 64 kilobytes when we could have line items as was originally intended.

With that said, if all of this is too much trouble then please switch the default recommended database system for Drupal from MySQL to PostgreSQL.

Alternatively, perhaps the config system could automatically convert \r\n to \n and then maybe the locale system perhaps would work correctly. Might be easy to create this patch.

Seems like this should be moved out of PNMI.

Tested by changing source field 'mysql_type' from 'blob' to 'longblob'. Why this could not be an option?

Can the issue summary be updated to use a standard template please :)

Also was previously tagged for tests still needed I'm assuming?