Problem/Motivation

This issue addresses how best to deal with problem variables that cause breakage during upgrade or updates.

Two schools of thought on how best to have Drupal react are discussed examining the benefits of each to either the novice user and/or the experienced developer. It appears a consensus has been reached that most examples of breakage are the result of older contributed modules not being properly (cleanly) uninstalled which can result in old variables remaining in the database or database table structure and then, upon new update/upgrade, being unrecognized which leads the upgrade/update to hard failure.

You get an error like
Notice: unserialize() [function.unserialize]: Error at offset X of Y bytes in variable_initialize() (line 916 of .../includes/bootstrap.inc

By clicking 'Clear all caches' on admin/config/development/performance the error comes back again.

Proposed resolution

Create a softer failure method for Drupal. Drupal should identify and communicate with the user that the update/upgrade process has failed, providing meaningful feedback not only on the source and nature of the failure, but providing adequate direction on resolving the failure.

Iterate over all variables and check each for their consistency. (borrowed from variablecheck module.)

Allow for a hotfix through settings.php $conf

Throw an exception when encountered inconsistent variable(s).
exception-bad-variable.png

Steps to reproduce

drush sql-query "update variable set value='unserialized value' where name='site_name'"
drush cc all

Visit the site. Exception is thrown.

Now hotfix by editing settings.php and uncomment the line with $conf['site_name']. Now site_name is valid again.

Visit the site again.

Workaround

Easy route: install http://drupal.org/project/variablecheck as mentioned in #10

There is some variable from the variables table misformed.

Check your database by running the following drush SQL command.

drush eval '$query = db_query("SELECT name, value FROM {variable}"); while ($variable = $query->fetchObject()) { $value = @unserialize($variable->value); if ($value === FALSE && $variable->value !== serialize(false)) { echo "Variable " . $variable->name . " has serialization issues\n"; } }'

Depending on your skill you can either drop the row using by replacing XYZ with the correct variable name

DELETE FROM variable WHERE name='XYZ'

or visit the page responsible for the variable. See some of the comments below.

Clear cache to make sure the variable tables is error free again.

Remaining tasks

Fix the test somehow as the failing patch from #14 should be reported by the testbot somehow.

+++ b/modules/simpletest/tests/bootstrap.test
@@ -267,6 +267,27 @@ class BootstrapVariableTestCase extends DrupalWebTestCase {
+    // next line fails with a PDOException: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'drupal_d7.simpletest992395' doesn't exist: DELETE FROM {} WHERE (cid = :db_condition_placeholder_0) ; Array ( [:db_condition_placeholder_0] => variables ) in cache_clear_all() (line 169 of /Users/clemens/Sites/drupal/d7/www/includes/cache.inc).
+    cache_clear_all('variables', 'cache_bootstrap');

Original report by Leon

I've updated my drupal from version 7.4 to 7.8 and getting the follwing error.

Notice: unserialize() [function.unserialize]: Error at offset 2 of 11 bytes in variable_initialize() (line 916 of /xxx/xxx/xxx/xxx/xxx/xxxxx.com/includes/bootstrap.inc

I've tested a drupal installation of 7.4 three times with the same results.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

catch’s picture

Category: bug » support
Status: Active » Postponed (maintainer needs more info)

This means you have a corrupted variable stored in {variables}.

There's not much can be done to figure this out from the report, please attach at least the following to this issue:

- a list of contrib modules installed on the site

- an sql dump of the the variable table from your site.

AndrzejG’s picture

I discovered that on update to Drupal 7.8, above conflict is raised by Pathauto module, probably with menu system.
I conclude this because:
- update.php has to be run twice to rebuild database, and 2 modules are still to update, including Menu module,
- the error above is displayed when using Main menu from newly opened page,
- error disappears after uninstalling Pathauto module.

I updated from Drupal 7.7 because Pathauto, on the Patterns screen, began to raise error on the page without styling (white) - something like that:
PDOException: There is already an active transaction in _drupal_session_write() (line 203 of /includes/session.inc). It is only a part that I derive from history of my browser.

The other error messages informed on the conflict with token.inc, line 14 AFAIR (usual for many smaller errors).

AndrzejG’s picture

Sorry, error still exists.
It appeared in the page with Views attached using EVA module. Probably due to the inconsistency of paging. Error disappeared after I set paging to "Display all items" in all the components of the View.

Note: Hmm, now I think it is not necessarily the cause of the error as it appeared only once. This effect could have appeared simply as a remains of that reported in my previous post. There are still some entity caching problems...

AndrzejG’s picture

Error still exists in my site. Taking #2 into account I suspect Pathauto corrupted some variable.

How to make dump of database?

clemens.tolboom’s picture

Issue summary: View changes

Updated issue summary.

clemens.tolboom’s picture

Issue summary: View changes

Removed some h3

clemens.tolboom’s picture

Status: Postponed (maintainer needs more info) » Needs work

Following the step from the Issue Summary I had this message for two variables.

site_offline_message (drupal 6 variable)
maintenance_mode_message (drupal 7 variable)

both had the same value but seemed to have a wrong ending as the error suggests Notice: unserialize() [function.unserialize]: Error at offset 422 of 422 bytes in variable_initialize()

The value is a string ending with a semi-colon.

site_offline_message	
422
s:415:"This is a development site for .... n enabled for a short period of time. We apologize for this.";

Navigating to site maintenance mode showed an empty message. Changing that I got this from the variable table.

maintenance_mode_message: "Under maintenance"

Deleting the D6 variable site_offline_message

drush sql-query "delete from variable where name = 'site_offline_message'"

got the notice go away.

clemens.tolboom’s picture

Category: support » bug
Priority: Major » Normal

Making this a normal bug ... I just re-tested this while upgrading a D6 site.

Is the serialization done differently based on PHP versions?

My two variables are named here

$ grep -r 'site_offline_message' *
includes/update.inc:    // Rename 'site_offline_message' variable to 'maintenance_mode_message'
includes/update.inc:    if ($message = variable_get('site_offline_message', NULL)) {
modules/system/system.install:  // Remove obsolete variable 'site_offline_message'. See
modules/system/system.install:  variable_del('site_offline_message');

Scanning modules/system/system.install I see a change in the variable table

/**
 * Converts fields that store serialized variables from text to blob.
 */
function system_update_7055() {

Does php serialize() handle blobs differently then text fields?

clemens.tolboom’s picture

Hmmm ... according to

/**
 * Split the 'access site in maintenance mode' permission from 'administer site configuration'.
 */
function system_update_7036() {
...
  // Remove obsolete variable 'site_offline_message'. See
  // update_fix_d7_requirements().
  variable_del('site_offline_message');

the variable should not be available ... drush updb shows the update number.

drush sql-query 'SELECT * FROM variable WHERE name LIKE "maintenance%" OR name LIKE "site%"'

What could be wrong. Is it drush? php.serialize()?

clemens.tolboom’s picture

Status: Needs work » Active

As the issue has no patch setting status to active.

@Leo.dil@gmail.com
@AndrzejG

Could you test for:
1. Visit admin/config/development/performance and clear cache (the error should reappear)
2. Run the SQL from the issue summary with value Y is replaced by error value Y

clemens.tolboom’s picture

Issue summary: View changes

Added replace warning and clear cache

AndrzejG’s picture

My Y is 40, and:
SELECT name, length(value), value FROM variable where length(value) < 40 - 5 and length(value) > 40 + 5
didn't return any record. Neither +/- 8

Leo.dil@gmail.com’s picture

Sorry for late answer but I've solved it with this module (http://drupal.org/project/variablecheck) Everything is working fine now.

Leo.dil@gmail.com’s picture

Issue summary: View changes

Added path to Clear all caches by clemens.tolboom

clemens.tolboom’s picture

Version: 7.8 » 7.x-dev

@Leo.dil@gmail.com tnx for the module.

I've updated the issue summary with the link to the project. And bumped it to 7.x-dev

The current D7 code is

      $variables = array_map('unserialize', db_query('SELECT name, value FROM {variable}')->fetchAllKeyed());

and that module uses (a little shortened from around http://drupalcode.org/project/variablecheck.git/blob/refs/heads/7.x-1.x:...)

$entries = db_select('variable', 'v')->fields('v', array('name', 'value'))->orderBy('name', 'ASC')->comment('Load all variables')->execute()->fetchAll();
...
foreach ($entries as $entry) {
  $value = @unserialize($entry->value);
  if ($value === FALSE && $entry->value != 'b:0;') {

@catch : should we patch this in core? The problem lies with contrib modules not clean up properly. The workaround is solid now we know this module.

AndrzejG’s picture

@Leo.dil@gmail.com - thank You.
I found that update_notify_emails variable is corrupted.
That is why I, and some people reporting, don't receive update emails.

Leo.dil@gmail.com’s picture

You are welcome glad to help.

clemens.tolboom’s picture

I created a patch based on variablecheck modules code http://drupalcode.org/project/variablecheck.git/blob/refs/heads/7.x-1.x:...

I failed to write a proper test. The test tries to inject an unserialized value into variables table. It reseeds the variables but in the end returns the default value.

This part is probably the wrong expectation.

cache_clear_all('variables', 'cache_bootstrap');
// Reseed the variables. This should load $name
variable_initialize();
// Value should be FALSE but become default
$value = variable_get($name, 'default_value');

Status: Needs review » Needs work

The last submitted patch, variable-unserialize-error-1284364-14-failed.patch, failed testing.

clemens.tolboom’s picture

Title: After update from 7.4 to 7.8 Notice: unserialize() [function.unserialize]: » Harnes variable_initialize against broken values - Notice: unserialize() [function.unserialize]:
Issue tags: -unserialize(), -update from 7.4 to 7.8

I changed the title to reflect the problem function.

Removed the tags as they were singletons.

Marvin Daugherty’s picture

Thank you #10! Variable Check solved my problem in seconds. I just clicked on my Status Report, clicked on the link, checked the box, affirmed the delete, done.

clemens.tolboom’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, variable-unserialize-error-1284364-14-failed.patch, failed testing.

clemens.tolboom’s picture

Ok so test variable-unserialize-error-1284364-14-failed.patchReview from #14 fails as we expected.

Only the testbot does not report this which is bad. I expected the PDOException as mentioned in the patch (See below)

+++ b/modules/simpletest/tests/bootstrap.test
@@ -267,6 +267,27 @@ class BootstrapVariableTestCase extends DrupalWebTestCase {
+    // next line fails with a PDOException: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'drupal_d7.simpletest992395' doesn't exist: DELETE FROM {} WHERE (cid = :db_condition_placeholder_0) ; Array ( [:db_condition_placeholder_0] => variables ) in cache_clear_all() (line 169 of /Users/clemens/Sites/drupal/d7/www/includes/cache.inc).

This is not reported in the issue.

+++ b/modules/simpletest/tests/bootstrap.test
@@ -267,6 +267,27 @@ class BootstrapVariableTestCase extends DrupalWebTestCase {
+    ¶

Whitespace

clemens.tolboom’s picture

Issue summary: View changes

Updated issue summary.

clemens.tolboom’s picture

Issue summary: View changes

Updated issue summary.

clemens.tolboom’s picture

Issue summary: View changes

Updated issue summary.

Heine’s picture

You are tying the detection to implementation details of PHP's serialize function.

At best the unserializable string should be compared to serialize(false).

Then again:

- How do unserializable variable entries occur?
- How does the proposed reporting _help_ the user vs. just the notice? Will the user be able to fix the problem with this information (I think not).

clemens.tolboom’s picture

@Heine thanks for the questions.

You are tying the detection to implementation details of PHP's serialize function.

I'm not sure where the bug comes from. Mine reported in #5 could be database encoding as the site_offline_message was quite long and contained afaik a weird char. I need to check this when the site upgrade runs it's next iteration.

At best the unserializable string should be compared to serialize(false).

Are you referring to the patch? The ok patch from #14 variable-unserialize-error-1284364-14.patch does that right?

How does the proposed reporting _help_ the user vs. just the notice? Will the user be able to fix the problem with this information (I think not).

It helped me seeing the reported variable names. That made me visit the correct page and fix the offline message. But it depends on the name of the variable(s) and the skills of the users seeing its name.

The cache_clear_all(); fatal error puzzles me. That occurs both locally with simpletest and by the testbot.

Damien Tournoud’s picture

If we really want to harden the variable system against broken values, we should just fail-safe here. If a variable fails to load, there is no way to be sure that it is safe to continue. We should just throw an exception wait for the administrator to fix the variable.

The current patch is in no way an improvement.

clemens.tolboom’s picture

@Damien Tournoud I don't agree (partly)

I do agree on 'let the admin figure this out'. But users are puzzled about what went wrong with the variables_initialise. By displaying it's name helped me to 'fix it'.

The usage of http://drupal.org/project/usage/variablecheck is a bad indication. (December 4th : D6 171, D7 779)

I have no clue about what the root cause is. Throwing an exception and let the admin solve it is a little harsh for a savvy admin. But we do have the module http://drupal.org/project/variablecheck available.

So I do think displaying the variable name is a good thing but that means the patch needs a test (which will probably never work due to calling cache_clear_all to early?).

I asked the project owner to comment on this issue in trying to find root causes through #1371704: Please add link to core issue(s) on your module frontpage. Maybe we can find the root causes.

Feel free to close the issue as won't fix as we do have a workaround.

Heine’s picture

I agree with Damien,

If a variable value cannot be determined, processing should abort. After all, the variable could be important for security or consistency.

Heine’s picture

Further, in response on how to detect an broken serialized string & how you had tied it to implementation:

  if ($value === FALSE && $db_value != 'b:0;') {
+          $err_variables[] = $name;
+        }

'b:0;' is an implementation detail. serialize and unserialize do not guarantee the structure of the serialized string.

The second part of the if clause should not compare to 'b:0;' but to $db_value !== serialize(FALSE)

Damien Tournoud’s picture

@clemens.tolboom: I'm not saying this issue should not continue. On the contrary, I think improving the robustness of the variable loading is important. But in case any variable fails to unserialize, we should just abort the whole process (by throwing an exception). There is no way to know if continuing is going to be safe in that case; as a consequence, we should fail early.

So, could you please reroll this patch with:

- throwing an exception instead of the drupal_set_message()
- using serialize(FALSE) instead of 'b:0;'

Damien Tournoud’s picture

Version: 7.x-dev » 8.x-dev
Issue tags: +needs backport to 6.x

Also, this should be a Drupal 8 issue at this point (to be backported all the way back to D6).

cafuego’s picture

What do you mean by root cause of the problem?

The cause of these notices that people can't turn off is that Drupal overrides the error_log settings defined in php.ini early in the bootstrap process, before it is able to apply its own "Logging and errors" setting from the database. The code suggests it should be defaulting to the values in php.ini, but I'm not convinced it does that on all systems I have D7 sites on.

Alternatively, it might be caused by people who have 'Show all' enabled in the 'Logging and errors', as that's what D7 defaults to. I suggest that should in fact have a "System default from php.ini" option (tastefully named) and default to that instead.

Re throwing an exception instead of displaying a notice: Why do you hate your users? :-) The very very large majority of instances of this problem that I've seen is from users who wouldn't know where to start removing a broken variable from their installations. Making Drupal *break* for them because of what (in 100% of cases I've seen sofar) is a trivial cruft problem is not what I would recommend as a fix.

clemens.tolboom’s picture

@cafuego

The ROOT cause is "why are these variables unserialisable?" suddenly. My case was with with site_offline_message when upgrading from D5 to D6 to D7 see #5 where I suspect system_update_7055().

@Damien Tournoud as @cafuego suggest in ##2 I agree with throwing an exception is not cool. As I understand from comments we should use code similar to

+++ b/includes/bootstrap.inc
@@ -918,7 +918,25 @@ function variable_initialize($conf = array()) {
+      if (count($err_variables)) {
+        $err_variables = join (", ", $err_variables);
+        // We cannot call watchdog as modules are not loaded so resort to drupal_set_message
+        drupal_set_message(t("There are errors with your variables %variables.", array('%variables' => $err_variables)));
Throw exception here instead of dsm?
+      }

I would opt for an auto offline instead. That is a lot kinder to our users right?

I'll try to build a patch this weekend and see what result of it.

Damien Tournoud’s picture

Again, throwing an exception is precisely what you want here. You can display a nice error message pointing the user to the variables that need fixing, but there is no way to know if it is safe to continue.

I have seen way too many production instances of Drupal that started doing weird stuff or destroying data because something failed during the variable loading.

clemens.tolboom’s picture

After some testing with D7 I agree throwing an exception is the way to go.

The exception below was created by

drush sql-query "update variable set value='unserialized value' where name='site_name'"
drush cc all
Exception: Cannot unserialize variables site_name. Check your database or hotfix through settings.php in variable_initialize() (line 939 of /Users/clemens/Sites/drupal/d7/www/includes/bootstrap.inc).

The attached D7 patch allows for a hotfix through settings.php. That way they can try to fix drupal through the drupal admin pages.

clemens.tolboom’s picture

Status: Needs work » Needs review
FileSize
1.45 KB

Attached the D8 patch ... which is almost the same as the D7 version :)

clemens.tolboom’s picture

Issue summary: View changes

Updated issue summary.

clemens.tolboom’s picture

FileSize
31.63 KB

exception-bad-variable.png

clemens.tolboom’s picture

Issue summary: View changes

Updated the current resolution.
Added steps to reproduce.

cafuego’s picture

I don't agree that throwing an exception and erroring out is the best idea, but since you want to go that way, would it make sense (given these problems occur when an older site is upgraded to D7) to actually run a check in upgrade.php, not allowing a user to proceed with the upgrade until the variable was fixed?

I'd suggest making the error message a bit less cryptic and more helpful (especially for novices).

Suggestion: The variables %variable are unreadable and need to be fixed. Please read $SOME_SECTION in UPGRADE.txt on how to resolve this problem.

Then add $SOME_SECTION to UPGRADE.txt and explain what the problem is, why Drupal won't allow them to continue, how to check the database, how to delete the variable from the database and/or how to add an override in settings.php

clemens.tolboom’s picture

@cafuego I agree with adding some text somewhere. But I'm not sure whether this is an UPGRADE and not an UPDATE problem.

As Leo.dil@gmail.com originally reported it was an UPDATE of 7.x

We still do not have variable names to find the root cause do we?

Is it system_update_7055 ?

cafuego’s picture

The variable names are from any contrib module that didn't clean up after itself, this isn't limited to just core. (Hence my disagreement with throwing an exception).

clemens.tolboom’s picture

Argh ... I somehow cannot get others to call names :(

I want proof by having a list of module + name of variables causing errors.

@cafuego I don't get

any contrib module that didn't clean up after itself

as I assume these modules are not writing these variables directly into the variables table right? They use variable_set/_get/_del I still assume.

clemens.tolboom’s picture

Attached patch have changed text and allow for translation.

The new text refers to UPGRADE.txt as @cafuego suggested in #35 (I dropped the section reference) which has also stuff about update. The are links to d.o so I think we are ok now. A needs documentation tag is for code and not for d.o documentation right?

cafuego’s picture

@clemens.tolboom

I want proof by having a list of module + name of variables causing errors.

In my case they were settings from the bittorrent module for D5/D6, I don't know what variables others have had issues with... I just heard about the problem often enough to think variablecheck was warranted as a way for (non-technical) people to simply resolve the notices.

as I assume these modules are not writing these variables directly into the variables table right? They use variable_set/_get/_del I still assume

With a good module, that is written to delete all the settings that are added via variable_set() and where the user runs the uninstall handler after disabling that module before doing an upgrade, there should be no problem.

However, in the real world that's not the case. Many people have used various contrib modules on 5.x that no longer exist for 6.x and 7.x; these may not have been uninstalled correctly and even if they have, they may not have removed all settings they created using variable_set()

I guarantee you that in the majority of instances of these notice, the problem is some left-over setting from a contrib module, not one from Drupal core that magically corrupted itself.

The proposed fix here will throw an exception and effectively break a user's Drupal if they at some point in the past had a contrib module that did not remove all its settings, or if the user disabled and then deleted it without running the uninstall hook, which I think is wrong and will trip up a lot of users.

It may not be wrong for a D8 upgrade, but doing this on D7 will likely lead to a horrible amount of pain and why I think there should be a somewhat more userfriendly way of notifying and helping users than throwing an exception and locking them out of their own site.

A mis-loaded variable can never be a security problem for user #1, so perhaps you could pull in the variablecheck module code and display an error to user #1, with a link to show exactly the variable that's causing the problem (and the link to remove it) rather than force them to manually hack the DB. That's all fine and well for advanced users, but there are also people who don't even know how to access their databases, think of Drupal deployments on shared hosts through Fantastico for instance.

D6 at least doesn't load and unserialize all variables in the bootstrap process, it runs a single SQL query each time variable_get() is called, so in that case the site will at least only break if a specific module tries to load an unserialized setting. Thus, D6 never tries to load variables for modules that are no longer installed, so cruft in the variables table won't cause an exception on each page load on D6... which is exactly the reason the unserialize notices were never a problem before.

clemens.tolboom’s picture

Could someone review my remark on system_update_7055 in #6

Reading http://dev.mysql.com/doc/refman/5.0/en/blob.html

BLOB values are treated as binary strings (byte strings). They have no character set, and sorting and comparison are based on the numeric values of the bytes in column values. TEXT values are treated as nonbinary strings (character strings). They have a character set, and values are sorted and compared based on the collation of the character set.

My site_name had AFAICR accents in it. Could this be a 'root' cause?

cafuego’s picture

The character set is used for comparision, sorting and collating in the sql server, which isn't relevant to what serialize() does.

However, what *may* happen with an accented string is that if the connection and field use a given character set when storing a multibyte character and that character set is changed (say from latin1 to uft8) without also converting the data you might end up getting trashing the data. I've had that happen with a client who changed their mysql configuration and changed the default character set without also converting their data to the new character set. Accented characters such as "á" ended up being displayed as "~Å" etc.

If that's what happened to you, MySQL may have returned the original single accented character in the serialized string as *two* characters, making the length be incorrect and unserialization to fail because of that.

In which case, I'm still not convinced Drupal should stop working altogether.

clemens.tolboom’s picture

This http://stackoverflow.com/questions/4149030/php-unserialize-error-at-offset is interesting in that it conclude having ie

string(18) "a:1:{s:3:\"sam\";}" length:18

this is only parseable when having magicquotes enabled.

Question is whether this is related to my upgrade from D6 to D7 or update 7.4 to 7.8 as Leo.dil@gmail.com originally reported.

function system_update_7055 is a db centric action doing who knows what :(

@cafuego @Damien Tournoud
I do agree with the both of you. Drupal should stop working _and_ we should help our savvy Drupal user base.
I propose to update http://drupal.org/upgrade with some documentation about how to set $conf variables. What do you think?

garbus2’s picture

Version: 8.x-dev » 7.12

I have installed 7.12 Drupal and I do get below info once per while:

Notice: unserialize() [function.unserialize]: Error at offset 34 of 40 bytes in variable_initialize() (line 943 of /includes/bootstrap.inc).

When I refresh the site the info dissapear. Shows on all sites and without any scheme

I have installed invalid variables and it reported one variable (my email) assigned to : update_notify_emails in table variables in database.
Variable removed and problem gone

marcingy’s picture

Version: 7.12 » 8.x-dev

Please do not change version numbers

xjm’s picture

Marked #1445806: upgrade D6 => D7 and menus with "umlaut" (ä, ü, ö) and other special characters (™, ®) gives unserialize() error and #1326484: Problem updating my drupal 7.7 installation to 7.9 as duplicates of this issue. Please test #39 and see if it resolves your issue, and in each case also provide the minimum steps to resolve your particular issue from a clean install of D7 or D8. Thanks!

Status: Needs review » Needs work

The last submitted patch, variable-unserialize-error-1284364-39-D7.patch, failed testing.

clemens.tolboom’s picture

This is a verbatim copy of http://drupal.org/node/1445806 as these hints are hopefully enough to write tests.

i just needed 2 days to migrate a D6 website with german "umlaut" (ä, ö, ü) and other special characters (™, ®) in the menu names and description to D7. the problem is that after the migration the following error is shown multiple times on every page.

•Notice: unserialize() [function.unserialize]: Error at offset 54 of 57 bytes in _menu_link_translate() (Zeile 869 in /var/www/html/includes/menu.inc).

additional it is not possible to edit the menu with such links after the migration. ther following error is shown on menu, list links:

•Recoverable fatal error: Argument 3 passed to l() must be an array, boolean given, called in /var/www/html/modules/menu/menu.admin.inc on line 102 and defined in l() (Zeile 2339 von /var/www/html/includes/common.inc).

the only way i got it running was to modify all menu links and desctiptions on D6, then migrate to D7.

not sure from where it comes and if it has to do with this problem - the mysql db from the hoster i use is utf8_general_ci, but some tables (also the drupal_menu_links) is latin1_swedish_ci.

the error can be reproduced very easy.

Note the differences in table encoding/collating. Are these pages relevant?
http://dev.mysql.com/doc/refman/5.0/en/charset-collations.html esp. http://dev.mysql.com/doc/refman/5.0/en/charset-binary-op.html

xjm’s picture

Title: Harnes variable_initialize against broken values - Notice: unserialize() [function.unserialize]: » Use variable_initialize() against broken values - Notice: unserialize() [function.unserialize]:

Updating the title because it bugs me every time this issue is updated. :P

ShannonMcG’s picture

Version: 8.x-dev » 7.12

I started seeing this same error after updating Drupal Core to 7.12.

"Notice: unserialize(): Error at offset 34 of 39 bytes in variable_initialize() (line 943 of /includes/bootstrap.inc)."

My update_notify_emails variable has a length of 39 so I'm guessing this is this issue.

When I try to edit update_notify_emails in Variable Editor (/devel/variable/edit/update_notify_emails) I can't. It shows "Old value" and "New value" but no form fields.

I installed Variable Check and it identified update_notify_emails as an issue.

I deleted update_notify_emails and the problem seems resolved.

marcingy’s picture

Version: 7.12 » 8.x-dev

Moving back to d8 bugs are fixed in d8 first and then backported.

clemens.tolboom’s picture

@Shannon McGlathery: what did you do to upgrade your drupal?

Moving to another host/database/? Was the Issue Summary helpful?

madhusudan’s picture

Status: Needs work » Needs review
cweagans’s picture

Issue tags: +Needs backport to D7
tim.plunkett’s picture

Status: Needs review » Needs work
Issue tags: -needs backport to 6.x +Needs backport to D6

CNW for tests.

xjm’s picture

Let's also update the issue summary to indicate that this is caused by bad variable data and document how that data might get corrupted. Also mention known culprits (if any).

xjm’s picture

Issue summary: View changes

Added image with exception.

tracycarroll’s picture

Issue summary: View changes

Added issue summary

tracycarroll’s picture

Issue summary: View changes

Edit Issue Summary

tracycarroll’s picture

Issue summary: View changes

Edit Issue Summary

tracycarroll’s picture

Issue summary: View changes

Edit Issue Summary

xjm’s picture

Summary was updated by tracycarroll during NYC Camp 2012. Thanks Tracy!

eltermann’s picture

It seems that site_name is no longer stored as a variable, but rather handled by CMI.

New "steps to reproduce" is required.

gmellema’s picture

kbasarab’s picture

With CMI in place this issue changes quite a bit it seems. Discussed with Cottser and zendoodles in IRC and next steps is I'm going to take the tests that have been written in #14 as a starting point and write that test to check if data is serialized in the "config" table rather than "variable" table.

We'll probably want to check with heyrocker or someone working on CMI Core to see if the other portion to this issue is relevant still for 8.x: If there is a broken value when we initialize.

gdd’s picture

The config table works completely differently than the old variables table, so I don't think this issue applies to the config system at all. Additionally, with almost all config moved out of variables, I don't think the original issue applies to D8 anymore. It is possible that some state information may remain in variables for D8, but that wouldn't apply to this issue as far as I can see.

So I think this no longer applies for D8 and should be moved back to being a D7 issue.

kbasarab’s picture

Version: 8.x-dev » 7.x-dev
Issue tags: -Needs backport to D7

Moving version to D7 as this is not applicable to D8.

Next steps:
Apply #14 to 7.x and ensure proper testing.

Then we can reconvene any other issues/functionality.

kbasarab’s picture

Status: Needs work » Needs review
FileSize
2.97 KB
1.61 KB

Lets see what happens with this. This rolls up #14 into HEAD and then adds testing in for variables. Failure is on the PHP Exception that is thrown with variable_initialize().

clemens.tolboom’s picture

@kbasarab what was wrong with #39? That has better wording iirc.

kbasarab’s picture

I mistook and didn't realize #39 had the D7 patch as well there. I'm perfectly ok with using that as the original patch. The main goal of my patch was to get the Testing working and integrated and get this issue moving again and committed. I'd be happy to integrate #39. Probably won't get to it until later today or possibly next week though.

kbasarab’s picture

Integrates #39 and keeps test cases from #64.

Status: Needs review » Needs work

The last submitted patch, variable-unserialize-error-1284364-67.patch, failed testing.

kbasarab’s picture

Oops. Threw the D8 cache set in instead of the D7. Try this again.

This will fail though because of the exception that throws now. I need to look up how to test if an Exception is thrown again.

EDIT:
Tried this to catch the exception. Gotta do more research as to why this fails as $this->expectException() function isn't found.

try {
     $this->expectException();
     // Clear the variable cache to reread the new value                 
     cache_clear_all('variables', 'cache_bootstrap');                    
    // Reseed the variables. This should load $name
    variable_initialize();                                              
  } catch (Exception $ex) {
  $this->pass();
}
clemens.tolboom’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, variable-unserialize-error-1284364-69.patch, failed testing.

kbasarab’s picture

Status: Needs work » Needs review
FileSize
3.02 KB
1.48 KB

Figured out the exception catching. Here is the patch.

kbasarab’s picture

Issue tags: -Needs tests

Removing "Needs tests" as those have been added now.

chx’s picture

Looks good to me but serialize(FALSE) needs to be stored in a variable outside of the loop to avoid calculating that over and over again and also !isset($conf[$name]) this should not be there because it is still wrong IMO to have a broken variable in the database even if we override it. Or we want $conf to cover errors? Discuss.

jackson_lee’s picture

Your workaround states this sql code to find the offending messages in the variable table:-

SELECT name, length(value), value FROM variable where length(value) < Y - 5 and length(value) > Y + 5

but although I'm no great shakes as a programmer, the less than and greater than - aren't they the wrong way round?

e.g. shouldn't the statement be:-

SELECT name, length(value), value FROM variable where length(value) > Y - 5 and length(value) < Y + 5

then it worked for me by finding the offending rows and I was able to fix them.

Mine were caused by transferring from windows host to linux host with /r/n transfers in sql inserts.

cafuego’s picture

Status: Needs review » Needs work
+++ b/includes/bootstrap.incundefined
@@ -898,7 +898,28 @@ function variable_initialize($conf = array()) {
+        $err_str = "The variable(s) '@err_vars' are unreadable and need to be fixed. Please read UPGRADE.txt on how to resolve this problem.";

I see no documentation pertaining to resolving unserialize() problems in UPGRADE.txt. Not in there now and none is in this patch.

I've said it before and I'll say it again: if you make people's sites stop working on a bad variable you had better have clear instructions for resolving the problem and getting their site going again.

Xiuqing Li’s picture

Status: Needs work » Needs review
ryan.prather’s picture

Version: 7.x-dev » 7.20

I was getting a similar error to this on line 916 of bootstrap.inc. Inside that else block I just added a try/catch block and that fixed it. Why not do that at each spot?

Summit’s picture

Hi @ryanprather, could you say that in code please? I use the variable info module which tells me which variable has problems!
Greetings, Martijn

Summit’s picture

Issue summary: View changes

Edit Issue Summary

usama009’s picture

due to large database (47 mb) it ask me to resubmit but when i resubmit it gives me duplicate table or variable error but when i open website in localhost it open along with error mention blow but when i click on admin interface it won't open does any one knew about that error i am new in drupal and want to fix this error thank you

Notice: unserialize(): Error at offset 23 of 69 bytes in variable_initialize() (line 935 of E:\xampp\htdocs\testing\includes\bootstrap.inc).
Notice: unserialize(): Error at offset 23 of 232 bytes in variable_initialize() (line 935 of E:\xampp\htdocs\testing\includes\bootstrap.inc).

andreavr’s picture

clemens.tolboom’s picture

Version: 7.2 » 7.x-dev

@usama009 what's wrong with the issue summary? Just follow the several resolutions

jcharlesberry’s picture

Hello,
I entered incorrect paths into the XHprof settings in devel and the site went WSOD. After I set devel_xhprof_enabled in the variable table of the database to 0, then the site came back up. Now running Variable Check I get 2 unused variables: devel_xhprof_directory (empty value) and devel_xhprof_enabled (0 value). Are these safe to delete?

MarkDQ’s picture

I copied a site (files and db) for development purposes:

  • root: public_html copied to public_develop
  • db: dbprod exported with "drush bam.backup" and restored in dbdevelop using phpmyadmin

The development web site returned immediately

Notice: unserialize(): Error at offset 216 of 228 bytes in variable_initialize() (line 1174 of /home/xxx/public_develop/includes/bootstrap.inc).

Executing the SQL query returned:

  • name = maintenance_mode_message
  • length = 228
  • value BLOB =
    s:209:"The message of the maintenance mode...";

Wondering why an identical copy of a site generates the error, while the original site does not?
The length(value) in dbprod was 218, in dbdevelop 228... is it caused by bam.backup or phpmyadmin or ...?

cafuego’s picture

Is there a difference in character set between the export and import databases? Are there non-ASCII characters in the maintenance mode message?

MarkDQ’s picture

@cafuego No non-ASCII characters in the message (but layout codes starting with \ for a new line etc.)

crystaldawn’s picture

I found this to be much more effective at finding bad variables in the variables table since it actually tests them against PHP unserialize rather than try to guess at it with an SQL query which is very inefficient and super prone to errors.

drush eval '$query = db_query("SELECT name, value FROM {variable}"); while ($variable = $query->fetchObject()) { $value = @unserialize($variable->value); if ($value === FALSE && $variable->value !== serialize(false)) { echo "Variable " . $variable->name . " has serialization issues\n"; } }'
crystaldawn’s picture

Issue summary: View changes
crystaldawn’s picture

Issue summary: View changes