So, I updated Drupal/Webform/Webform Civicrm Integration in a not-thoughtful-enough way on my multi-site drupal installation.

However, after a clumsy updating process, 2/3 of my sites are now using Webform 4 and seem to have automatically updated their conditionals and tokens (thank you to whoever made that happen!).

However, one site will not update. I updated the conditionals manually, and I can update the tokens manually, but I know for a fact that some of the tokens are not working (specifically, civicrm tokens that used to work before and that I have reformatted from %value format to [submission:values:...] format).

The tokens are processing (the text of the tokens does not slow up with my reformatted ones, as it shows for the %value ones) but they do not pull data -- they just show up blank.

Because tokens that updated on the other sites are still in the %value format in this site, I know that some kind of update process did not happen. And, I worry that even if I update tokens manually, there are other hidden pieces missing of the puzzle missing.

Also, when I updated the other two sites, I got a note saying that webform conditionals had been updated. Running the update.php script on this problem site, I get no such message.

I feel like the Webform Conditional script that is meant to update to the new conditional process in Webform 4 is not running. Is there anyway to force it to run?

FWIW, I had Mollom activated on the problem site. But, even when I try to update with Mollom disabled, I can't get any sense that the conditionals are updating.

Any thoughts? I'm sure that I'm doing something wrong, but I can't figure out what it is!

Thanks in advance for any avenues to try.

Comments

quicksketch’s picture

quicksketch
he/him
commented

automatically updated their conditionals and tokens (thank you to whoever made that happen!).

Yay, I'm glad that worked on at least two of three sites. :)

In the event of a failed upgrade, usually Drupal will abandon the update process and stop in the middle of an update. This usually means that you can try running update.php a second time, and it will start where it left off. Your site thinks that Webform is fully up-to-date, it won't be able to re-run any previous updates, since running some updates twice may cause corrupted data.

The token replacement is the very first update in the 4.x branch, so if it failed, I'm surprised that anything would be working at all (there have been 14 subsequent updates since that one). Other than the failed token and conditionals, does everything else seem to be working?

If at all possible, the best approach to solving this problem is restoring a database backup of the 3.x branch, run the update from scratch, and pin down where in the update process it's broken. Then we can fix that problem, restore the database again, and run the whole thing from start to finish. Trying to repair a broken/corrupted structure can be a lot more trouble than starting fresh. Plus this way we'd be able to prevent other people who are upgrading from having whatever this problem is.

So if you can, would you start with the 3.x version database and try upgrading again? Do you get any errors during the update?

edboost’s picture

edboost commented

Hi quicksketch,

Thanks for the speedy reply (and the pretty smart update procedures!).

Shockingly, the module is working and the complicated webform that is giving me so much headache is actually submitting (with Civicrm integration and all!). The site is live and I didn't pull tbe webformdown because it's not used that frequently. Of course, someone used it last night and I'm thankful that it actually worked (with a few tiny glitches in the data -- all fixable).

I would be happy to go back to the 3.x version database, but I'm ashamed to tell you that although I have taught myself a lot about Drupal (building several sites from scratch, maintaining, etc.)... I don't actually know how to do this (quite a testament to how great things work and to my continuing newbieness!). :(

I'll hop around drupal.org and try to figure it out (if you have a link to a good step by step or any advice -- please feel free to share -- I really have no idea where to begin).

I do have a backup ... so that's a start.

In the meantime... as we go through this process, what info should I make sure to catch so that we can troubleshoot?

Thanks for your help!

edboost’s picture

edboost commented

Found this, which looks straightforward enough.

https://drupal.org/node/345225

So, drop my current db and put my old backup back in its place, right?

Then run update.php?

And just check the log for errors (and obviously any errors that pop up on the update page)? Or is there somewhere else to look so that we can track down the issue.

I'm guessing that Mollom was the problem. Should I disable/uninstall that before I proceed? Or leave it in place?

Thanks!

quicksketch’s picture

quicksketch
he/him
commented

So, drop my current db and put my old backup back in its place, right?

Yep! Though of course I'd recommend backing up your current database again before you drop it. :)

Should I disable/uninstall that before I proceed? Or leave it in place?

It'd be great if you could do the exact same thing you did before, to try and cause the problem again. If the problem reoccurs, check your system's PHP logs (and Drupal's Recent Log Entries) for errors that may have happened during the time of the updates (so it's good to record when you start the updates as well).

edboost’s picture

edboost commented

OK, have reverted and tried to update two times with similar results.

The first time, I ran into this rules issue:

The second time, with the newest version of Rules, I do not seem to be having that issue (although, important to note, we did use the Commerce Kickstart a long time ago to set up our Commerce stuff... and I think I have some rules enabled (e.g., when people who are not logged in try to fill out these webforms, they are told that they should make an account -- so perhaps that is why these webforms are not updating?

Here is the message I get after update.php runs:

The following updates returned messages

webform module

Update #7409
New webform columns added.
Update #7410
Removed "teaser" column. All forms that had the "Show complete form in teaser" option disabled will now show forms in their teasers. Use view modes to hide the form if desired.
Update #7411
Replaced tokens using [submission:values:x] with [submission:values:x:withlabel].
Update #7413
New webform columns added.
Update #7414
Custom email columns successfully changed.
commerce_checkout module

Update #7101
A new local action link on order edit forms for simulating checkout completion for an order has been disabled by default; enable it on the order settings form if desired.
Update #7102
A new core checkout completion rule has been added that updates order creation timestamps to the time of checkout completion. It has been disabled by default to not interfere with existing order workflows, but you may enable it in your checkout settings if desired.
commerce_cart module

Update #7101
A new local action link on order edit forms for applying pricing rules to an order has been disabled by default; enable it on the order settings form if desired.
Update #7102
New order settings have been added to let you reduce the frequency of the shopping cart refresh. This update set it to occur continuously as it has been, but we recommend you implement some delay unless you have a unique product pricing situation that demands pricing updates every time a shopping cart is loaded.
panels module

Update #7302
Added panels_pane.uuid column. Added panels_display.uuid column. Generated UUIDs for database-based panel displays and panes.
webform_conditional module

Update #7201
38 webforms using same page conditionals updated to the new conditional system. You may now uninstall the Webform Conditional Module.

Perfect, right? Except that neither tokens nor conditionals seem to actually be updating.

We have one token on the Webform Configuration page that is my "test token" -- and it is still in %title format (the subject of the email that gets sent when webforms are submitted).

My drupal logs show nothing out of the ordinary.

Ahh, but my php logs show that the Rules issue continues to be a problem:

[10-Apr-2014 12:12:46 America/Los_Angeles] PHP Fatal error: Class 'RulesEventHandlerEntityBundle' not found in /home/vsource4/www/www/sites/all/modules/rules/modules/node.rules.inc on line 147
[10-Apr-2014 12:12:57 America/Los_Angeles] PHP Fatal error: Class 'RulesEventHandlerEntityBundle' not found in /home/vsource4/www/www/sites/all/modules/rules/modules/node.rules.inc on line 147
[10-Apr-2014 12:14:08 America/Los_Angeles] PHP Fatal error: Class 'RulesEventHandlerEntityBundle' not found in /home/vsource4/www/www/sites/all/modules/rules/modules/node.rules.inc on line 147
[10-Apr-2014 12:14:09 America/Los_Angeles] PHP Fatal error: Class 'RulesEventHandlerEntityBundle' not found in /home/vsource4/www/www/sites/all/modules/rules/modules/node.rules.inc on line 147
[10-Apr-2014 12:14:24 America/Los_Angeles] PHP Fatal error: Class 'RulesEventHandlerEntityBundle' not found in /home/vsource4/www/www/sites/all/modules/rules/modules/node.rules.inc on line 147

What I can't figure out -- if this is holding everything up, why do the updates seem to run?

Anyway, back to the Rules thread for me to try to figure this out... but, quicksketch, do you think this is the problem? And why would the update return such a rosy picture if it is erroring out?

Thanks again for your help, and please let me know if there is other info I can gather to help figure this out.

edboost’s picture

edboost commented

Several more tries later:

I seem to be able to "fix" rules by installing the latest dev version and then using the rules_fix.php script suggested here: https://drupal.org/node/2090511#comment-8462723

When I install the new Rules, here are the errors I get up running update.php:

[10-Apr-2014 13:34:38 America/Los_Angeles] PHP Fatal error: require_once() [function.require]: Failed opening required '/home/vsource4/www/www/sites/all/modules/rules/includes/rules.plugins.inc' (include_path='.:/usr/local/php53/lib/php') in /home/vsource4/www/www/includes/bootstrap.inc on line 3139
[10-Apr-2014 13:34:41 America/Los_Angeles] PHP Fatal error: require_once() [function.require]: Failed opening required '/home/vsource4/www/www/sites/all/modules/rules/includes/rules.core.inc' (include_path='/home/vsource4/www/www/sites/default/files/civicrm/extensions/org.civicrm.sms.clickatell/:.:/home/vsource4/www/www/sites/all/modules/civicrm:/home/vsource4/www/www/sites/all/modules/civicrm/packages:.:/usr/local/php53/lib/php') in /home/vsource4/www/www/includes/bootstrap.inc on line 3139
[10-Apr-2014 13:34:45 America/Los_Angeles] PHP Fatal error: require_once() [function.require]: Failed opening required '/home/vsource4/www/www/sites/all/modules/rules/includes/rules.core.inc' (include_path='/home/vsource4/www/www/sites/default/files/civicrm/extensions/org.civicrm.sms.clickatell/:.:/home/vsource4/www/www/sites/all/modules/civicrm:/home/vsource4/www/www/sites/all/modules/civicrm/packages:.:/usr/local/php53/lib/php') in /home/vsource4/www/www/includes/bootstrap.inc on line 3139
[10-Apr-2014 13:34:50 America/Los_Angeles] PHP Fatal error: require_once() [function.require]: Failed opening required '/home/vsource4/www/www/sites/all/modules/rules/includes/rules.core.inc' (include_path='/home/vsource4/www/www/sites/default/files/civicrm/extensions/org.civicrm.sms.clickatell/:.:/home/vsource4/www/www/sites/all/modules/civicrm:/home/vsource4/www/www/sites/all/modules/civicrm/packages:.:/usr/local/php53/lib/php') in /home/vsource4/www/www/includes/bootstrap.inc on line 3139
[10-Apr-2014 13:35:07 America/Los_Angeles] PHP Fatal error: require_once() [function.require]: Failed opening required '/home/vsource4/www/www/sites/all/modules/rules/includes/rules.plugins.inc' (include_path='.:/usr/local/php53/lib/php') in /home/vsource4/www/www/includes/bootstrap.inc on line 3139

Then I run the fix and I see no errors.

The problem is, that fix comes after update. My webform issue seems to come during update. So, if it is the Rules issue that is messing up the update for all, I can't figure out how to fix it.

And, as the site still reports, upon updating, that it IS updating conditionals and tokens, is that really the problem?

A few more tries, including installing the latest (today!) version of Webform to see if I could trick it into updating again. No luck.

Not even sure where to look anymore. :(

edboost’s picture

edboost commented

One more bit of info:

My last attempt to update with the newest webform and webform conditionals 7-2 in place threw these errors:

[10-Apr-2014 13:59:14 America/Los_Angeles] PHP Fatal error: Call to undefined function webform_menu_load() in /home/vsource4/www/www/includes/menu.inc on line 593
[10-Apr-2014 13:59:42 America/Los_Angeles] PHP Fatal error: require_once() [function.require]: Failed opening required '/home/vsource4/www/www/sites/all/modules/webform/views/webform_handler_filter_webform_status.inc' (include_path='.:/usr/local/php53/lib/php') in /home/vsource4/www/www/includes/bootstrap.inc on line 3139
[10-Apr-2014 13:59:59 America/Los_Angeles] PHP Fatal error: Call to undefined function webform_menu_load() in /home/vsource4/www/www/includes/menu.inc on line 593
[10-Apr-2014 14:12:00 America/Los_Angeles] PHP Parse error: syntax error, unexpected T_ENCAPSED_AND_WHITESPACE, expecting ')' in /home/vsource4/www/www/sites/all/modules/webform/webform.module on line 1941
[10-Apr-2014 14:12:00 America/Los_Angeles] PHP Parse error: syntax error, unexpected $end in /home/vsource4/www/www/sites/all/modules/webform/webform.module on line 2316

Just in case that helps.

quicksketch’s picture

quicksketch
he/him
commented

The following updates returned messages

webform module

Update #7409
New webform columns added.

Wait, did you get this message the *first* time you tried to update? Webform's 4.x updates start at number 7401. If the first 8 updates aren't being run, that would explain things.

It sounds like your last attempts broke Webform entirely:

[10-Apr-2014 14:12:00 America/Los_Angeles] PHP Parse error: syntax error, unexpected T_ENCAPSED_AND_WHITESPACE, expecting ')' in /home/vsource4/www/www/sites/all/modules/webform/webform.module on line 1941

That would probably prevent your entire site from working correctly. Sounds like the webform.module file itself ended up with stray characters.

edboost’s picture

edboost commented

Yes,

I think that has, consistently, been the first update that I get with a fresh database.

Any thoughts on how to fix it and get the first 8 updates to run?

For now, reverting, rolling back, disabling Rules, and trying this one more time.

edboost’s picture

edboost commented

Just tried again and the 7409 update is definitely the first one.

quicksketch’s picture

quicksketch
he/him
commented

Just tried again and the 7409 update is definitely the first one.

This is definitely the main problem. The issues with Rules might just be a red herring. To get the updates to run from the beginning, you'll need to update the entry in the "system" database table to set the update number back to 7300. (Really it should be whatever 73xx number your 3.x version of the module last ran, but that might be difficult to determine).

Running this SQL query should set the schema to the beginning of the 4.x updates:

UPDATE system SET schema_version = 7400 WHERE name = 'webform';

Then try running update.php and it should start from the 7401 update for Webform.

edboost’s picture

edboost commented

I feel like there's hope!

reset the schema_version and got this:

The following updates returned messages

webform module

Update #7401
Your existing webforms have been upgraded to use the global Drupal 7 token system.
Update #7402
Failed: DatabaseSchemaObjectExistsException: Table webform_conditional already exists. in DatabaseSchema->createTable() (line 657 of /home/vsource4/www/www/includes/database/schema.inc).

So, it's starting from the beginning.

Is it erroring out because some of the changes have already taken place?

Revert again and start from scratch?

quicksketch’s picture

quicksketch
he/him
commented

Is it erroring out because some of the changes have already taken place?

Hm, yes. You'll need to completely empty your database and restore your webform 3.x version of the database. Did you have the "Webform conditional" module installed on this site?

It also sounds like what may have happened on this site is that it was upgraded to Webform 4.x at some point, but then downgraded back to 3.x. That would explain both why the "webform_conditional" table already exists and why your schema version is at 7409 instead of at a 73xx number. I have a bad feeling that any conditionals you have on this site aren't going to be able to be upgraded.

If this upgrade-then-downgrade situation is what happened, then I'd actually suggest you don't start over. Instead just skip the conditional upgrade function and do the next ones.

UPDATE system SET schema_version = 7404 WHERE name = 'webform';

Then run update.php again. Updates 7403 is the one that was supposed to create the webform_conditional table and 7404 is the update to convert old conditionals to the new ones.

edboost’s picture

edboost commented

Reverted and updated two more times, once with schema_version set to 7300 and once with it set to 7400.

Same error:

The following updates returned messages

webform module

Update #7401
Your existing webforms have been upgraded to use the global Drupal 7 token system.
Update #7402
Failed: DatabaseSchemaObjectExistsException: Table webform_conditional already exists. in DatabaseSchema->createTable() (line 657 of /home/vsource4/www/www/includes/database/schema.inc).
panels module

Update #7302
Added panels_pane.uuid column. Added panels_display.uuid column. Generated UUIDs for database-based panel displays and panes.
commerce_cart module

Update #7101
A new local action link on order edit forms for applying pricing rules to an order has been disabled by default; enable it on the order settings form if desired.
Update #7102
New order settings have been added to let you reduce the frequency of the shopping cart refresh. This update set it to occur continuously as it has been, but we recommend you implement some delay unless you have a unique product pricing situation that demands pricing updates every time a shopping cart is loaded.
commerce_checkout module

Update #7101
A new local action link on order edit forms for simulating checkout completion for an order has been disabled by default; enable it on the order settings form if desired.
Update #7102
A new core checkout completion rule has been added that updates order creation timestamps to the time of checkout completion. It has been disabled by default to not interfere with existing order workflows, but you may enable it in your checkout settings if desired.
webform_conditional module

Update #7201
38 webforms using same page conditionals updated to the new conditional system. You may now uninstall the Webform Conditional Module.

Maybe I have to update the webform_conditional schema-version also?

quicksketch’s picture

quicksketch
he/him
commented

Maybe I have to update the webform_conditional schema-version also?

If you have webform_conditional.module installed, you have to uninstall it before trying to upgrade to Webform 4.x. Or if you like, just drop the webform_conditional table.

But as I said before, I think this is an issue because Webform was upgraded previously, downgraded, and now being upgraded again. If this is the case, you'll already have the webform_conditional and webform_conditional_rules database tables. Options: You can either drop both tables and try upgrading from 7400 again. Or you can leave them, run from 7400, get the error, and then run from 7404.

edboost’s picture

edboost commented

I'm sorry. I misunderstood. I thought that I had to have webform_conditional (the upgrade script version) installed to push the conversion.

I went with option 2 (running from 7404) and updates began with 7409. Does that seem correct? Or, should they have started at 7405?

Tokens did update to the new version. But they still are not working (at least the civicrm ones are not). They are processing, but they show up blank.

Also, conditionals did not update. Not a big deal to update manually, but seemed worth mentioning in case it's a symptom of something else.

Thanks so much for all the help.

quicksketch’s picture

quicksketch
he/him
commented

After the trouble in this issue, I reworked a few of our updates in #2239079: Increase the robustness of our update hooks where possible, to help prevent these kind of partial-upgrade issues. With those changes (included in the RC1 version), you should have a much easier time updating. I'd recommend starting over (again), download RC1, start the updates from 7400 as described in #13, and run updates again. The new checks in the updates should gracefully skip the updates that already have been run and execute the ones that haven't been run. It'll also check for the existence of webform_conditional module, providing recommendations to uninstall the module if necessary.

edboost’s picture

edboost commented

Quicksketch,
Thanks again for all of the help. Everything is working as it should, but I will follow your recommendations above to feel like things are done properly.
Thank you so much!
-edboost

quicksketch’s picture

quicksketch
he/him
commented
Status: Active » Closed (fixed)

Phew, well glad we got that all worked out. From what I can tell Webform itself is all set up okay, and the fixes I put in #17 should help others from getting into pickles if they re-run updates.