Our server times out when trying to run the upgrade of RC2-RC4, when I get back to the error page it says:

The following views were using the index Product Display: display_products. You should disable or delete them.

Comments

alexp999’s picture

Ok, the two things seem to be unrelated. The "error" is just a message, its the timeout that is the real problem.

I had to increase my timeouts massively, the updates were taking about 2-3mins, there seems to be a massive query being run and even then when I first open up the site, I have to rebuild menus etc, so much has changed, no longer are the product categories FURLs with the name.

I thought these are release candidates? i.e. feature frozen, bug fixing only.

I'm going to have to do so much work getting it back to a usable state as it was pre-update.

bojanz’s picture

That's highly unusual. The updates are not supposed to take that long (and they have been relatively painless for people so far).

How does your installation look like? Did you create a big number of menu items? Blocks? Products? Would like to know what's the main problem here, so we can avoid it with future updates.

alexp999’s picture

We have 175 terms in the product category.

2 extra blocks beyond the default install. (superfish menu and a menu block)

750 products.

brephraim’s picture

Just to throw in my two cents, I also had to up my php timeout limit in order to get the updates to go through.

And, as I mentioned in the other issue, I got errors relating to search API.

bojanz’s picture

Interesting.
Unfortunately, updating Drupal distributions uses the same mechanism as updating between Drupal versions (6 -> 7), and works just as well (barely).
We'll have to be more conservative about future changes, and perhaps limit cleanups and better functionality to new installs only.

@alexp999
Can you rerun the updates (perhaps by installing a backup somewhere, and retrying the update)?
What would help is knowing which update function is taking so long.
My guess is on commerce_kickstart_update_7205(), it indexes a hundred products at the end of the process.
Could you find the line

search_api_index_items($index, 100);

replace it with

search_api_index_items($index, 10);

see if that changes anything?
(Updates are in commerce_kickstart.install)

alexp999’s picture

I tried replacing both instances and it still times out.

If I go back to the update page after the error then update 7205 is the one still outstanding. If you look at the update code there is a lot of stuff happening in that one "update". Including looping through every term and rebuilding the entire category menu.

The irony is, I had to rebuild the menu manually even after I increased the timeouts as it didn't work.

jsacksick’s picture

Title: Update 7205 is prone to timeouts » Error during upgrade from RC2 to RC4
Version: 7.x-2.x-dev » 7.x-2.0-rc4
Priority: Major » Normal

Can you try to replace taxonomy_get_tree($collection->vid); by taxonomy_get_tree($collection->vid, 0, 1);

bojanz’s picture

Title: Error during upgrade from RC2 to RC4 » Update 7205 is prone to timeouts
Version: 7.x-2.0-rc4 » 7.x-2.x-dev
Priority: Normal » Major

Okay, so we need to fix #7205.
It's too late to introduce new update functions, but we can at least fix the situation inside, through batching.

Let's introduce a batch that does the work in steps:
a. Enable the new module, rebuild the menu
b. Disable the old view, update facets
c. Delete the old taxonomy menu, create the new one
d. Delete old pathauto aliases, generate new ones
e. Update the search api index, reindex.

Let's also make sure that when we're doing $terms = taxonomy_get_tree($collection->vid); we are limiting the depth to 1 (additional parameter), so that it doesn't load unneeded terms (like most of the 175 ones that alexp999 has).

This is something we can implement in 24h. After that we'll know whether additional steps are needed.

bojanz’s picture

Also, alexp999 please try #7.
That should tell us if the taxonomy term fetching is our main problem here (and might allow you to complete the update before we roll a fix).

alexp999’s picture

Should #7 be:

taxonomy_get_tree($collection->vid, 0, 1)

or:

taxonomy_get_tree(taxonomy_get_tree($collection->vid, 0, 1))

It looks like there is either a close bracket missing or one too many function calls, or I may just be reading it wrong....

jsacksick’s picture

Title: Error during upgrade from RC2 to RC4 » Update 7205 is prone to timeouts
Version: 7.x-2.0-rc4 » 7.x-2.x-dev
Priority: Normal » Major

taxonomy_get_tree($collection->vid, 0, 1);

bojanz’s picture

It should be $terms = taxonomy_get_tree($collection->vid, 0, 1);
So you're just adding the zero and the one as the arguments to the existing line.

adrianmak’s picture

I came across the timeout issue of the update process

alexp999’s picture

I found the offending bit of code.

taxonomy_pathauto_bulk_update_batch_process($sandbox);

I had tried your suggestion and it made no difference, so I set about working my way through the code of update 7205 and finding what may be causing the timeout.

If I comment out that line on an otherwise untouched RC4 install profile, it sails through no problem. So even with 100 items to index and the entire taxonomy tree to load, it only took about 15 secs to process.

I have of course been restoring the database to a pre update backup each time I have changed something to run it from fresh every time.

vasike’s picture

Status: Active » Needs review

here is a gerrit commit for bojanz #8 directions
https://code.drupalcommerce.org/#/c/541/

bojanz’s picture

Thanks alexp999, your clue was essential.
The problem is, we are passing $sandbox to pathauto, so if that batch decides to call itself again, it restarts the whole update function, causing all the caches to be recleared, etc.

I will fix that on top of vasike's commit, and give you a version to test before committing.

Big thanks!

bojanz’s picture

Status: Needs review » Fixed

This has been merged.

brephraim’s picture

Version: 7.x-2.x-dev » 7.x-2.0
Status: Fixed » Needs work

Still happening on 2.0.

bojanz’s picture

Can you provide us with good instructions to reproduce it? (For example, install RC3, create 100 terms in vocabulary X, attempt upgrade).
Or a sanitized db dump, whatever...

brephraim’s picture

Start from RC3, 49 total taxonomy terms.

Need more info?

An AJAX HTTP error occurred. HTTP Result Code: 200 Debugging information follows. Path: http://localhost/update.php?op=selection&token=nLJkSDF-o9z5sPbJwG2o63BDez52vgFnxdWWj0N_HoM&id=1501&op=do StatusText: OK ResponseText: Fatal error: Maximum execution time of 30 seconds exceeded in /var/www/includes/menu.inc on line 2768 Call Stack: 0.0007 455040 1. {main}() /var/www/update.php:0 4.7013 96893540 2. _batch_page() /var/www/update.php:473 4.7018 96901092 3. _batch_do() /var/www/includes/batch.inc:80 4.7018 96901092 4. _batch_process() /var/www/includes/batch.inc:161 4.7025 96990184 5. call_user_func_array() /var/www/includes/batch.inc:284 4.7025 96990224 6. update_do_one() /var/www/includes/batch.inc:0 4.7025 96990512 7. commerce_kickstart_update_7205() /var/www/includes/update.inc:963 4.7905 97124564 8. taxonomy_pathauto_bulk_update_batch_process() /var/www/profiles/commerce_kickstart/commerce_kickstart.install:857 5.0547 97140352 9. pathauto_taxonomy_term_update_alias_multiple() /var/www/profiles/commerce_kickstart/modules/contrib/pathauto/pathauto.pathauto.inc:193 34.1130 101761716 10. pathauto_taxonomy_term_update_alias() /var/www/profiles/commerce_kickstart/modules/contrib/pathauto/pathauto.module:680 34.1132 101763284 11. pathauto_create_alias() /var/www/profiles/commerce_kickstart/modules/contrib/pathauto/pathauto.module:652 34.1175 101765120 12. _pathauto_set_alias() /var/www/profiles/commerce_kickstart/modules/contrib/pathauto/pathauto.inc:416 34.1175 101765164 13. _pathauto_path_is_callback() /var/www/profiles/commerce_kickstart/modules/contrib/pathauto/pathauto.inc:509 34.1175 101765164 14. menu_get_item() /var/www/profiles/commerce_kickstart/modules/contrib/pathauto/pathauto.inc:468 34.1175 101765208 15. menu_rebuild() /var/www/includes/menu.inc:459 34.8959 102175536 16. _menu_navigation_links_rebuild() /var/www/includes/menu.inc:2714 35.1015 103602928 17. menu_link_save() /var/www/includes/menu.inc:2853 35.1016 103607984 18. _menu_find_router_path() /var/www/includes/menu.inc:3163 35.1016 103607984 19. _menu_router_cache() /var/www/includes/menu.inc:3340

jsacksick’s picture

So you attempted upgrade from RC3 to 2.0 and you got this error ?

brephraim’s picture

My mistake, it was actually RC2.

lsolesen’s picture

Issue summary: View changes

@brephraim Can you try when updating to the latest dev version of CK 2.x?

bojanz’s picture

Status: Needs work » Fixed

I think we're done here.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.