drush update destroys menu_links table entries for book

drush has no code to delete these links. please try without any contrib modules. someone will need to trace this and provide more information.

Lots of people are reporting this issue over at http://drupal.org/node/467770

I can duplicate the problem using a clean install with only book module enabled.

The steps are outlined in http://drupal.org/node/467770#comment-2493896

I'm not sure how to proceed with tracing this bug but will take a look in the php.

is there a way to make drush output any SQL it executes?

I've made sql logs of all the calls made by drush, and all the calls made by update.php.

Drush has these at the end, where update.php does not.

	  426 Query	DELETE FROM menu_links WHERE mlid = 3
		  426 Query	DELETE FROM menu_links WHERE mlid = 4
		  426 Query	DELETE FROM menu_links WHERE mlid = 5
		  426 Query	DELETE FROM menu_links WHERE mlid = 6
		  426 Query	DELETE FROM menu_links WHERE mlid = 7
		  426 Query	SELECT mlid FROM menu_links WHERE plid = 11
		  426 Query	DELETE FROM menu_links WHERE mlid = 11
		  426 Query	DELETE FROM menu_links WHERE mlid = 12
		  426 Query	DELETE FROM menu_links WHERE mlid = 13
		  426 Query	DELETE FROM menu_links WHERE mlid = 19
		  426 Query	DELETE FROM menu_links WHERE mlid = 21
		  426 Query	DELETE FROM menu_links WHERE mlid = 31
		  426 Query	DELETE FROM menu_links WHERE mlid = 44
		  426 Query	DELETE FROM menu_links WHERE mlid = 78
		  426 Query	DELETE FROM menu_links WHERE mlid = 81
		  426 Query	DELETE FROM menu_links WHERE mlid = 83
		  426 Query	DELETE FROM menu_links WHERE mlid = 84
		  426 Query	DELETE FROM menu_links WHERE mlid = 85
		  426 Query	DELETE FROM menu_links WHERE mlid = 92
		  426 Query	DELETE FROM menu_links WHERE mlid = 95
		  426 Query	DELETE FROM menu_links WHERE mlid = 112
		  426 Query	DELETE FROM menu_links WHERE mlid = 113
		  426 Query	DELETE FROM menu_links WHERE mlid = 114
		  426 Query	DELETE FROM menu_links WHERE mlid = 122
		  426 Query	DELETE FROM menu_links WHERE mlid = 123
		  426 Query	SELECT mlid FROM menu_links WHERE plid = 124
		  426 Query	DELETE FROM menu_links WHERE mlid = 124

Looks like there is an older issue at #467770: Book structure disappears repeatedly that just got moved to the drush queue from core, so lets continue over there.

Oh wait, nevermind - looks like that one was already marked as a duplicate

This problem has affected me on two sites - so I'm very keen for a resolution.

It would seem from the above reports that running updates with Drush does cause the above error, but from my look at the Drush code, it equally seems unlikely (I looked in drush/commands/core/drupal/update_6.inc) as there simply doesn't seem to be any commands that do this type of thing.

Could it be then that this is a core function that isn't being run properly when initiated through Drush?

We have no confirmation yet at #467770: Book structure disappears repeatedly that all affected site administrators are using Drush. Also I am updating several sites through Drush that don't show the disappearing-books-problem (yet).

Greetings, -asb

@asb: what version of drush are you using? 2.1?

I think this may be something to do with the book_update_6000() function which updates from drupal 5.x to 6.x. It seems that this is being executed by Drush when it shouldn't.

I think thats because its running a multi-part update depending on whether $_SESSION is set - can Drush even have a session from the command line?

function book_update_6000() {
  $ret = array();

  // Set up for a multi-part update.
  if (!isset($_SESSION['book_update_6000'])) {

    $schema['book'] = array(
      'fields' => array(
        'mlid'    => array('type' => 'int', 'unsigned' => TRUE, 'not null' => TRUE, 'default' => 0), 
        'nid'     => array('type' => 'int', 'unsigned' => TRUE, 'not null' => TRUE, 'default' => 0), 
        'bid' => array('type' => 'int', 'unsigned' => TRUE, 'not null' => TRUE, 'default' => 0), 
      ),  
....

Wow, that update is appalling in its abuse of the batch api. That update is simply out of spec and should be changed. If you want to see this fixed, I suggest pushing along #198980: code multipass updates by the book. You can use the patch there and see if it prevents this bug. Until the patch is proven, book using sites should use good ole update.php to upgrade to D6.

Dupe of #198980: code multipass updates by the book

Not sure that this is duplicate. Even if the update code is bad, Drush should not be running it every time. update.php doesn't do it.

#692210: Drush is destroying some navigation and #670340: Admin module - menu items disappear when I run drush update are related. This is a bigger problem than just book.

update.php doesn't do it because that update is authored to work only with update.php

"that update is authored to work only with update.php"--good point! So why is Drush forcing it to execute?

Drush is running code it should not be running.

Drush is running code it should not be running.

It is pretty clear that you are not familiar with updatedb's purpose nor its internals. As such, please do not override a status set by a maintainer.

[deleted comment]

Ach. Bother. And I was becoming so fond of drush.

subscribing

Note #12 of #198980: code multipass updates by the book: "I removed the book_update_6000() function altogether and drush up still kills the menu structure..." Does this need more investigation?

True that. It sure does.

I've encountered this countless times not really knowing why it was happening. I Drush day in, day out.

Anyway, the fix was simple, should you have a backup of your menu_links table before executing "drush up"

SELECT * FROM menu_links WHERE menu_name LIKE "book-toc%";

Export that to an SQL dump. Re-import it. Bada Bing, Bada Boom.

That saved me at least.

Oliver
President
www.web.net

Subscribe

While I didn't manage to reproduce the problem, I found the Trace module useful in telling me what is going on (and yes, I found a bug in my theme along the way).

Note: Please read the usage warnings. You don't want to run this on a busy site; I ran it when I could ensure that only I was accessing my site.

@ozielke (#22):

Thanks for the tipp. This works, indeed, at least to some degree. However, the problem is, that the book structure disappears again quite soon. To make this usable one would have to
* dump the menu_links table after every edit in the book structure,
* restore these backups on a regular basis, e.g. based by cron runs every day.

It is obviously that such a workaround would fall apart after a very limited amount of time.

I've just been bitten by this issue on a second site, without having done any significant upgrades (no core update, also the site is native D6 and as such never experienced a major core upgrade). So far, I'm experiencing this issue only on native D6 setups; so far all migraded sites started on D5 or earlier are still running properly after the D6 upgrade and several module upgrade with Drush.

Greetings, -asb

I decided to take a chunk of time out today to try and find concrete reproduce steps for this problem. Although the issue is also discussed in #467770: Book structure disappears repeatedly, this particular reproduce case is drush-related. Here are the steps to reproduce the issue:

1. Install Drupal 6.14.
2. Enable the book module.
3. Create a hierarchy of book page nodes.
4. Update to Drupal 6.15 by replacing the source code and running update.php.
5. Install drush 2 somewhere on your system.
6. Run "drush update" on your site.

After step 6, the book hierarchy will have disappeared (but not before then).

Significantly, if you replace drush 2 with drush 3 and repeat these same steps, the book hierarchy _doesn't_ disappear.

Will report more as my tests proceed.

Thanks!
Adam

OK, more to report. The reproduce case from the my last comment (#26) can actually be reduced to the following:

1. Install Drupal 6.14.
2. Enable the book module.
3. Create a hierarchy of book page nodes.
4. Install drush 2 somewhere on your system.
5. Run "drush update" on your site.

In other words, the update from 6.14 to 6.15 isn't necessary to reproduce the problem. I imagine the problem would also occur in the same way on a native 6.15 install, though I haven't tried that yet.

Thanks!
Adam

Also worth noting: the cli output for drush2 and drush3 differ slightly:

nt6tb3yh1:drupal-6.14 jazzslider$ ../drush3/drush update
Warning: The command name "update" is deprecated.  Please use a recommended form instead (pm-update,up).                                             [warning]
Refreshing update status information ...
Done.
Update information last refreshed: Wed, 02/24/2010 - 13:49

Update status information on all installed and enabled Drupal projects:
 Name    Installed version  Proposed version  Status                    
 Drupal  6.14               6.15              SECURITY UPDATE available 

NOTE: A code update for the Drupal core is available. 
Drupal itself can't yet be updated by this tool. Please update Drupal manually.

No code updates available.                                                                                                                           [ok]
No database updates required                                                                                                                         [success]
Finished performing updates.                                                                                                                         [ok]

nt6tb3yh1:drupal-6.14 jazzslider$ ../drush2/drush update
Refreshing update status information ...
Done.
Update information last refreshed: Wed, 02/24/2010 - 13:50

Update status information on all installed and enabled Drupal modules:
 Name    Installed version  Proposed version  Status                    
 Drupal  6.14               6.15              SECURITY UPDATE available 

NOTE: A code update for the Drupal core is available. 
Drupal itself can't yet be updated by this tool. Please update Drupal manually.

No code updates available.                                                                                                                           [ok]
No database updates required                                                                                                                         [success]
Cache cleared.
Finished performing updates.                                                                                                                         [ok]

Two differences I can see? (1) In drush3, the "update" command has been deprecated in favor of "pm-update". That's probably not relevant to this issue. (2) In drush3, the cache is no longer cleared following the update (although it _might_ be cleared if code were actually updated).

Thanks!
Adam

OK, that last post (#28) was a red herring; running "drush cc" (in either version of drush) does not reproduce the problem. Seems to be update-related, as many in this thread have supposed.

Thanks!
Adam

Decided to try this on a Drupal 5 to Drupal 6 upgrade as well, since some users have reported the issue occurring then (and that's where we first noticed it ourselves). However, I don't think it's the 5-6 upgrade that does it, as the following steps do NOT delete the menu links:

1. Install Drupal 5.21.
2. Enable the book module.
3. Create a hierarchy of book page nodes.
4. Update to Drupal 6.15 by replacing the source code and running update.php.

However, if you continue on to install drush 2 and run "drush update" (as in comment #26), the menu links disappear, just as they did on my from-scratch 6.14 install.

From my perspective, this pretty much conclusively proves that the problem is somewhere in drush 2, and is not related to what happens when you run update.php. It also seems clear that the drush maintainers have done something to fix this, since it behaves correctly in drush 3.

Thanks!
Adam

Thanks for all of the detailed work in reproducing this problem. I was just wondering if the recent improvements to drush update might have taken care of this problem. (Interesting, but probably irrelevant, is whether drush-3-alpha-1 will also delete book content.)

@Moshe: So, where we are today is that this issue has been fixed in drush-3, and the drush-2 branch is no longer being maintained. Should we therefore go ahead and mark this issue (and any related issue not already closed as a duplicate) as "won't fix"? I think this gets us pretty close to being able to do a drush-3-rc1 release.

Is this confirmation that there was indeed a problem with Drush?

@aren: yes.

Then shouldn't status and version indicate that this is fixed in 3.0?

This bug is posted against version 2.1; it is "won't fix" there. It was fixed in version 3. The final status of a closed bug is a mighty fine point, but in this case, "won't fix in 2.1" is better than "fixed in 3.0", because the fixed bug list is used to generate release notes, and there is already a closed bug somewhere that covers this fix.

Drush 3.0 beta 1 behaves as it should with "drush up." No broken book pages.

So if I understand this correctly, Drush has destroyed something needed by the book module which might cause that book hierarchies are erased either immediately, or with some time delay; in Drush 3.0 beta 1 the bug causing this damage shouldn't be there anymore, but this release doesn't fix the damage already caused on D6 sites, and it can't/won't repair the stuff that it damaged?

Greetings, -asb

That is correct.

Thanks, ozielke; the questions now would be

a) can the damage be repaired permanently?
b) who can do it?
c) does an appropriate issue queue exist?

Greetings, -asb

[deletia ...]

[deletia ...]

I have no idea what "Drush" is. If it's a module, I'm not using it, and I STILL get disappearing books.

Then don't post issues in the Drush module queue. Try the book module's queue: http://drupal.org/project/issues/drupal?text=&status=All&priorities=All&...

Hello!

@Hetta: for what it's worth, there are some other folks in #467770: Book structure disappears repeatedly that share your concerns; up until comment #47, that issue was posted in the Drupal core issue queue, which is more appropriate for something reproducible w/just the book module (i.e., no drush).

I didn't mean to shut down the concerns of those who have experienced this problem _without_ drush …it's possible that there is another, unrelated problem that exhibits some of the same symptoms. I, however, have only ever been able to duplicate this problem using the drush 2 update command (as in my test cases above). I think that's why this particular issue was (appropriately, IMO) closed …it's about the problem as it occurs with drush.

What I'd suggest (though I'm not really any kind of authority on this) is to first try to duplicate the problem in the simplest way you possibly can without using drush, and then to post your test case either on the issue I linked above, or create a new one (referencing these two issues for context).

Thanks!
Adam