After upgrading from 6.11 to 6.12 my book disappeared and book pages are scattered around. This also happened when I upgraded from 5.x to 6.x

CommentFileSizeAuthor
#21 book-coremodule1.png30.75 KBOle Martin
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

brianV’s picture

Assigned: Ole Martin » Unassigned

Can you verify that the book module is still enabled?

Ole Martin’s picture

Yes, the module is enablet. It was the main book that disappeared while all of my bookpage became to independent nodes without contact to each other.

brianV’s picture

Title: My book dissaper » Book disappears during upgrade leaving orphaned pages
Version: 6.12 » 6.x-dev
aniruddhay’s picture

My book has also disappeared. This did not happen as a result of an upgrade though. I started on my site with 6.12 and it has disappeared midway. I can still see the 'book' directory in the 'module' directory of my site but do not see the book module on the module page.
Thanks in advance for your help

acrollet’s picture

I have also experienced the problem described by Ole Martin. A user on one of our sites created two books with several child pages on Friday, and when she returned Monday, the hierarchy was gone. In this case, no site updates had been performed, and I was unable to find any evidence of related changes in watchdog. The specific problem seems to be that the two entries for the book parent pages had gone missing from the menu_links table.

I was able to restore the hierarchy by following the procedure at the following link:

http://davidherron.com/content/repairing-book-navigation-drupal-site

I appreciate any advice or help anyone can offer...

markpenny’s picture

I've had the same problem for several months through several versions of Drupal. I thought I'd lost all the data, but it looks like it's all there in the database. NIDs, BID, MLIDs are all intact. My guess is some change in the module has caused it to stop grabbing the IDs. Unless some other module is somehow interfering.

Just today I created three books, which are recorded in the database and which are listed as books under Content, but which are not listed under Book outline/Book or in the book list.

Ole Martin’s picture

I have had several experiences with this. I take now further education and made me a book for all my notes. Used Taxonomy Access to begin with. Several of my students, teachers and the school was interested in how this was built up. Changed then from the Taxonomy Access to Organic Groups, so that more students and teachers would have to test this out in a separate group.

Now, it happened again. All my book pages was an orphan. This problem has been in the version 6, and it does not appear that there will be fikset.I use latest stable version of all modules on my site. Do not have much knowledge of programming so I am considering now to stop using this module and either look for another way to write down my notes. Very sorry when I'm simultaneously trying to get promoted Drupal for all other students, teachers and school administrator. The school currently uses Joomla as the main program for all IT-related training.

Here, the developers has a serious problem as they should have been fixed, especially with regard to the big market the school environment is.

markpenny’s picture

This may explain the problem, though I've no idea how to actually solve it. It looks like some sort of migration/conversion is supposed to occur, but hasn't. book.install lines 46-58

/**
 * Drupal 5.x to 6.x update.
 *
 * This function moves any existing book hierarchy into the new structure used
 * in the 6.x module.  Rather than storing the hierarchy in the {book} table,
 * the menu API is used to store the hierarchy in the {menu_links} table and the
 * {book} table serves to uniquely connect a node to a menu link.
 *
 * In order to accomplish this, the current hierarchy is processed using a stack.
 * The stack insures that each parent is processed before any of its children
 * in the book hierarchy, and is compatible with batched update processing.
 *
 *

There are no Book entries in the menu_links table of my site. Since I've created new books using 6.x versions, I'd say that book.module or book.install is not getting something done.

In short, the module is half working. It creates books, but doesn't link the pages.

Drupal 6.14

optidel’s picture

subscribing

Lakeside’s picture

subscribing

asb’s picture

Priority: Normal » Critical

Hi,

similar issue here: the book navigation keeps disappearing on a pure D6 site (started with D6, never upgraded from D5 to D6, but always running the latest D6 core and module releases, thus running update.php frequently; currently running Drupal core 6.14).

Symptoms: Nodes can be added to a book outline with the http://mysite.tld/node/{nid}/outline tab. Sometimes the book hierarchy shows up at ./admin/content/book/list, sometimes it doesn't. It reproducably does dissapear after a few days. This is highly irritating since to my experience the 'book' mechanism has been always very reliable and robust (opposed to the menu system).

First analysis: When looking into the 'book' table database, the node's NIDs are partially connected to BIDs (BID 78); other book hierarchies (BID 4 and 49) just have a single node attached; other nodes are "gone" from the 'book' table. All recorded entries in the 'book' table have menu links (MLID) :

mysql> select * from book order by bid;
+------+-----+-----+
| mlid | nid | bid |
+------+-----+-----+
| 6748 |   2 |   2 |
| 3762 |   4 |   4 |
| 6754 |  38 |  40 |
| 6749 |  40 |  40 |
| 6750 |  34 |  40 |
| 6751 |  35 |  40 |
| 6752 |  36 |  40 |
| 6753 |  37 |  40 |
| 3763 |  49 |  49 |
| 3777 | 101 |  78 |
| 3778 | 102 |  78 |
| 3779 | 103 |  78 |
| 3780 | 104 |  78 |
| 3781 | 105 |  78 |
| 3776 | 100 |  78 |
| 3775 |  99 |  78 |
| 3774 |  98 |  78 |
| 3758 |  89 |  78 |
| 3759 |  88 |  78 |
| 3760 |  87 |  78 |
| 3761 |  90 |  78 |
| 3764 |  91 |  78 |
| 3765 |  92 |  78 |
| 3766 |  93 |  78 |
| 3767 |  94 |  78 |
| 3768 |  95 |  78 |
| 3769 |  96 |  78 |
| 3773 |  97 |  78 |
| 3757 |  78 |  78 |
| 5500 | 199 | 199 |
+------+-----+-----+
30 rows in set (0.00 sec)

Have a look into 'menu_links' table:

mysql> SELECT menu_name,mlid,plid FROM menu_links where module like '%book%' ORDER BY mlid;
+-------------+------+------+
| menu_name   | mlid | plid |
+-------------+------+------+
| book-toc-2  | 6748 |    0 |
| book-toc-40 | 6749 |    0 |
| book-toc-40 | 6750 | 6749 |
| book-toc-40 | 6751 | 6749 |
| book-toc-40 | 6752 | 6749 |
| book-toc-40 | 6753 | 6749 |
| book-toc-40 | 6754 | 6749 |
| book-toc-4  | 6755 |    0 |
| book-toc-49 | 6756 |    0 |
| book-toc-4  | 6914 |    0 |
+-------------+------+------+
10 rows in set (0.00 sec)

If I understand the data model correctly, menu_links.mlid should match book.mlid, right?

Whatever is recorded in the 'book' table, it doesn't show up at all or at least not reliably in the management interface (./admin/content/book/list). Also, once defined books don't show up in the menu system, if the book navigation block is enabled.

Troubleshooting approach: I disabled the 'book' module and uninstalled it; then I re-enabled it and ran update.php. At fist the 'book' module worked like usual, after a few hours the book hierarchy disappeared again. I can not provide a recipe to reproduce this on another site, but at least on the affected site the behaviour seems to be reproducable.

Next steps?

I have no idea where to go from here; thus I have to leave the in-depth analysis to someone with a proper understanding how the 'book' module is supposed to work.

This issue affects a core functionality of Drupal, and it destroys the navigation in websites based on the 'book' module. Thus setting this issue to 'critical'.

Thanks for any suggestions & greetings, -asb

asb’s picture

Hi,

I read Repairing book navigation on a Drupal site by David Herron who experienced obviously the opposite problem (mlid records missing in the 'book' table).

Is there a way to "regenerate" the missing records in the 'menu_links' table?

Additional observations:

  • If I explicitly remove node 2 via ./node/4/outline from any book, the record (see 'books' table above) remains in the database.
  • If I explicitly add node 4 again to the book hierarchy via ./node/4/outline ("create new book"), the database record isn't changed and the new top-level book doesn't show up at ./book. It's exactly the same with other nodes like 49 or 78.
  • If I add nodes 2, 4, 49, and 78 as sub-pages to another book, e.g. bid 160, and then visit ./admin/content/book/160, no sub-pages are listed.

Is there anything I can do to fix this?

Greetings, -asb

markpenny’s picture

Try this from No menu links, no books.

By fixed I mean that I've gotten close enough to the bottom of the problem to understand what I need to do to get the desired results on my site.

The first problem was that the new menu_links-generating version of Book did not generate menu_links for pre-6.x books (books created before installation of Drupal 6). It added the field "mlid" to the book table and assigned all books an mlid, but it added no records for them to menu_links. Since Books 6.x pulls everything together through menu_links, there were no titles in the Book List (/book), even though there were plenty of nodes of type Book.

The second problem, the one that led me to believe the module wasn't generating new books, was that I forgot that books in the Book List have to be created through Book Outline (or Outline, if you want to skip clicking the Edit tab). That is, a node of content type Book is not automatically added to the Book List. The Book List lists the top-level page in a tree of pages and each top-level page must be designated through Book Outline (or Outline).

The quickest solution for me came in two stages.

Stage One 

1. In the database, open the book table and delete all records (necessary to avoid duplicate entry errors for the bid, safe because deletion from the book table does not affect a node's content type).
2. On the site, navigate to Content (Content management > Content or admin/content/node).
3. Filter for Books.
4. Open each node desired as a top-level page. 
5. Designate it as a top-level page through Outline (or Book Outline).

This populated the Book List and gave me books to assign pages to.

Stage Two

1. Open each node desired as a page of a book.
2. Designate it as a page of a given book through Outline (or Book Outline).

All done.

Fortunately for me, I only had about thirty or forty nodes to deal with. Sites with larger book and page populations might prefer a slicker, code-based solution.

Hopefully future versions of Book will either make no changes to the database structure of the module or automatically update all instances to the new structure. I had the impression 6.x was supposed to do that, but it didn't in my case. I was not alone.

asb’s picture

Hi Mark,

I was afraid a suggestion like this would be made ;) Stuff like this is pretty scary since I don't know enough about the inner workings of Drupal, e.g. if other tables need to be updated when you're manually editing the database tables.

However, here we go. At first I got rid of all created books like so:

mysql> delete from book where bid=40;

Then I added one of the nodes that used to be root nodes to a newly created book outline. Ghost entries appeared in the book outline. So I "cleaned" the 'menu_links' table, also:

mysql> delete from menu_links where module="book";

That removes all records with a mlid that might have been crated by the book module. I don't know if other tables, e.g. 'menu_router', would need updating, also.

However, currently the 'book' module behaves as I'm used to it. Let's see if it stays this way...

Thanks & Greetings, -asb

markpenny’s picture

Sounds like your situation was slightly different from mine, but it sounds like things worked out for you, too. Glad to help.

Ole Martin’s picture

I will also try this when I have some time. But what about next time i will do something with my page? How can we create a new nodetype which has a similar effect?
Until this is officially fixed, I will not use the "Book".

optidel’s picture

All we're doing is fixing the symptom. No one seems to have found the underlying cause yet. I do not know enough PHP to dig through the code. As Ole Martin had mentioned, this is a serious enough bug to make Book module unusable.

Who is the maintainer of the Book module?

asb’s picture

@maragnis (#17):

...
// $Id: book.module,v 1.454.2.6 2009/02/25 11:47:37 goba Exp $
// $Id: book.install,v 1.20.2.1 2009/01/06 15:46:36 goba Exp $
// $Id: book.pages.inc,v 1.5.2.1 2008/08/13 23:59:13 drumm Exp $
; $Id: book.info,v 1.5 2007/07/30 18:20:21 dries Exp $
/* $Id: book.css,v 1.6 2007/11/26 16:19:37 dries Exp $ */
// $Id: book.admin.inc,v 1.8.2.3 2008/10/22 19:26:01 goba Exp $
// $Id: book-all-books-block.tpl.php,v 1.1 2007/11/04 14:29:09 goba Exp $
// $Id: book-node-export-html.tpl.php,v 1.1 2007/11/04 14:29:09 goba Exp $
...
asb’s picture

@maragnis (#17):

> All we're doing is fixing the symptom. No one seems to have found the underlying cause yet.

Right, and as I understand, nobody here is able to dig through the code to even search for this cause. All way know so far is that the problem might have something to do with updating Drupal core (Ole's original issue) and/or running update.php (in my case).

Since this issue was originally posted by Ole Martin on May 20, 2009 and there's still neither a patch nor a workaround, the 'book' module could - by sane standards - be labeled as 'unmaintained'. That's the real problem IMHO.

> As Ole Martin had mentioned, this is a serious enough bug to make Book module unusable.

I agree, at least for those sites where the bug surfaces. But we have such critical issues in several places, including e.g. CCK, the caching/performance subsystem, and the menu system. So we just have to find a way to live with those issues, unless we're not willing to shut down all affected sites.

And I might have to add that my book hierarchy again vanished from the administrative interface, so the steps I described in #14 are not even fixing symptoms, just wasting time.

However, in the database all 169 re-added book nodes still exist, but it makes no senses to repeat this b***t once more. And yes, this is definitely a critial problem - in my case it's a showstopper for a time critical project that has to be finished until the end of this year. And so far I have no idea how to deal with this, the whole site's navigation was based on the 'book' module.

Frustrated greetings, -asb

Lionel Triay’s picture

subscribing

Ole Martin’s picture

FileSize
30.75 KB

Now I have done like "#13" wrote. But still I have my old "Book" in my list. See "File attachments" What should I do now?

asb’s picture

Hi,

since I have to consider this core module as 'unmaintained' (more than six months since the initial issue was posted, absolutely no patches available, unclear maintainership, unclear cause for the different problems), I had to disable and uninstall the 'book' module.

I'm trying my luck now with the contributed Node hierarchy module.

Regarding functionality it's at least a partial replacement, regarding usability the 'book' module is clearly superior.

Has anyone else made a similar switch already, with which experiences?

Greetings, -asb

Ole Martin’s picture

Great asb
This is very interesting, I will now set up a test site to test it out. Notice my fine sub domain name -> http://book.drupal.no :D| Coming back with a report afterwards. If anyone is interested in testing this, let me know via the connector form, and I can send the admin username / password

Ole Martin’s picture

Drupal has come with version 6.15 now, but have not seen if it's done something with the module "book" yet. (??)

optidel’s picture

Ole Martin,

No one has even found the cause of the issue. Then some one has to patch it and then we need to test it and finally committed. So we're no where close releasing it.

Ole Martin’s picture

Thats bad.
But I'm trying out the solution to "asb" (# 22) with "Node hierarchy". This seems to be an equally good solution as a "book module". Have to test it out a bit more. I made a separate sub domain for it (http://book.drupal.no/) , so if someone will test it out as admin, they can take contact with me for a username and password.

asb’s picture

@maragnis (#25): As far as I understand the procedure for patches against core, D6 will never be patched directly; it requires a patch against D7 HEAD which has to be accepted and then be backported to D6; also, making patches against D7 ready requires a bunch of regression tests ("simpletests"), which make sense, but load a lot of learning overhead on the developer. I can give examples how this usually works, if someone is interested. It takes a lot of expertise and even more time (rather months/years than days/weeks).

@Ole Martin (#26): Setting up a test site is a good idea - Node Hierarchy (as well as the other node relationship modules like Node relativity) mosty offer an infrastructure and little or no user interface. Node hierarchy has at least a (simple) Site outliner (other modules are even more abstract), but compared to the comfort of a working "book" module this is still very primitive and slow to work with.

Judging solely from a user perspective (!), Node hierarchy doesn't give you anything the book module offers: It has neither pre-built overview pages (like ./book) nor "previous"-/"next"-/"parent"-navigation elements. Also, re-ordering a hierarchical structure of nodes is a pain in the ass (I spent half a day just to re-structure a few hundred nodes which "book" ate).

On the "pros" side, Node hierarchy has the more flexible backend and integrates with Views. Navigation elements like "Previous" and "Next" can be simulated with Custom pagers (that's what I did) or with some code snippets in page.tpl.php (required to get a link to the node's parent). So far I haven't managed to make the Custom pager display the previous/next node's titel (the "book" module does this by default) nor could I translate the "previous"/"next" strings (I think the additions to template.php don't run the new functions through t()). Anyway, with some PHP skills this could be fixed.

However, this is going far off topic; so just in a nutshell: The functionality which the "book" module provides by default requires a lot of handwork when you're replacing it with Node hierarchy. So far I haven't run into serious bugs, but - of course - Node hierarchy isn't free of issues. One has to consider carefully if it's worth to replace one (existing) bug with (potentially) another... ;)

Greetings, -asb

brianV’s picture

@asb, @Ole Martin:

If the cause of this bug can be replicated, I would be happy to push a fix through D7 and backport into D6. Once a patch is ready, this can go as quick as a few days.

However,

1. I can not reproduce this bug with any test sites.
2. Examining the code hasn't turned up anything awry.

There is just not much information available to me to try to isolate exactly what is causing this. For some users it happens during a core upgrade. For others, it happens when upgrading modules within a core version.

Part of me wonders if what we are dealing here is multiple separate bugs with the same symptoms (missing books).

What would really be helpful is if someone with a site affected by this bug could isolate a set of circumstances / steps needed to trigger it.

Ie:

'I upgraded from version X.x to X.x via update.php. I have X, X, X, Y modules installed, which were also updated. Several books created with module version X.x were partially (or fully) unlinked...

etc. etc.'

Until we get that kind of information, any further posts in this bug report about alternate book modules, complaints that this isn't fixed yet, or anything else that doesn't actually contribute to finding a solution to the issue are just noise.

DanGarthwaite’s picture

I would think, in our experience, it should be recreatable via:
1) Install D5
2) Create books and pages
3) Upgrade to D6 > r13

Here is the section of update.php notices where the book_temp table is created to convert pages:

book module
Update #6000

* UPDATE {permission} SET perm = 'access printer-friendly version, access gallery, access content, access all panels, access print, search content, view uploaded files' WHERE rid = 1
* UPDATE {permission} SET perm = 'access printer-friendly version, access gallery, access content, access all panels, vote on polls, access print, search content, access tinymce, view uploaded files, change own username' WHERE rid = 2
* UPDATE {permission} SET perm = 'access printer-friendly version, access gallery, access content, access all panels, access print, search content, view uploaded files' WHERE rid = 13
* UPDATE {permission} SET perm = 'create events, edit own events, create blog entries, edit own blog entries, create book content, create new books, edit any book content, edit own book content, administer book outlines, create images, edit images, edit own images, view original images, create ext_event content, create news_item content, create page content, create story content, edit any ext_event content, edit any news_item content, edit own ext_event content, edit own news_item content, edit own page content, edit own story content, edit any page content, edit any story content, view revisions, create url aliases, cancel own vote, create poll content, access own projects, access projects, maintain projects, access own project issues, create project issues, edit own project issues, assign and be assigned project issues, upload files, access own webform submissions, edit own webform submissions, add content to books' WHERE rid = 16
* CREATE TABLE {book_temp} ( `nid` INT unsigned NOT NULL DEFAULT 0, `parent` INT NOT NULL DEFAULT 0, `weight` TINYINT NOT NULL DEFAULT 0, PRIMARY KEY (nid), INDEX parent (parent) ) /*!40100 DEFAULT CHARACTER SET UTF8 */
* INSERT INTO {book_temp} (nid, parent, weight) SELECT b.nid, b.parent, b.weight FROM {book} b INNER JOIN {node} n on b.vid = n.vid
* DROP TABLE {book}
* CREATE TABLE {book} ( `mlid` INT unsigned NOT NULL DEFAULT 0, `nid` INT unsigned NOT NULL DEFAULT 0, `bid` INT unsigned NOT NULL DEFAULT 0, PRIMARY KEY (mlid), UNIQUE KEY nid (nid), INDEX bid (bid) ) /*!40100 DEFAULT CHARACTER SET UTF8 */
* Relocated orphan book pages.
* Relocated existing book pages.
* DROP TABLE {book_temp}

asb’s picture

@brianV (#28): In my case, this was no issue with core updates, but simply caused by running update.php after updating any other (contributed) module through Drush (system environment: #11). The number of books seemed not to matter; I (re-) created the book hierarchy three times with about 200-250 nodes, sorted into 4-12 top level books and a bunch of book hierarchies below the top level books. The "books" usually stayed there for a few days and then vanished from the ./book overview page and from the node editing forms, every time after having updated another module and having ran ./update.php. Last updates were made to 'amazon' and 'skinr', last installed and enabled were 'similar' and 'relevant_content' (I doubt if there are lots of matches with other affected installations). Installing and updating contributed modules is managed through Drush, except for 'mediawiki_filter' which is managed through SVN.

Two times I disabled and uninstalled the 'book' module, the third time I tried to "repair" the database manually (steps provided in #14). "Book" nodes were only used as containers for other nodetypes and before uninstalling the 'book' module either deleted (once) or converted with "Nodetype". Detailed views of the database tables of "damaged" book structures are provided above; as far as I understand the database structure, the book hierarchy was still there, it just was no longer "displayed". A first try at a code review thus could look after the logic which is responsible to display the ./book overview page and would fail with the described inconsistencies. Also one could write a small helper script that "fixes" possible inconsistencies inside the 'book' database tables; this could be used as a workaround to keep book structures usable. However, this requires a deeper understanding of the Drupal database model than I have so this was not a feasible option for me.

I'm running five D5 sites which make heavy use of hierarchical structuring through the "book" mechanism (with several hundred of nodes outlined in "books"). I never experienced any inconsistencies or failures with the book module on these sites, so the problem is most probably related to changes introduced with D6. Also, according to my experiences from the past years, the core "menu" module had always it's (partially still unsolved) issues, while "book" has always been the more robust alternative to structure content. Taking both experiences together makes the approach from #8 sound plausible - a possible 2nd shot for a code review. This still leaves at least two questions open: a) Why is this happening only on certain sites and b) how can it be reproduced.

Regarding the latter, #29 would be worth a shot; however, it wouldn't apply to my case since the affected site was never updated from D5. Currently installed modules of the affected site:

# ls -X sites/all/modules/
about_this_node  date            google_analytics  memcache       nodetype          similar             translation_helpers        votingapi
admin_menu       dhtml_menu      i18n              mimedetect     nodewords         site_map            translation_overview
advanced_help    diff            image             moduleinfo     panels            skinr               transliteration
amazon           emfield         image_pub         mollom         pathauto          synonym_collapsing  update_advanced
auto_nodetitle   fivestar        jquery_ui         node_clone     recent_changes    systeminfo          views
cck              flag            l10n_client       nodehierarchy  relevant_content  taxonomy_manager    views_bulk_operations
ctools           geshifilter     link              node_import    scanner           taxonomy_vtn        views_slideshow
custom_pagers    globalredirect  mediawiki_filter  nodereferrer   search404         token               views_slideshow_imageflow

The default theme is a modified acquia_prosper. This is a multilingual site, running the core modules "Locale" and "Content translation" plus the contributed "i18n". 'custom_pagers' and 'nodehierarchy' were installed the first time after the books vanished the 3rd time (see above), so they're definitely not suspicious of causing the issue. Other noteworthy behaviour: Even after disabling and uninstalling the 'book' module I still have a ./node/add/book page which gives me an node editing form.

The suspected 'dhtml_menu' seems not to have anything to do with this issue, also: It was at least one time disabled when the books "vanished". Anyway, I don't know how deep dhtml_menu nests itself into the menuing system, so some strange incompatibilities are not impossible. Also I'm not running dhtml_menu on my "stable" D5 sites. That's anything I can tell from my end so far.

Greetings, -asb

gilf’s picture

I don't know if this is any help, but this is what I've encountered:

Books disappeared (all books were from a pre-6 installation, but worked fine, until they disappeared one day.)
Followed instructions on http://davidherron.com/content/repairing-book-navigation-drupal-site, and books reapeared
Added a new book
Two days later all books disappeared again

Grum’s picture

I started with D6 and am having trouble with a multilingual site. Until I came across this thread I was completely baffled as to what was going wrong, but the descriptions above do seem to fit. This is happening with only a dozen or so book pages.

I have no problems with my main site which is single language.

Trying to work out what the batch process does I looked at the Schema and came across the following entry under ./admin/build/schema/describe for batch (system module)

timestamp int NO
A Unix timestamp indicating when this batch was submitted for processing. Stale batches are purged at cron time.

This has me wondering what a "stale batch" is and what happens if the cron job coincides with a run of update.php. I also note that translations are imported using batch processing, so the occurrence of batches is likely to be higher on multilingual sites.

Anonymous’s picture

Subscribing...

danimatielo’s picture

I had the same problem and was upgrading from 6.14 to 6.15. I did not try any solution yet because as far as I read this thread, it seems that after a feel days all the structure is lost again. Could someone find a solution for this? Is it not possible to import the structure from my old side? Why doesn't it work?

Thank you,

Dani

asb’s picture

@Grum (#32):

in my case, the affected site uses a localized interface (German), and is set up for multilanguage content (German and English). Maybe this is the common denominator for at least one of the problem scenarios described here (site started with D6, not upgraded from D5)?

Feedback, please ;)

Greetings, -asb

Ole Martin’s picture

To information.
Book module also has an impact on the DHTML Menu. See http://drupal.org/node/483920#comment-2450162
It's best if we know most about the issues that are related to the Book module.

asb’s picture

Interesting observation. Yes, I had DHTML menu also running on the affected site. However, when the issues with my 'book' pages started, I disabled it for all menus but 'navigation'.

OK, now we've got:

  • the issue is somehow connected to running update.php, be it for a major update of Drupal core, or for contributed module's database updates;
  • the issue might have something to do with running a multilanguage site; question: Who has the core module 'Content translation' enabled? (out of my active eight D6 sites, one is running 'Content translation', and that's the one experiencing the 'Book' issue);
  • the issue might have something to do with running the contributed module 'DHTML Menu'; question: Who has the contributed module 'DHTML menu' enabled? (I have this running basically on every D6 site)

Your turn...

gilf’s picture

Not running DHTML menus, but is multilingual with content translate.

boabjohn’s picture

@asb Thanks for running a structured and intelligent approach to this problem. I just encountered it today for the first time on a rececntly upgraded (D6.14>D6.15) site.
- I use drush for module updates
- Content translation is not turned on
- DHTML menus are enabled

My book table appears to be intact, but none of the mlid values in the book table appear in the menu_links table.

In addition, the site appears to have randomly dropped a few other menu listings. Initially I thought these were user errors (and chastised the client accordingly), but combined with the book problem it looks much more like something is amiss with menu_links.

I have not seen any common thread between the other missing menus...in each case they were simple page nodes with a menu value set. At some point they simply disappeared out of the menus and we only caught the problem today (about 1 week after upgrade).

Please let me know if further info will help.

+1 to the sentiments that this is a *very* unfortunate behaviour for our dear droplet to exhibit. Clients (ie ordinary humans) are NOT impressed with this sort of instability.

Magnity’s picture

I'm seeing this too.

Updated via drush, and all book structures have disappeared. Also, I don't seem to be able to recreate them via the outline tab, as it never saves the created books (no error msgs, just doesn't do anything).

Looking into the logs, whenever I try to recreate the books, the following PHP message has occured:
Duplicate entry '8' for key 2 query: INSERT INTO d6_book (nid, mlid, bid) VALUES (8, 1345, 8) in /[path]/public_html/modules/book/book.module on line 466.

boabjohn’s picture

Looks to me like we've been sidelined in this thread as everyone is sucked into D7-land. Perhaps one of the genii will show up again soon and sprinkle some clarity dust on our woes.

Magnity’s picture

D7 or not D7, this is a pretty critical problem when it happens. If it needs to be fixed first in D7, it should be.

I have now experienced it on two sites.

Both were updated via Drush.

Neither used content translation or DHTML menus.

When I have a minute, I will try to find a way to reliably reproduce this problem based on the times that I've seen it.

asb’s picture

@Magnity (#42):

I'm also updating my contributed modules through drush on all sites I'm currently operating; so this might be just another number in the calculation.

However, of my currently 12 sites on D6, I'm experiencing the book issue only on one site so far. Five of these sites were updated recently from D5, and none of these sites show the 'book' module degradation yet (doing core upgrades was the initial suspicion for what is causing this problem). The "disappearing books" issue still seems to be highly erratic.

@all:

I've done some more reading about how maintainers handle issues with Drupal core; if I understand it correctly, a critical issue would be considered a showstopper for releasing the upcoming Drupal 7; regarding issues with stable releases of Drupal core (e.g. 6.x), the maintainers usually do not try to fix or even analyze the issues themselves; they're supposed to monitor patches as they are coming in, and they review those patches in terms of coding style and side effects for other core subsystems; providing analysis of the problem, writing and testing patches is mostly considered a community task.

Examples for critical issues with Drupal core (D6): #444228: Optimize CSS option causes php cgi to segfault in pcre function "match", initial description with problem analysis posted on April 24, 2009; precise instructions how to reproduce the issue provided on April 25, 2009; patch against current D6 release on May 15, 2009; status set to "won't fix" by maintainer (?) on September 28, 2009; issue re-opened on the same day; several new patches provided, including simpletests, between September and December 2009; current status: "needs work"; so far a total of 94 replies. Another example: #284392: db_rewrite_sql causing issues with DISTINCT; initially posted: July 18, 2008; so far 437 replies; current status: RTBC (meaning: still not in core after 18 months, and after major efforts by the community).

Having this in mind, I think, there can not be much (read: probably zilch) help until someone comes up with clear instructions how this issue can be reproduced at least.

So far, we have a couple of contributed and core modules under suspicion to be somehow involved (Drush, DHTML Menu, Content Translation), but at least one report to experience the 'book' issue without running any of those; all of us seem to have run update.php, so this is still the only common denominator we'Ve come up so far. I'm afraid, in terms of analyzing the issue we're not making much progress since the initial post, dating back from May 20, 2009. So far, we have neither instructions of how to reproduce the issue, nor even an attempt in patching Drupal core.

Greetings, -asb

samhassell’s picture

I have experienced the same bug twice now. Quite annoying and scares me about putting this site into production.

I'm running the following contrib modules:
admin cck date gmap nice_menus taxonomy_treemenu
admin_menu comment_bonus_api devel imageapi panels user_import
advanced_help conditional_fields email imagecache panels_tabs views
ajax_comments content_multigroup features imagefield phone views_calc
auto_nodetitle content_profile filefield link print views_customfield
backup_migrate ctools geo location tabs views_groupby

I'm also updating with drush.

samhassell’s picture

I have a recipe for this.

1. Download drupal 6 via drush (drush dl drupal)
2. Run through the install.
3. 'drush enable book'.
3. Create a page and add it to a new book.
4. Add some child pages.
5. Go to admin/content/book where the book will be listed.
6. Run 'drush update'
7. The book will no longer be listed.
8. Try and add the parent node to a menu.
9. user warning: Duplicate entry '1' for key 'nid' query: INSERT INTO book (nid, mlid, bid) VALUES (1, 210, 1) in /home/www/d6book/modules/book/book.module on line 466.

Also, if i perform the update at update.php, the issue does NOT occur.

Looks like the problem is with Drush, so we have a good chance of getting it solved.

samhassell’s picture

Status: Active » Closed (duplicate)

There is a post on the drush issue queue. Marking this as a duplicate as the other already has maintainers attention.

http://drupal.org/node/667298

Magnity’s picture

Title: Book disappears during upgrade leaving orphaned pages » Book disappears during Drush upgrade leaving orphaned pages
Project: Drupal core » Drush
Version: 6.x-dev » All-Versions-2.1
Component: book.module » Code

Moving queue + making title more specific.

samhassell’s picture

http://drupal.org/node/198980 is the issue that will solve the book-update problem.

iss’s picture

subscribing

Lakeside’s picture

subscribing

quentinsf’s picture

I recovered the lost structure using an old SQL backup (created using the excellent Backup and Migrate module).

I grep-ed the SQL for 'book-toc' and then edited it down to all the lines, or partial lines, which did insertions of the form:

INSERT INTO `menu_links` VALUES ('book-toc-73'…

Copied and pasted the appropriate bits into phpmyadmin and, lo and behold, the books were recreated.

I make no guarantees that this will work for you too… but it may be worth a try if you understand the basics of SQL.

Lionel Triay’s picture

subscribing

asb’s picture

Title: Book disappears during Drush upgrade leaving orphaned pages » Book structure disappears repeatedly
Project: Drush » Drupal core
Version: All-Versions-2.1 » 6.16
Component: Code » book.module
Status: Closed (duplicate) » Active

This is just a long shot:

Has someone of you, by any chance, renamed the machine-readable type 'book' at ./admin/content/node-type/book (and not only the name of the content type)?

The Drupal user interface allows such an operation and doesn't even warn about possible side-effects; I'm used to rename content types like 'article' and 'page' for proper localization of non-English sites since Drupal 4.x and have never experienced problems.

However, I noticed that all of my sites that are affected by this problem had such renamed machine-readable content type names, while other (unaffected) sites were not 'localized' in this manner.

I'm also modifying this issue's queue once again since Drush imho was just one factor that triggered the disappearing books. The Drush related issue is claimed to be fixed, but I still get book hierarchies disappearing after disabling and uninstalling the 'book' module, then re-enabling it and manually re-creating the book hierarchies. Basically there seems to be no way to continue using the 'book' module after a site once has been affected, so if if can't be caused by Drush anymore, it has to come back to Drupal core's issue queue. At least completely uninstalling and then re-enabling a core module should reset this module to a properly working state, everything else has to be considered as unexpected behaviour.

Greetings, -asb

joel_guesclin’s picture

I have just completed an upgrade of a small site with two books - the book structure has been destroyed. I was not using drush but the standard upgrade system.
I was upgrading from D5.22 to D6.16 - I had disabled all non-core modules before the upgrade, and the only non-core I had are Excerpt, XMLSiteMap, and Nice Menus.
I had previously upgraded a large site with a lot of books without any problem, going from D5.22 (or possible 5.20) to D6.15. On the latest upgrade I notice an error with non existent tables which I don't remember ever seeing before.
The other issue is that I cannot create a new book from an existing book page. I was going crazy (trying to repair this by hand), editing the top level book page, selecting "Create new book" in the outline, and then finding no new book. The reason for this is a duplicate key error, as so:

Duplicate entry '12' for key 'nid' query: INSERT INTO book (nid, mlid, bid) VALUES (12, 447, 12) in /Users/me/Documents/WebSites/bn/modules/book/book.module  on line 466.

This is seriously bad news. I can manually repair this site because it is quite small, but that solution is out of the question for my larger sites; it is also out of the question to try to repair this problem by hand fiddling in the database - too many entries and too many possibilities of making mistakes.

boabjohn’s picture

@asb: interesting angle.

My known problem site had no re-naming BUT does have a missing default content type, ie, the Story type had been deleted for simplicity.

Just adding a possible data point to the picture...I will check other sites and have a think.

joel_guesclin’s picture

This is bizarre to me because I already successfully upgraded a site which is in French and uses books extensively to organise the content. I have not renamed the book type. I wondered whether the problem comes from upgrading to 6.16 as opposed to 6.15 - I did not get this problem when upgrading to 6.15 from 5.22.
The site where I experienced the problem is in Bengali - since no translation existed at the time, all the interface is in fact in English. The book type has not been renamed.
However, I have found a way out of the problem which does not involve doing everything by hand. As far as I can see this works, though I have a small doubt about the "book structure" block, which seems not to appear at all when you set it to "only appear on book pages", although it seems to work fine when set to appear on every page.
I cannot say whether the book structure is now "stable" - I will keep an eye on the site in the coming weeks.
What I find most disconcerting is that this critical bug in what you might call a "core core" module (used in wikis and interest groups for example), not only has not been resolved for almost a year, but does not seem to have attracted any attention from the core developers who I presume make their living out of ensuring that Drupal is rock solid.

joel_guesclin’s picture

I just tried another upgrade, this time from D5.22 to D6.15. I did not get the error described and linked to in #54, and my book structure worked ok afterwards. Apparently you must go at least through 6.15 before upgrading to 6.16.

Grum’s picture

@ joel_guesclin (#56)

Did you do the initial installation of your French site in English or French?

If I do a Drupal installation in German (by loading the language files before running install.php), then the book content type is initialized with the German for book (Buch); it is not necessary to rename the content type. Did you modify the book content type for your Bangladeshi site for localization purposes?

What are others doing here for non-English sites? Are you running install.php in English or the target language?

joel_guesclin’s picture

Errrr.... it was all so long ago! As I recall, the language installation process was very different in D5 (unless I'm mixing it up with D4, but the French site was first installed in D4). You installed the site always in English, then you loaded the translation files (which were not distributed in the modules as they are in D6), then you set the default language to French, and in my case I always altered the site admin account to English. This meant that the site admin always saw the site in English and everybody else in French. As far as I am aware this has not modified the name of the content type.
The Bengali site was even easier, because at the time there was no Bengali translation, so I just created the language but there were no translation files to import. The Bengali site was also created in D5. At all events, I hope that the content type machine identifier is not translated because otherwise I think you could be in real trouble....

Grum’s picture

You are correct. The machine readable content type for book is not translated.

asb’s picture

Related issues:

#645214: No menu links, no books ("Unassigned" and "active" issue against book.module 6.x-dev)

Lost Books (Forum posting from November 20, 2009)

fehin’s picture

subscribing

px’s picture

I tried to track it down but found the bug to be rather elusive. I'll try to give some circumstances anyway:

* The bug hit me on two sites. Affected Drupal versions are/were 6.14 - 6.16. These where always 6.x installs, no upgrade from 5.x was ever done.
* Modules installed (but not nescessarily enabled):
advanced_forum
advanced_help
advanced_profile
apply_for_role
author_pane
auto_nodetitle
bbcode
better_formats
captcha
casetracker
cck
cmf
comment_subject
computed_field
content_profile
ctools
cumulus
decisions
drush
ed_readmore
email
event
faq
filefield
filefield_privacy
googtube
image
imageapi
imagecache
imagefield
imce
imce_wysiwyg
link
markdown
mimedetect
og
og_book_inheritance
og_forum
og_mandatory_group
og_vocab
panels
phone
potx
print
privatemsg
simplenews
smileys
spamspan
tagadelic
taxonomy_delegate
token
transliteration
vertical_tabs
views
votingapi
webform
wysiwyg

###And on the other site:

adsense
auto_nodetitle
better_formats
blockquote
blockreference
boost
browscap
captcha
cck
clickpath
codefilter
computed_field
ctools
cumulus
devel
drush
drush-All-Versions-2.1.tar.gz
faq
filefield
filefield_paths
fixdaycountreset
globalredirect
imageapi
imagecache
imagefield
jquerymenu
link
linkchecker
markdown
og
og_book_inheritance
panels
pathauto
path_redirect
similar
statistics_advanced
tagadelic
taxonomy_menu
taxonomy_vtn
token
views

* The bug likes to show up after doing some heavy duty enabling/disabling of modules.
* I grepped the mysql bin logs for the string "DELETE FROM menu_links", which strangely did not show any results.

Description of symptons and fix (well, not actually fix, just repairing till the bug strikes again):
http://www.onyxbits.de/content/fixing-drupals-broken-book-navigation-aft...

Edit:
Some updates: I had hook_uninstall() of the book module commented out for safety (actually, not only uncommented but replaced by something that would email me a stacktrace). Yet, I just lost all my books again (this is getting a bit frustrating). Also, my taxonomy_menu module got messed up. The only administrative thing, I did today was using drush to check for updates (none available).

Edit^2:
Ok, I seem to be able to reproduce the bug now. I loose my books after starting "drush update" on the condition that there are no pending updates and I am accessing /book while the command is running. I'm not quite certain about the drush version, I have installed, but I have a tarball lying around, named "drush-All-Versions-2.1.tar.gz". After updating to drush 3.0, the issues seems to go away.

asb’s picture

Yes, older versions of Drush had this issue and destroyed the book hierarchy. This is fixed in current versions, but there's no fix to repair the destroyed book menu structures.

lilkim529’s picture

Title: Book structure disappears repeatedly » Book Child Pages Stopped Showing Up! Please Help!
Version: 6.16 » 5.9

PLEASE HELP! About a year later, my child pages have stopped showing up. They show up in the back end, but not front end. They no longer show up in the side navigation. If you search for the content, they do not show up either. However they do exist back end. How do I fix this??

Thanks ahead of time! ; )

Ole Martin’s picture

I started this thread for over one year ago and have followed it. It has repeatedly changed its name and version.

My advice is: Do not use it, get all the book pages to the Page instance, and use views for display and a menu to navigate, or wait and see what happens in version 7.

It may still be wise to upgrade from version 5.x as it will soon no longer be updated.
I will never use this again, one year without fixing it.....

asb’s picture

Title: Book Child Pages Stopped Showing Up! Please Help! » Book structure disappears repeatedly
Version: 5.9 » 6.x-dev

@lilkim529:

Please don't change the subject and version of this issue.

If you believe to encounter a distinct bug, please open an new issue with a proper description and details how to reproduce it. If you do not provide these information, no maintainer will be able to care about your problems.

fehin’s picture

See post #16 in this thread http://drupal.org/node/645214. It fixed the issue for me.

asb’s picture

@fehin:

Thomas makes one very important assumption in his recipe: 1.) drop table book in D6 (if there are no diffs to your backup of D5). Months after migrating our sites, whatever might have been in the D5 table is worth a sh** for most of us today.

Also it might be worth remembering, that some of us even tried to completely disable and uninstall book.module (resulting in loss of all structural data), and then re-enabling the module and re-creating the book structures, and then lost this data again; even this rather extreme approach failed in some cases - one of these cases I experienced myself.

This issue is still extremely erratic, and almost every occurence seems to behave somehow different than others; the reason for this incredible complexity might be that different issues were interwoven (e.g. the damage 'Drush' caused for a while, and bugs in the database upgrade from D5 to D6, plus side effects of the marriage of book.module and menu.module). In my two cases, I belived for a couple of weeks to have solved the issue, and in both cases my attempts to repair the book structures failed after a while. In one case I could save the book mechanism by uninstalling book.module (like explained above), in the 2nd case these structures vanished after some time. Even if fixing works for a while, it doesn't necessary need to last permanently. So be warned and make backups freqently ;)

Greetings, -asb

lilkim529’s picture

Title: Book structure disappears repeatedly » Book Child Pages Stopped Showing Up! Please Help!
Version: 6.x-dev » 5.9

@Fehin & @ASB -

Thank you. I'm relatively a beginner here. Will I lose any content if I uninstall and reinstall??

For everyone, let's say I just recreate a brand new site in Drupal 7. If I have over a years worth of content and subscribers, can I easily move them over???

brianV’s picture

Title: Book Child Pages Stopped Showing Up! Please Help! » Book structure disappears repeatedly
Version: 5.9 » 6.x-dev

Please do not change the title or the assigned version.

VortexCentrum’s picture

Title: Book structure disappears repeatedly » Book structure disappears repeatedly

I did a clean install of 6.17. The site is used for one or two "Pages" and the remainder is books.

Using the Books module, the content does not publish, no matter how many times I tick the box. I have to go into the database and change the status flag.

Every time I modify a book page, it unpublishes it, even if the "publish" box is ticked.

If a page is unpublished, it is impossible to create a child (the link is not present). This means that a book cannot be created prior to publication.

I have tried several different themes and the same problem occurs with each, suggesting that the problem is with the Book module.

I'm at a loss and currently considering aborting using books and building with pages - but it's a poor solution and I'd rather not do it.

Before I do that, has anyone any ideas other than those this thread which, regrettably, don't work for me?

Thanks

mbria’s picture

subscribing

Anonymous’s picture

http://drupal.org/node/645214#comment-2454738 fixed it for me. All books are back.

avi.wollman’s picture

it would be nice to see a simple backup\export and retrieve\import of book outlines

bjorpe’s picture

Just an idea - judging from that "upgrading again" seems to fix the problem (at least temporarily).

Couldn't it be that some other upgrade related task (like menu_rebuild) is deleting the items in the menu-links table?

bjorpe’s picture

So, looking into my logs, I find lots of this (same as reported here).

You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ') ORDER BY fit DESC LIMIT 0, 1' at line 1 query: SELECT path FROM menu_router WHERE path IN () ORDER BY fit DESC LIMIT 0, 1 in C:\xampp\htdocs\drupal\includes\menu.inc on line 2099.

The line is in function _menu_find_router_path which happens to be called when menu_rebuild is executed. From what I can see, this error will cause the rebuild function to delete the menu items. (The empty parenthesis in the SQL indicates that the $placeholders variable is empty).

I hope this can help someone to find the root cause, I can't really see when menu_rebuild gets called in the update process but it seems like it gets called.

bjorpe’s picture

To clarify, I did an test update from 5.22 to 6.17 and the book structure was disappearing (>200pages). Was able to re-create it from the D5 book table as described in other comments.

Digging a bit deeper in my error log, I find 3 identical errors before all the SQL syntax errors:

Location	http://localhost/drupal/update.php?id=1&op=do
Referrer	http://localhost/drupal/update.php?op=start&id=1
Message	Table 'ballong_d.semaphore' doesn't exist query: SELECT expire, value FROM semaphore WHERE name = 'menu_rebuild' in C:\xampp\htdocs\drupal\includes\lock.inc on line 149.

It looks like menu_rebuild, called somewhere by update.php, tries to use the semaphore table before it is created.

bjorpe’s picture

Did a bit of reading and the missing semaphore table is well-known. #732024: Updating from 6.1 user warning: Table 'semaphore' doesn't exist

However, if menu_rebuild incorrectly wipes out the book menu entries the first time running update.php in D6, there is no way of getting them back... I will try and repeat the upgrade when I have the time to see if the semaphore table is really the problem.

bjorpe’s picture

I've repeated the D5->D6 upgrade process a few times and the semaphore table seems to make a difference.

By creating the semaphore table manually before running update.php, the books are preserved. In MySQL:

CREATE TABLE semaphore ( `name` VARCHAR(255) NOT NULL DEFAULT '', `value` VARCHAR(255) NOT NULL DEFAULT '', `expire` DOUBLE NOT NULL, PRIMARY KEY (name), INDEX expire (expire) )

This is a long-lived issue but with the introduction of the semaphore table in 6.17, is it now possibly a D5->D6 upgrade issue only? Anyway, I will go for this work-around solution for now.

boabjohn’s picture

@bjorpe: Interesting and excellent sleuthing. Thanks for sharing your investigations. I wonder if anyone has been having trouble on 6.19? I'm not brave enough to try one of my production sites that had this problem back in 6.14....

oadaeh’s picture

Those of you who are still having problems with upgrading from D5 to D6 should check out #767408: semaphore table need to be created during update_fix_d6_requirements(). Applying that patch to my D6 installation before running the upgrade.php script fixed my book outline disappearing problem.

joel_guesclin’s picture

Don't know if others have had this experience, but I have found that upgrading from 5.23 to 6.15, then from 6.16 to 6.19 works for me - at least so far...

DanGarthwaite’s picture

If the solution to the upgrade is found, then what is now needed is some googleable code (hint: here) to convert the tables after that fact. Meaning post d5 -> d6 upgrade.

My client's site just wrote off the books that where lost many 6.x versions ago. I'd like to port them back.

oadaeh’s picture

@dan@garthwaite.org: the books weren't lost, just the structure. All the data should still be there. What needs to happen is to link the nodes into the new menu system, and that's not a very pretty thing to do by hand.

EvanDonovan’s picture

Issue tags: +book structure, +data loss

I'm trying to get a handle on this issue but it looks like different people are experiencing different problems. This makes it hard to diagnose and solve. So far, I've seen in this issue at least the following:

* Drush update error that causes people to lose book data (now fixed)
* Problems when the machine name of the book content type is changed
* Issues with updates since the Book module was changed in D6 to use the menu module for hierarchy
-- this last may be only when upgrading from D5->D6 & sounds like an upgrade from 5.23->6.15, then 6.15->6.19 may fix it

As for me, I only ever lost book structure when I was trying to submit the hierarchy page while someone else was editing a book node, but that is a different issue, one which I believe has been fixed already.

EvanDonovan’s picture

Issue tags: +menu, +menu_rebuild

Adding tags.

yngens’s picture

Experiencing the same problem on a website upgraded from D5 to D6. Subscribing.

joel_guesclin’s picture

In reply to #86 I can confirm that I have experienced the "disappearing book hierarchy" problem, and that:
- I don't use Drush
- I have not changed the machine name of the book content type
- I have upgraded several sites from D5.22 to D6.19, going via D6.15, without experiencing problems.
Hope this helps.

swing4cat’s picture

Experiencing the same problem in 6.20. Subscribing.

hefox’s picture

For d6 only issues, if menu_rebuild is done with incomplete data, it will remove the book menu links.

In our case, something was calling menu_rebuild when module_list had incomplete data (due to bad interaction with a custom thing).

1661Design’s picture

Have a book with few pages in a new site with D7 and lost the entry point (content is there) no drush involved, did not realize it happen after few days so don't know what may caused the problem :( …

Subscribing,

Status: Active » Closed (outdated)

Automatically closed because Drupal 6 is no longer supported. If the issue verifiably applies to later versions, please reopen with details and update the version.