Mapper for Book module (D7)

Not possible atm.

typo.

What makes it impossible?

Grayside: Somebody needs to sit down and write a mapper for the book module :-) http://drupal.org/node/622700#mappingapi

I've begun looking into this, but it's also my first experimentation with Feeds. I'm making an assumption that a feed will be processed from top to bottom through a hierarchy, meaning parents will always exist before children.

I've run into a problem. How do you identify the parent page of the currently processing node from the import/mapping side of the process?

1. If it were populated, the guid field of feeds_node_item could be used to cross reference the parent identifier on the source site with it's new nid on the import site.
2. You could pull the title of the previously created parent. This would impose a requirement of unique titles. Unique titles is a horrible imposition to place on book content, especially when books are worked on by different groups of people.
   There are a couple methods of reducing the need for unique titles to a single book's namespace:
   • Use the current feed source or feed node as a common factor with any node titles you might query as a parent node. This would allow the site builder to do something like pull in one book at a time, requiring unique titles only within that book. How can the mapping callback get ahold of the current feed source?
   • Less effective, but use the book title as a unique namespace. If the incoming nodes carry the titles of their parent, as well as the title of their book, the one-book-per-feed-source method of reducing the unique titles to a single book is eliminated. (Assuming you have unique book titles.)

This all is completely separate from determining how to piece together an RSS feed that has a section for a book membership reference. I suppose that's modifying XML output on one end, and building a custom feeds parser for the import side. (EDIT: Oh, or just dealing with CSV.)

I think there should be a distinct mapping target 'parent page' that the book mapper exposes, this mapping target will accept any source field and will try to use it to determine a book's parent.

Here is the major potential pitfall: In order to build a hierarchy, you will have to have all imported items available. You can guarantee that by implementing hook_feeds_after_import(). Look at Geotaxonomy module for an example:

http://drupalcode.org/viewvc/drupal/contributions/modules/geotaxonomy/ge...

I figured on 'parent page'. I'm just uncertain how to take data from the feed and correlated it to an identifier the local drupal install understands. (Except something kludgy like the title lookup.)

Thank you very much for pointing out that hook, that's one of those things I couldn't piece together from the docs.

I would have book mapper expose two target fields:

1.

Title: "Book page id"
Description": "The book page's unique id, used to identify parent pages."

2.

TItle "Parent book page id"
Description: "The book page's parent id."

You then use "Book page id" as a basis to build a hierarchy with the information of "Parent book page id". You will have to track which node ids have been generated for which book page ids in the first pass, then on hook_feeds_after_import() you can use this information to look up the parent node ids for a given parent book page id.

Thus your book id / parent book id can be anything, from a GUID to a title to an integer etc. It just needs to uniquely identify a single book page.

In the hook_feeds_after_import() example, I am not following where the tids are stored.

The Book ID/Parent ID cannot be added to the node until that hook is called.

Feeds Book Mapper module attached. Imports hierarchies into books flattened to two levels.

While I will respond to bugs in the current functionality, I have no further availability at this time for writing tests or rounding out the other to-dos. I wrote this primarily as a time-saver for a Drupal → OpenAtrium migration.

See the bundled README for documentation.

Can we get a patch? (please!) Check out http://drupal.org/patch and cvsdo add

Thank you!

I can certainly provide a patch, but it was unclear to me where to place this code in the feeds structure. As currently structured, it has it's own database table. I felt feeds should only install a database table like that if it were set up to more generically support other modules collecting relationship data for hook_feeds_after_import().

Should I patch the schema into feeds.install? I can drop the rest of the code as an include file in the mappers/ directory if the table is otherwise handled.

Should I patch the schema into feeds.install? I can drop the rest of the code as an include file in the mappers/ directory if the table is otherwise handled.

Yes.

Further thoughts about this code:

1. The database table is really only needed to track orphaned hierarchies. It could delete records that are successfully added to an outline.
2. The Book ID Mapping (and Database Column) is really a convenience. Level 2 pages can readily determine that their parent's are top-level, and the process of building out a complete hierarchy will propagate the book information down to child pages.
3. I'm using feed_nid and guid as a primary key for the feeds_book_mapper table, that was my poor method of using the feed source to namespace the guid. That seems like a bad solution, and I'm sure it could corrupt hierarchy structures not pulled in through a feed node.

Patch attached. Had some trouble with cvsdo so I jury-rigged the patch some. It works, but it dumps the book.inc file into the main directory instead of mappers/.

README:

Feeds Book Mapper

Mapping a hierarchy to the book system assumes the entire hierarchy is
coming in from the same feed source. It is currently written with the
assumption of the same feed node, and a hierarchy with no cycles.

To use the Book Mapper effectively, the GUID's for the current feed source
must be unique so the relationships between pages can be correctly built.

The Feeds Book Mapper provides two new mappers--Book ID and Book Parent ID.

Book ID
The Book ID is the GUID of the top-level page in the current page's
hierarchy. A page is identified as being its own top-level page when it's
GUID is equvalent to the Book ID.

Book Parent ID
The Book Parent ID is the GUID of the current page's immediate parent in the
hierarchy. This mapping is currently ignored.

Building Your Feed

To be on the safe side, please sort your feed by depth in the hierarchy.
This will help reinforce that parent pages are created before book pages,
which might help avoid bugs.

Creating the Book Hierarchy

The top-level page in every book is created. Below that each direct child is attached to the outline, then each of their children, in preorder traversal.

Updating the Book Hierarchy

This is entirely untested, but it should theoretically work.

* Child pages will be promoted to full books.
* Child pages placed in different books will be moved.
* Top-level pages assigned to another book will be demoted to child pages.

Removing Pages from the Hierarchy

You cannot remove a page from all hierarchies. Once it's in the system, you
are done. This is how book's are handled in D6 (at least in the UI).

Fixing the Page Hierarchy

If the Feeds Book Mapper corrupts the hierarchy, visit /admin/content/book/<bid> for some manual book administration.

Alternatively, you could make use of the DraggableViews project's Book Handler,
which would allow you to more easily share that task with non-admin users.

http://drupal.org/project/draggableviews

To-dos

* Implement the full parent hierarchy.
* Implement page weight for proper sibling order
* Use something other than feed_nid to key on source feed.
* Clean up database entries for successfully outlined items. (This would actually break the ability of future children to be added to the outline. What was I thinking!)

Okay, I couldn't let it ago. Attached is an updated patch that, with greatly reduced testing, appears to build out the entire book hierarchy. It's also been tightened up a little in other miscellaneous ways.

I will now edit the readme above.

I've not written tests, but here's some that should be covered:

• Multiple book creation, multiple books in one import.
• Hierarchy iteration works: A hierarchy at least three levels deep properly imports.
• Importing a hierarchy more than 9 levels deep.
• Multiple-import hierarchies: Attaching pages to a previously imported book/parent.
• Updating the book/parent relationship: Import an update that converts a top-level book to a child page of another book.
• GUID collision: Import book pages from two unrelated feeds that coincidentally use the same guids.

Warning: Something about the book mapper breaks something in the book hierarchy. Do not use in production.

When moving a top level book as a child of another book, the book you attempt to move behaves unpredictably. Mostly it vanishes. If you attempt to restore the top level book to it's original position, it may or may not restore the book hierarchy.

I have a theory it relates to the same stuff that causes problems like #507082: Book parent problem - cann't see parent in drop down menu!!??, and which I've seen (but cannot again find) a patch in the D7 queue.

This also raises the point that changing the book hierarchy requires updating the feeds_book_mapper database table, so future nodes can find their properly assigned books. The mapper needs the following:

1. The ability to update the book_id to something like "nid:###" so books can be reformulated at will on the site, and new content find the proper position. This would be on hook_nodeapi() update.
2. The book mapper understanding nid:### and acting accordingly. This includes changing the book_id of incoming content if the book_id they specify has a book_id of an "nid:" type.

When saving any imported book page after it's original book-assignment on import, the book page (and any children) have their "menu_name" saved not as book-toc-<nid>, but as primary-links. The other elements of the book hierarchy (bid, plid, p1, etc) are not rebuilt.

This results in orphaning all the pages from their books, despite having retained all the actual relationship information.

EDIT: Problem Solved, New Patch Forthcoming

The problem with the hierarchies related to books being imported with $node->book['module']='menu'. As a result, any save of a book page broke the book-toc-menu as described above.

By explicitly setting the module to 'book' when the hierarchy is built, it seems to work.

The issue I raised in the second half of #19 still maintains. Is that a blocker? Setting to needs review for feedback.

I'd like to test this patch and give feedback.

Could you provide a sample CSV file to be imported in a book, so that I can start testing faster?

Here's an untested CSV file I whipped up when I started thinking about writing some tests.

There is something wrong in the patched feeds.install that prevents the patch from correctly creating the feeds_book_mapper table:

• the new table is not defined in the schema definition, but only in an update function
• the new table schema is added to update 6009 while it should be in an update by itself (6010), otherwise it cannot be executed by running update.php.
• in the update function, db_create_table is never called for the feeds_book_mapper table.

Looks like something unexpected happened. I will look into a reroll this weekend. Thank you for the feedback!

Just read up on the patch, my questions:

1 Shouldn't _feeds_book_assign_children() be _feeds_book_assign_parents() - at least this is what the function looks like it does.
2 feeds_book_mapper table needs to be created in separate update hook.
3 There should be no need for shuttling the feed_nid with the help of a static variable in _feeds_book_get_feed_nid(), look at $source->feed_nid in book_feeds_after_import().
4 Why do you not remove items from feeds_book_mapper once you've imported a book?
5 What do the two calls to _feeds_book_record_exists() that don't record a return value in book_feeds_set_target() do?

Hello

I'm just new in feed and I try understand how it suppose be to can feed outlines of book. Ca someone write me how need look Mapping for Node processor for this? What source to what Target? And what mean that GUID must be unique? For what?

Sorry but I'm little lost in it...I'm trying correctly map book but it is always without outlines and hierarchy.

Thanks

P.S

I'm using first path cause second generate some errors

You sound really confused, are you asking how to set up your CSV file?

I am getting back to this patch next week.

No sorry I just try feed book from other website into new one and I cannot feed outlines correctly (hierarchy). Anyway I will wait on new path cause I think is something wrong with this.

Thanks

The patch in #21 doesn't work for the latest version of Feeds. I fixed this and also changed a few other things:

1. Moved _feeds_nodeapi_book_mapper_cleanup to feeds.module to fix compatibility with latest version (apparently book.inc isn't included at feeds_nodeapi() anymore).
2. Renamed _feeds_nodeapi_book_mapper_cleanup to _feeds_nodeapi_book_processor for consistency.
3. Renamed _feeds_book_assign_children to _feeds_book_add_children (see #26, I think this is a more fitting name).
4. Added feeds_book_mapper table to feeds_schema (see #24).
5. Create feeds_book_mapper table in separate update function (see #24 and #26).
6. Added a target for book weight.
7. Removed mapping_counter from book_feeds_set_target, this messed things up when an item had a Book ID but no Book Parent ID (or Book Weight).

Somehow changing the main module for this mapper doesn't feel 'clean', but I can't think of another way as Feeds doesn't seem to expose a hook when deleting feed items.

This patch still doesn't fix:

1. The issue with changing the book hierarchy between imports (second half of #19). I only need to import my book structures once so this doesn't bother me.
2. Removing the items from feeds_book_mapper after import, as suggested in #26. This will cause problems for me because I'm doing the import of one book in different stages.
3. The other points raised in #26 (3 and 5). I didn't look into them.

hmm cant get this to work.. it does't seem to build the book structure, although it does fill the feeds table with book guid and parent guid

is it normal that feed_nid is always 0 ?

also, i always get a weird looking row that doesnt get removed when i delete the nodes
looks a bit like this (hope formatting works):

guid      feed_nid      book_guid      parent_guid      weight
              0             3936                         -13
3946          0                           3936            0
3945          0                           3936            0
3944          0                           3936            0
830           0                           830             0
.
.
.

any progress on this? Need something like this in terms of migrating from drupal to spaces/og environment. Will help debug if anyone has any additional code for this as I sat down to start mapping this out.

I haven't touched this for a long time, unassigning from me.

@btopro, the original tarball in #11 was kind of a hack, but it worked for me for the same use case. A little handholding and it might do the trick. Otherwise, it's just pushing forward from the last patch in the queue.

hmm ok thanks for the heads up. I just wrote a simple one for role I'd with the idea being feeds tamper could help make it very useful. Maybe the same can be applied here. Its on my roadmap

Here's some catch all code I'll be moving to a full release in the near future that attempts to enable this functionality. I'm able to import into the current group based on the current group context (which groups have books in my rig) -- https://github.com/btopro/elms/blob/master/profiles/elms/modules/elms_co...

The patch in #30 worked well enough for me for import but, as mentioned, breaks the hierarchy if it changes between imports (I get duplicate outline entries for the modified nodes).

The code referenced in #35 (feeds_node_helper) didn't work for me; the target callbacks didn't get run, never mind work correctly.

I use feeds_node_helper in production so it is working. There are two features in elms that implement feeds_node_helper's UUID / book outlining stuff that might be worth checking out. The one is called elms_content_import which has the feed packaged. The other is called elms_content_export which will provide a link to the book admin page to export and will dump to xml for use in the importer. Importer by default imports into organic groups but that's easy enough to swap out from the default.

http://drupal.org/project/elms_features

Any body have experience with D7?
Feeds came a long time, but still not fix this issue :-(

Just found this issue. I have some code somewhere that does most of this.

The code in #30 is way off. I only glanced at it, but there's not way that we're creating a new table to handle book mapping.

http://drupal.org/project/feeds_node_helper now has a full release for D7 that handles book parents correctly. It also comes with an example feature module that setups up the feed to ingest example xml output. Unless you'd want to move this functionality back into feeds which I don't think is necessary then this module provides the functionality requested via UUIDs.

This work is happening in #3073275: Mapping to books

@irinaz
This is a request for the D7 version though.

I think that only critical updates should be made for D7 as it is EOL software. Those who want to keep working with Drupal7-based CMS should plan to move to D9 or Backdrop and we can make updates for those modules https://github.com/backdrop-contrib/feeds