Integrate with other external data sources (WorldCat, LibraryThing, Open Library)

Also see code from:
http://drupal.org/project/bookpost
which is based on:
http://wordpress.org/extend/plugins/openbook-book-data/

This article http://arxiv.org/ftp/arxiv/papers/0805/0805.2855.pdf mentions other "external datasets" (useful for Geographic tags?)

GeoNames (http:///geonames.org) and the CIA World Fact Book
(http://www4.wiwiss.fu-berlin.de/factbook/) for geographic headings.
· the RDF BookMashup (http://www4.wiwiss.fu-berlin.de/bizer/bookmashup/) for links
to items that prompted a LCSH concept to be created.
· dbpedia (http://dbpedia.org)
Furthermore, there are additional vocabularies at the Library of Congress such as the Library
of Congress Classification, Name Authority File, and LCCN Permalink Service which could be
made available as RDF. The authors are also involved in the conversion of the RAMEAU, a
controlled vocabulary that is very similar

Also check out http://inkdroid.org/journal/2008/01/04/following-your-nose-to-the-web-of...

Would this help?

guessing publisher from ISBN prefix
http://worldcat.org/devnet/blog/2009/01/guessing_publisher_from_isbn_p.html

See this:

Open Library embeddable Book Reader
http://openlibrary.org/dev/docs/bookreader

And more:
http://wiki.code4lib.org/index.php/OSBW_Existing_Software

Perhaps this is all just a subset of linked data? In that case, any useful information about the item (current prices, ratings at amazon or others), the authors (biographies, pictures, etc.), the publisher (homepage, addresses?), people who own/read it, libraries that hold it, etc... would all be targets for this issue. =)

Another data source: the insight web service from random house.

To get direct links to cover images, TOCs and sample pages, see: http://www.randomhouse.biz/webservices/insight/spec.php#G

To embed a widget with a book, see: http://www.randomhouse.biz/webservices/insight/widget/userguide

For now I went ahead and committed to 6-DEV an embedded Google Books widget. Probably needs some work though.

I think that this should be something more modular; I plan on having additional modules that tie into the drupal_alter() hooks, add other hooks (e.g. alter the biblio data table, alter the holdings table, etc) so those other modules would be in charge of adding more information.

This would mean I would move the code that adds the Library of Congress information and Google Book Search link and widget into other modules.

This would also open up the possibility of these new modules actually not depend on millennium.module at all, since in fact, Google Books and other just need an ISBN or other information to work, and are not at all tied to Millennium. For instance, these modules could work using information from Biblio module, a CCK field deemed to hold an identifier of some sort, or use the Millennium.module's stored biblio array.

We could then have modules at different steps:
* Enrichment during import. Say your III record is missing the number of pages; it could be fetched from another source and added to the record. Or say you want to import the Table of Contents from LOC (like we do now).
* Enrichment during viewing (adding online fulltext viewers, adding links, etc.), kind of like WebBridge (or whatever it's called these days?) does on III.

How does this sound?

Awesome!! Please share when you have something/anything =)

Just another note:

Library thing API:
http://www.librarything.com/services/librarything.ck.getwork.php

I'm thinking this is *most* useful; I thin ka module could ask for the API key, and manage the upper limit of calls per day to keep in line with the terms of service.

See latest commit for the beginnings of (very basic, humble, horribly unscalable) configurable enrichment options for the module =)

http://drupal.org/cvs?commit=297112

BTW, yes, I killed, like, 4 kittens with that commit.

For examples for cover images, see:

http://dobby.darienlibrary.org/websvn/listing.php?repname=locum&path=%2F...

http://github.com/eby/sopac2-contrib/tree/master/covercache/

http://cheerfulcurmudgeon.com/2008/08/11/caching-free-librarything-book-...

This patch (committed) adds a cover image "finder" to the "enrichment" module.

It doesn't download the images but it finds them grabbing a portion of the file from a variety of services, and uses that fragment to determine if it's usable (by checking the image width).

Probably next would be to have a simple caching mechanism, and some way for the user to configure this.

I am not happy with the current enrichment setup though (it's dog-slow during manual imports); I think I could probably do best to separate it out into another module that can hook into modules like Millennium, biblio, views and CCK and can run independently of Millennium imports OR hook in to use pipelined requests to speed things up.

Note: see Feeds and if it's looking into aggregating data taking in "keys" to search for and storing them in "any" MySQL storage. (Guessing it *might*)

I've committed the beginnings of a new approach to fetch metadata, changing millennium_enrichment.module. See this commit: http://drupal.org/cvs?commit=324336

From there it might be simple (or simpler?) to make the new metadata classes fetch/show data via AJAX.

The rundown of that commit is this:
* Extra metadata, if stored locally, lives in its own table
* Harvesting can be done separate from millennium importing. (With the commit it can *only* be done separately for now)
* Millennium.module's hooks are implemented in Enrichment module so that the biblio array can be changed by Enrichment's metadata classes (say, you want to add links to OpenLibrary as if they existed in the MARC in the first place)

The enrichment/metadata classes still need some love, I think it's a good start to have something extensible. I have been thinking if it's preferrable that it live as a different module instead of inside millennium--- but for now I don't have the time to imagine all possible use cases to make it generic enough to be a standalone API module, or something that integrates with Feeds... I just want to ship with something useful =) Input welcome!

I've been looking at feeds and had a quick chat with alexb (the author). I think Feeds *could* support Metadata harvesting, but might not handle anything that would allow to aggregate stuff to a node (e.g. adding the metadata for indexing or viewing).

So, I'm tempted to still follow the current path (release Millennium Integration 6.x-2.0) with some basic metadata harvest/display functionality like what I mentioned in #19 and is already committed... and maybe for a later release Millennium could be more of a discrete API to just communicate with Millennium and leave the node creation and metadata aggregation to Feeds, views, etc. IF those modules can scale for massive fetches like we're doing now (using the Bookcart, for instance).

Still, I have not had time recently to ponder things much =)

This has been committed and is a good start IMO. Just needs documentation and maybe a little more testing.

Thinking of Linked data, see http://dilettantes.code4lib.org/blog/2010/11/linked-marc-codes/ .. we can probably just theme parts of the record to "speak" RDFa.