[meta] Provide a central repository of library information (i.e. libraries.drupal.org)

Very good question. :)

irakli just recently added the first documentation page: http://drupal.org/node/735160

Though I'm not sure whether this is the proper way to move forward. #719896: Add a hook_libraries_info() contains some additional considerations around this topic -- namely, whether Libraries API itself (or a separate project with plenty of co-maintainers?) shouldn't register libraries that are used by more than one module.

Furthermore, the more modules are using Libraries API, the sooner we will have to deal with #465908: Scan: Name clashes

Now that #719896: Add a hook_libraries_info() is in, I think we should have a documentation page, which is basically a table listing all known implemented libraries. Whether Libraries API ends up implementing some libraries itself or not should not hold that up. What I imagine is basically something like the attached screenshot (done with Firebug). I would have actually created it already, but I'm not on the Docs team, so can't add < table > 's. Thoughts?

While a documentation page like I proposed in #2 wouldn't be inherently bad, I think, we need something which is easier to access for the ordinary module developer.
1. Only people on the docs team can write < table > 's.
2. Editing a table that could get potentially huge is a pain.

I put together a little install profile, which sets up a node type ('library'), which lets you enter a name and the latest (supported) version of a library, and then lets you upload a libraries.info file. It's not really finished yet, but I do like the direction.
The code is here: http://drupal.org/sandbox/tstoeckler/1194784
and you can check it out live here: http://libraries.euve23100.vserver.de/

I'd love to hear your thoughts.
@sun: Just in case you have some time to spare :), you of course have commit rights.

wow, nicey! And clever! :)

Unfortunately, one issue with that: Not all libraries are defined in .info files. Many are defined in PHP :-/

Otherwise, this looks like a nice idea. Let's think about it some more.

---
Just an architectural note: While install profiles are treated like modules now, it isn't really safe (yet) to rely on them being 100% treated like modules. The profile should contain the actual module functionality in a separate module in a sub-directory of the profile.

Hi,

I'm in the middle of developing a large module (for doing Electronic Thesis and Dissertations), which is dependent on four or five external javascript libraries (including, for example, json-js). I'm planning on using the Libraries module to support this, but I want to make sure that the names and other info I've got associated with the libraries are set up in a way that makes sense.

json-js, as mentioned above, doesn't have version numbers, but it does have a date embedded in the code. I've been using that as the version number, but perhaps that's not the right approach.

In my opinion, what's needed is a sort of mini drupal.org, but only for library definitions. Each library has a maintainer, or couple of maintainers, who are not necessarily the authors of the library. The maintainers are responsible for keeping a canonical .info file, which developers can use when they want to interact with the library, by transforming the .info file into php code for hook_libraries_info().

This platform could even host a 'drupalfied' version of the library with a .info file attached. The maintainers would be responsible for keeping it up to date, just like a Debian package manager, for example.

I realize that I am proposing a massive amount of infrastructure, but I'd like to get this discussion started again. Even a list on a wiki somewhere would be an improvement over the current shooting in the dark.

Re-purposing this issue, due to latest discussions.

The plan is to have a libraries.drupal.org (or equivalent) which acts as a central repository for libraries. Libraries API would then interface with that site to automatically download library information (what is currently library info files (in 2.x)), and then use that library information to automatically download the library.

I will write a proper issue summary soon.

We have not tagged a 7.x-3.x yet, so for now I am simply tagging this. @sun, if you can think of a cooler tag, feel free. :-)

Note that this is intentionally still "critical", this is a central piece of the envisioned 7.x-3.x Libraries API.

Forgot the reason, I initially wanted to comment: Update the issue title to match the new proposal.

Also, something I just thought of: http://drupal.org/sandbox/tstoeckler/1194784
This is quite some time ago, but I cooked a really small install profile that provides such a library repository. Note that the in light of the new proposal, the code is almost certainly completely obsolete. I'm mentioning this only for reference.

Also, I had no idea what "Component" this is, but "Documentation" felt wrong.

Tagging with the new version and removing unused/superfluous tags.

Changing version to 8.x-3.x

I have packaged the Libraries API for Debian and Ubuntu

This means that library dependencies can be fulfilled by packages in Debian or Ubuntu systems, the details are explained here:

http://danielpocock.com/drupal-on-debian-libraries-api

I'd be interested in understanding how this can be co-ordinated with the libraries.drupal.org effort, if that progresses. It would be helpful to avoid duplication of effort, the Debian pkg-javascript team already does a great job of locating genuinely free GPL compatible libraries and packaging them for distribution.

Sadly, the progress on this has basically halted for the last months. So not a great risk of duplication :-/

I am really interested in what the best way forward for this problem is, i am working on the module https://www.drupal.org/project/library_pack which currently has around 30 library definitions inside it.

The problem I face is that i when deciding what the machine names will be used for each library and what directory structure the library should have, i.e. should it be the same directory structure you get when you clone the git repo so that the source files are in /dist or /build or should the source files be extracted by the user and placed in the root of the library folder, i'm not sure what the best thing to do here is.

I don't want to have to change machine names or directory structure at later date as this will affect any sites using the old machine names.

wow, that looks great.

Wasn't aware of library_pack module. I always thought that starting a central & community-maintained registry like that might be fruitful. We didn't want to kick-start such a registry directly in Libraries API, because we didn't want to get distracted from architectural/technical work. My hope was that someone would simply try it out in a separate project first, and then see how it goes.

In case it will be adopted, we could/should consider to merge the registry directly into Libraries API, so people don't have to download/install a separate module + consider to add 2-3 co-maintainers (must be multiple) who are able to create/update/maintain definitions (only).

Regarding library directory naming:

The dist/release archive should be the leading factor, because users primarily want to use published releases. (Also, the raw source files often contain an unprocessed version placeholder only.)

The most common and most widely adopted approach is to use the top-level directory name of the dist package/tarball as-is, so that a dist archive can simply be extracted from within the ./libraries directory.

But of course, that only works if the contained directory name is unique, consistent, reliable, and sensible.

Some dist archives do not contain a top-level folder name. Some other dist archives are using a folder name that (A) contains a version suffix, (B) is a semi-random [commit] hash, (C) is potentially ambiguous with other libs, or (D) is an otherwise weird/confusing name.

Some dist archives may also use superfluous/excessive nested folders (e.g., "two" nested top-level folders whereas the inner folder is the actual/consistent one and the outer folder is unreliable).

In a couple of such cases, we reached out to the library vendor/developers/maintainers, so as to discuss whether they can adjust their release packaging scripts/tools to produce a better dist archive. That worked out in most cases (for the benefit of everyone, not just Drupal).

To sum up:

In most cases, where the dist archive contains a reliable/sensible top-level folder, that folder name is used as-is in the library definition, so that users can simply extract and do not have to rename anything.

Only if that isn't the case (and can't be adjusted upstream), a custom folder name is "invented", which is typically the all-lowercase + underscored/hyphened transliteration of the library's official name, potentially including a dot-delimited framework namespace prefix (à la 'jquery.'). For example, "jQuery Tooltip NG" may transliterate into jquery.tooltip_ng or jquery-tooltip-ng.

Thanks for your reply, much appreciated.

So to clarify using a real example i.e. Enquire.js

The github repo is https://github.com/WickyNilliams/enquire.js

Cloning that into the libraries folder gives the following structure

libraries/enquire.js/dist/
libraries/enquire.js/spec/
libraries/enquire.js/src/
..
..
..

The files we want are located in the dist directory which is fine, what I am unsure about is if there is any downside (be it security or otherwise) to having the rest of the files hanging around.

There probably isn't any real choice to make here as the ease of adding a library to a 'make' file with the commands

libraries[enquire.js][download][type] = git
libraries[enquire.js][download][url] = git://github.com/WickyNilliams/enquire.js.git
libraries[enquire.js][directory_name] = "enquire.js"
libraries[enquire.js][destination] = "libraries"

means that you get whatever you are given from the source repo.

I keep going back and forth for some projects on what the machine name should be, for example i chose jquery.cycle2 instead of just cycle2 which violates the standard previously defined, what i could really need now is the ability to define 'aliases' for a project so that the same library can be referenced by either with no ill effects. So if there is libraries_load('jquery.cycle2') call out in the wild it would just be an alias of libraries_load('cycle2') and the world wouldn't end if I misguidedly choose a machine name that is different to a machine name already used in another project that i wasn't aware of.

If anyone could possibly give me some guidance on what to do in the situations when there is no version present in the main javascript file for a library, I have two options availble to me at the moment.

The first is to force a version number and ignore detection.

The second is to use another file that may have the version number in it, however is it safe to rely on a file such as bower.js being present for example and using the version number defined inside it?

For example https://github.com/paulirish/matchMedia.js has the only reference to a version in the bower.js file.

I have done an update to https://www.drupal.org/project/library_pack using the feedback from Sun above, there are now approx 50 libraries in the module, i have split them all out into their own sub directories to ease maintenance.

Please note that some work to shift back toward a working model of this concept for 8.x-3.x is shaping-up in #2770983: Implement canonical repository of library definitions (iterative steps toward libraries.drupal.org).

I think it's safe to say that #2770983: Implement canonical repository of library definitions (iterative steps toward libraries.drupal.org) is now the place to further discuss this. There is still lots to figure out (naming conventions, etc.) but there is now a project dedicated to this canonical registry effort at https://www.drupal.org/project/libraries_registry and base architectural support for it (and plenty of other remote registry possibilities) committed in 8.x-3.x.

Note also that data from Libraries Pack was used to "seed" the current registry for proof-of-concept testing.

Plenty of documentation and planning is still needed though. Also, I suppose the potential to backport this would now really need to be a separate issue.