This is a master issue to track progress of the 8.x-3.x release of Libraries API. This will be a completely revamped release. As in Drupal 7 Libraries API remains an important requirement for D8 sites and overall CX. Though Drupal 8 core has introduced improved library management tools (libraries.yml and unified library loading) it still does not offer a solution for handling external library dependencies that may be shared across multiple extensions. As a result this remains the primary problem space for the Libraries API module.

Latest dev release is https://www.drupal.org/project/libraries/releases/8.x-3.x-dev

How to Use the New API

The main concepts and terminology of the new API are documented in libraries.api.php. This is required reading for the following. Please do comment below if things there are lacking/missing/incorrect so we can improve it. That way the best documentation will be available to anyone installing Libraries API without having to dig through the issue queue.

The following will be an explanation of how a module (or theme) in Drupal 8 can attach/load an external library and what happens behind the scenes to achieve that with links to issues where functionality is missing or lacking.

Declare a dependency on the library in the *.info.yml file.
The syntax for declaring library dependencies is as documented in libraries.api.php:
      name: My module
      type: module
      core: 8.x
      library_dependencies:
        - flexslider
        - jquery_mobile
    

Libraries API will validate that information when the module is installed (#2774313: Validate library dependencies when extensions are installed and more gracefully deal with library-specific errors at runtime):

  1. It checks whether we have a library definition by that name on the library registry (#773508: [meta] Provide a central repository of library information (i.e. libraries.drupal.org))
  2. If not it aborts installation. If so, it downloads the library definition
  3. If the library has dependencies it goes through this whole process for those
  4. If the library requires installation (that is not always the case) it will try to install it

(If either the library definition is already there or the library is already installed it skips the respective steps, of course.)

Use the library using just its name
For asset libraries:
Libraries API registers all external asset libraries that are dependencies of modules with the core library system. Thus, given the above example module info file, the Flexslider library can be attached to a render array as documented in AssetLibraryInterface:
    $build['#attached']['library'] = ['libraries/flexslider'];
    

For PHP file libraries:
PHP file libraries can be loaded in an ad-hoc fashion by calling

    \Drupal::service('library.manager')->load('some_php_library');
    

.

Missing features

In addition to the issues linked above, some features are missing to provide a rich developer experience

Versioned metadata
Drupal 7 Libraries API has a versions key, which - albeit there being some issues with the specific implementation - we conceptually need to bring into the Drupal 8 version. The idea is that library metadata can change between versions, i.e. libraries can rename files or change their version detection scheme and we need to cope with that. No issue for this yet.
Loading of minified assets
Drupal 7 Libraries API has the concept of variants which allows libraries to specify a source and a minified variant. The problematic part about the Drupal 7 implementation is that modules have to specify which variant to load themselves. We should handle this generically and simply put a checkbox on /admin/config/development/performance and then load either source or minified assets for all external libraries that support this.

Overall Architectural Considerations

Meta issue pending? - Encapsulation of library metadata
A topic of critical importance that will be addressed in new ways in 8.x-3.x is the management of library definition metadata. In D7 modules implement hook_libraries_info() to register a library, and though this offers a conceptually simple model, having library definitions in modules means there is poor encapsulation of the base library metadata. This limits the utility of an intermediary API. The new D8 intent is to somehow leverage a canonical registry of library metadata, either locally or remotely, such that library definitions can be packaged independent from implementing modules. Some discussion of this topic has been introduced in #2090623: Turn hardcoded library_load_files() into a more modern, flexible system though it may warrant its own meta issue (it's not yet clear if there's a one-size-fits all solution or if multiple configurable options are needed). Ideas introduced include:
  • A remote central repository of library definitions managed by/for the Drupal community (see next section - #773508).
  • Leveraging definition data from existing cloud services (cdnjs.com).
  • A hybrid approach that allows use of remote definitions with the ability for local overrides.
#773508: [meta] Provide a central repository of library information (i.e. libraries.drupal.org)
We want to provide a central repository of the Library classes described above. We need a canonical list of library machine names each of which maps to a certain Library class, which is canonically developed/maintained. The current proposal is to develop a libraries.drupal.org site, which acts as that repository. How the code for the class will be maintained will have to be discussed. Given a library machine name, Libraries API will then query that central repository to obtain the Library class. How this can happen safely and conveniently will also have to be discussed. There was also an alternate idea to commit the Library classes directly to the Libraries API module. That would make things easier for users, but comes with an increased maintenance effort.

Non-Asset Libraries

The extent to which Libraries API can (or should) be a solution for non-asset libraries (beyond JS and CSS assets) is an open topic. If this continues to be a priority (do we need a simple y/n issue for that?) the following issues are potentially relevant:

Libraries 7.x-2.x already supported 3 types of libraries (JS, CSS, PHP). We want to
  1. Support more library types (Autoload, PEAR, ...)
  2. Allow certain libraries to have their own business logic with regards to loading, etc.

Both of these points can be easily achieved in an object-oriented fashion. Since (a) we gain automatic autoloading of the Library information files and (b) Drupal 8 is moving towards an object-oriented design, this seems like a natural fit. We will organize both our own files, as well as the Library classes in accordance with PSR-0.

#542940: [meta] Support for downloading libraries via Composer
Since http://packagist.org/, which uses Composer, seems to be the PHP package archive, we want to download all libraries (possibly even including JS and CSS libraries) from there. That means that if a certain library does not have a composer.json file, and/or a package on http://packagist.org/, this needs to added "upstream" first, in order for Libraries API to support it. Each Library class will include a method called "getComposerURL", which Libraries API will use to download the library.

All issues of the 8.x-3.x branch

Comments

tstoeckler’s picture

Issue summary: View changes

Add link to issue list.

gdd’s picture

Issue summary: View changes
tstoeckler’s picture

Version: 7.x-3.x-dev » 8.x-3.x-dev

Changing version to 8.x-3.x

tstoeckler’s picture

Title: [master] Libraries API 7.x-3.x » [master] Libraries API 8.x-3.x

Also updating title now (fail...). I had updated the issue summary before.

tstoeckler’s picture

Issue summary: View changes

Updating for 7.x-3.x -> 8.x-3.x

rjacobs’s picture

I just wanted to see if there have been any updates on any of this. From what I can see there have not been any new (meta) notes on the D8 branch since 2012. I think it would be especially interesting to know if #1704738: [meta] Make Libraries classes implementing LibraryInterface is still in the works as that would have a big impact on how other developers make use of the API (and it would be good to plan accordingly). There have not been any updates on that issue since 2012 either.

webchick’s picture

Similar question here. I've heard varying things from various folks about this module in D8, ranging from:

- We don't need it anymore; use libraries.yml.
- Core's libraries.yml is too limited; this module remains a big requirement for D8 sites.
- This module's going to do crazy sci-fi stuff that no one ever imagined, which is awesome, but only needed for special projects.

Any help/guidance on this would be lovely. :)

rjacobs’s picture

Tobias has more of a definitive vision on all this than anyone but I think it’s safe to say that it’s closer to #2 - “Core's libraries.yml is too limited; this module remains a big requirement for D8 sites”. To be more specific the module is needed for robust handling of external lib dependencies that may be shared across multiple extensions, which core library handling does not address.

Currently the foundation work to implement a decoupled architecture that will work for D8 (while also being flexible enough to hopefully address some lingering limitations of previous D7 versions) is being addressed in #2090623: Turn hardcoded library_load_files() into a more modern, flexible system. Tobias has really gotten the concept moving, though plenty of work and support is pending.

It's also worth noting that I believe the intent (at least for asset-based libraries) is to fully leverage and integrate with core's library handling in such a way that some of its limitations can be addressed here in contrib. In this regard there have been a couple core issues geared specifically toward extensibility of core lib handling, such as #2358981: Provide a mechanism for dynamic library declarations, which were somewhat driven by the use-case of Libraries API.

Getting this issue summary updated to reflect some of the more topical/active issues that are children of this overall port topic will likely help a bit, and that might be something that I can take a humble stab at.

@tstoeckler, I hope I'm summarizing things effectively :)

rjacobs’s picture

I also now see #2607762: (Duplicate) Libraries API from the Drupal 8 Contrib Porting Tracker project, which points here. I suppose that will be an entry point for many people looking for port info, so we should factor that in.

tstoeckler’s picture

Yes, all that is very much correct @rjacobs, thanks!

I will prioritize providing an updated battle-plan here soon, it's very unfortunate that this issue has been rotting for so long, sorry about that.

webchick’s picture

That's a great update, thank you so much!

I'll update the corresponding ticket in contrib_tracker.

rjacobs’s picture

Issue summary: View changes

I edited the issue summary in a first attempt to get things a bit more up-to-date. It's pretty verbose at the moment, but I just wanted to get as many dimensions covered as possible. As priority issues become more clear (and maybe some new meta ones opened), this can always be refined. I'll also hold off on marking any other issues as children until things are a bit more clear.

Don't hesitate to revert/add/implode/refactor in any way needed.

I'll also try and help-out in #2607762: (Duplicate) Libraries API.

rjacobs’s picture

Issue summary: View changes

I made a few more small edits to the summary. However, I'm not really sure what to make of #1704738: [meta] Make Libraries classes implementing LibraryInterface. A lot of thought and work seems to have gone into that issue, and it appears to have some potential relevance for asset libraries as well as other (php) libraries. That said, the work and logic in #2090623: Turn hardcoded library_load_files() into a more modern, flexible system seems potentially conflicting (though admittedly I have not looked in detail at the new PHP-loading class that has come out of #2090623). I suppose the bigger question is whether-or-not support for non-asset libs will be a priority.

ptmkenny’s picture

Could this issue please be added to the project page so that those looking for D8 compatibility information can easily locate it?

rjacobs’s picture

Hummm, I was actually under the impression that a little notice was automatically added to the project page for any project referenced in an issue queue thread of the Drupal 8 Contrib Porting Tracker, but I see that that is not the case here. I guess that only applies to project with 8.x branches that have tagged releases? Anyway, I added a note to the project page about 8.x-3.x.

Edit: I think the contrib_tracker notice was working on the project page, but something is going wrong as a dup Libraries API thread was opened in contrib_tracker. That dup seems to be causing a conflict in the d.o. porting status calculation. Hopefully that will be rectified soon. Anyway having the text message that I just added to the project page, in addition to the contrib_tracker notice, will probably not hurt.

rjacobs’s picture

Updated issue relationship to correct contrib_tracker issue.

cilefen’s picture

@rjacobs Nice work on keeping this updated. Thank you.

I don't understand this:

To be more specific the module is needed for robust handling of external lib dependencies that may be shared across multiple extensions, which core library handling does not address.

I have been able to use the core system to define external dependencies, in YAML and in hook_library_info_build(). Can you explain further what robust handling of external lib dependencies means in this context?

rjacobs’s picture

@cilefen, I think the keyword is "shared". External lib dependencies can certainly be declared within an extension using only core APIs, but that declaration becomes very tightly coupled to the extension itself. So core does not offer a way for a common lib dependencies (e.g. a common JS lib) to be declared canonically and then used gracefully among multiple extensions. Furthermore, if multiple extensions declare a common dependency separately via core's APIs, and they each try to load that dependency during the same request, a whole load of conflicts could occur.

Of course a extension could declare a library that other extensions could use (core's jquery lib can be used as a dependency in any other module), but that's not very useful in a contrib world when both moduleA and moduleB require fantasticJSLib but there is otherwise no need for moduleA and moduleB to depend on each other. On the other hand if both moduleA and moduleB use (and depend on) Libraraies API, we can handle the declaration of fantasticJSLib as a core library definition on their behalf. This ensures the lib is always loaded in a consistent way for all sites that use any combination of moduleA, moduleB, etc....

SKAUGHT’s picture

tstoeckler’s picture

Issue summary: View changes
tstoeckler’s picture

Issue summary: View changes

Just added some important info to the issue summary. It's still not super straightforward to read through all this, but we're getting there. :-)

oadaeh’s picture

Issue summary: View changes

The link to libraries.api.php was broken. I fixed it, but you might want to make sure it's pointing at what you wanted.

tstoeckler’s picture

Oops, thanks. Yes, that's perfect.

oadaeh’s picture

Also, I wanted to add that having some part of the following from the libraries.api.php (especially the first sentence) on the project's home page under the 8.x-3.x heading would have saved me a bit of time and frustration:

* @subsection sub_types_asset Asset libraries
* With Drupal 8 relying on Composer for autoloading and dependency resolution
* of PHP libraries, asset libraries are the primary use-case for Libraries API.
* Because asset libraries cannot be loaded ad-hoc, but must be attached to a
* renderable element, Libraries API registers external asset libraries that are
* required by the installed extensions with the core asset library system. See
* AssetLibraryInterface for more information.

oadaeh’s picture

I added a note under the Description heading on this page: https://www.drupal.org/node/2170763

jrockowitz’s picture

Hi, I know how challenging this task is and I just wanted to share my workaround.

So my YAML form module is utilizing 6 external JS libraries and I needed to figure out an easy way to download everything into the /libraries directory and I came up with a very basic solution, which was to use drush make.

The YAML form module comes will a yamlform.libraries.make.yml file which I was originally just cut-n-pasting into my own project.make.yml file. I realized that drush make can be executed without downloading core, so using just drush make --no-core {MODULE_PATH}/yamlform.libraries.make.yml {DRUPAL_ROOT}/libraries I was able to download all the YAML Form module's libraries.

I then created a drush yamlform-libraries-download command that downloads all my module's libraries into the /libraries directory.

This is not the perfect solution and composer should be used for PHP dependencies but this does allow drush make to do all the heavy lifting. Personally, I prefer drush make over composer.

The other thing, I am experimenting with in the YAML Form module is loading third party libraries from a CDN when they have not been installed locally. (See yamlform.libraries.inc & yamlform.libraries.yml) This feature is also super helpful when someone is testing a module using Simpletest.me.

Hope this helps.

akalata’s picture

I'm going to go ahead and guess there isn't support for attaching this in templates via Twig's {{ attach_library('library_name') }} yet?

rjacobs’s picture

I'm going to go ahead and guess there isn't support for attaching this in templates via Twig's {{ attach_library('library_name') }} yet?

attach_library() in twig should work fine.

The module uses core's native library APIs for asset loading, so any core option for attaching a lib to content should work. Check out the usage section of the new docs for 8.x-3.x. So if your theme declares an external library dependency (managed via Libraries API) in its info.yml file you will be able to use that library directly in twig with something like {{ attach_library('libraries/library_name') }}

jeromewiley’s picture

UPDATE: I think I found it -- thanks user @eporama. There is a link in the sidebar for -nonmaintainer code.

I'm probably missing something super obvious -- but where can I download the 8.x version of libraries? I'm trying to test / use Splashify, and on their project page, they say "we need more people to use this on Drupal 8 so we can improve it" -- but Splashify requires Libraries module. However, Libraries does not have any D8 version listed, only a link to this issue page.

andypost’s picture

Issue summary: View changes
Status: Active » Needs review

That's a known bug at project page now, so makes sense to release at least alpha1 to make it usable with composer

D8 compatible releases could be found at https://www.drupal.org/project/libraries/releases?api_version%5B%5D=7234

PS: added link to IS Latest dev release is https://www.drupal.org/project/libraries/releases/8.x-3.x-dev

Alan D.’s picture

Priority: Normal » Major

+1 for an alpha release.

It is hard to track the stability of dev branches, yet this is particularly vital now with the nasty Drupal 8.4.x bug that could force everyone into the latest dev version: #2915332: Constructing service "validation.constraint" from a parent definition is not supported at build time.

kenneho’s picture

Has anyone used the latest release (8.x-3.0-alpha1) in production yet, and would like to comment on how ready it is for a production environment?

fgm’s picture

We've been using Libraries 8.x-3.x for 15 months, in code that is both in production and regularly changing, so I'd say it's pretty stable.

I'm more worried about actual maintenance than stability issues: latest commit is 16 months old although there has been a few RTBC issues for months which have not been not merged or commented upon.

Chris Matthews’s picture

I was also wondering about the status of 8.x-3.x as it's now been 21 months since the last commit . Would it be possible to provide an update re: which issues are preventing the next release, be it -alpha2, -beta1 or even a stable 8.x-3.0?

Active (16 issues)
Needs work (3 issues)
Needs review (3 issues)
RTBC (2 issues)

Chris Matthews’s picture

I think the conversation over at issue #2939919: Tag an 8.x-3.0 release is important to mention for the folks following this thread:

#4 by @tstoeckler

I just tagged an 8.x-3.0-alpha1. I am not really happy with the current state of the module, so I am actually not comfortable with creating a stable release. We did have some quite ambitious plans for this module at some point, unfortunately we never achieved them. But even if we decrease the scope of the module I'm not happy creating a stable release with all the stuff that is alreay in there that is broken. So I am going to mark this issue postponed for now. If any of the other maintainers feel differently I certainly don't want to hold anyone up.

#6 by @sun

Libraries API is pretty important for D8 sites and installation profiles. If possible in some way, we should release a tagged release of 8.x-2.x in some stable-ish form to allow the whole (Drupal) project and community to make progress on many other fronts. The comments on this issue sound like that release could even be a 8.x-2.0.0 after quickly cleaning up some experiments? All code targeting a major rewrite / refactoring should be postponed to a new major 3.x at this point in time of the Drupal Core release cycle.

#7 by by @tstoeckler

So the problem is that the asset system got drastically revamped between D7 and D8 so a straight port is simply not possible. That's not just in terms of removed/revamped APIs, but conceptually Libraries API cannot work as it did in D7, as it relied on the runtime adding of assets (drupal_add_js/css()) which got completely removed (i.e. not just replaced with an OO-version). If someone has concrete plans/ideas or even code for a less ambitious version of a D8 Libraries API than what is currently in 8.x-3.x I'm fine with going for a 8.x-2.x, but so far that simply doesn't exist.

Chris Matthews’s picture

@tstoeckler in #2939919: Tag an 8.x-3.0 release said,

If someone has concrete plans/ideas or even code for a less ambitious version of a D8 Libraries API than what is currently in 8.x-3.x I'm fine with going for a 8.x-2.x, but so far that simply doesn't exist.


Would it make sense to open a 'Plan' issue to discuss a less ambitious version of a D8 Libraries API that would be released under a 8.x-2.x branch?

Chris Matthews’s picture