Spin-off from #1033392: Script loader support in core (LABjs etc.)

Problem

Javascript is starting to get big and is already too big for mobile. We need to make JS loading unobtrusive, fast and reliable to properly support mobile devices.

Goal

Provide a way to load all javascript async, handle dependencies and allow for ondemand loading.

Details

  • RequireJS is well documented and supports AMD (Asynchronous Module Definition) loading.
  • With the patch, page rendering time decreases. Initially loaded JavaScript decreases to 5.6 kb. All other scripts are lazy-loaded.
  • RequireJS is merely @nod_'s personal preference. Different loader could be used.
  • Alternatives:

Notes

  • Investigate usage of AMD; possible to decreases initial load to 950 bytes.
  • Current patch uses the order plugin, which shouldn't be needed with a proper JS architecture. Using AMD, dependencies should be declared in the module itself to increase performance and remove the need for the order plugin.
  • Not sure about the proper location for RequireJS itself (currently /misc).

Outside Drupal

  • since version 1.7 jQuery has an AMD and can be loaded with any AMD-compatible loader.
  • jQuery mobile is using RequireJS.

To use the full potential of this method core JS will need to be refactored to be more modular, refer to the parent issue to discuss the details of this.

Original report

by @nod_

I'm making a new issue because there are other important aspect to talk about in #1033392: Script loader support in core (LABjs etc.) and the actual loading is just a part of it.

The patch here is quick and dirty, it's a proof of concept and could be much more optimized should we use AMD (Asynchronous Module Definition) for core js files/modules. I've browsed around core admin, enlabed overlay all is working but I haven't tested scripts in footer and the like. Should work but you never know.

I've used RequireJS because it's well documented and it supports AMD loading. An other loader could be used, but today, i'm liking this one. I had to use the order plugin, long term this should go away by default, dependencies should be declared in the module itself hence loading even faster.

I've added requirejs files in /misc but that should be somewhere else ideally.

Apply the patch and see you page rendering time go down, way down. All js ever loaded in a script tag will weight around 5.6kb, the rest will be loaded after. If we go the AMD only road, it could go down to a 950 ish bytes file on initial load.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Status: Needs review » Needs work

The last submitted patch, core-load-requirejs.patch, failed testing.

nod_’s picture

I messed up inline scripts, here it is working, also loading the order plugin earlier. Not working very well for scripts in the footer yet.

klonos’s picture

...coming from #1033392: Script loader support in core (LABjs etc.)

I know this is D8, but I would love to test it as a D7 hack and see how things break ;)

nod_’s picture

Patches for groups, conditional include still fail but that's normal, scope footer isn't needed anymore with this kind of loading (at least if it's used, it's for different reasons than before).

Still a work in progress. I've looked at D7, but it's not as easy to port, so I won't be doing that soon, even if that's a really good idea.

nod_’s picture

Status: Needs work » Needs review

oups

Status: Needs review » Needs work

The last submitted patch, core-load-requirejs-1423500-4.patch, failed testing.

sun’s picture

Issue summary: View changes

Updated issue summary.

RobLoach’s picture

We'll have to set up a JavaScript registry via hook_libraries_info() for RequireJS to read for to get the dependency tree.

alexweber’s picture

+1 for RequireJS.

I've used Head.js, LABjs and $script.js a lot so somewhat partial to them ($script.js is my favourite at the moment) but after reading up on RequireJS it seems like a pretty solid project.

JohnAlbin’s picture

Issue tags: +frontend performance

Adding tags so this shows in the Mobile Initiative queue too.

ethanw’s picture

It is awesome to see work on this starting. Note that Require.js is not just about loading, it's also about being able to implement the Module/Exports JS pattern which is needed for more complex JS apps. Backbone architecture patterns use modules extensively for organizing and encapsulating Views, separating large apps into different files, building whole apps and enclosures, etc.

One note that a proper Require.js implementation will require (no pun intended) some port or inclusion of an optimization script like r.js. As I understand it, asynchronous loading works best with 3 or 4 parallel loading files, but suffers significant slowdown if used for a ton of small files ("priming the connection", etc), as discussed in this comparison by the LABjs and RequireJS creators. r.js aggregates files needed for different Require sets and makes the loading much more efficient. I'm guessing that in our case this would mean some sort of smarter "Aggregate JS" function which, when turned on, would aggregate site-wide, page-specific and theme related code into separate files an generate the appropriate Require code for that optimized state. I can imagine this involving implementation of the same format used by r.js or needing to modify Require.js to work with an alternate optimization scheme. Thinking aloud there, but wanted to make sure this consideration was included in development. Maybe after DrupalCon hits I can try to pull something together...

Edit: Added citation for JS optimization use case.

nod_’s picture

Yep that's what would happen in the ideal world. to use r.js we need to have AMD, and sadly that's not happening yet.

But there are a bunch of issues taking care of this already :

#1033392: Script loader support in core (LABjs etc.)
#352951: Make JS & CSS Preprocessing Pluggable

I see this issue as only taking care of the dirty work of building the RequireJS call, for how JS is architectured and all, that's somewhere else. You can try this patch with js aggregation enabled, it works :)

I think the perfect solution using requirejs properly will end up in contrib, we have until december to change that.

nod_’s picture

Some details about RequireJS license, it was originally "Modified BSD/MIT/GPL" and was later changed to "Modified BSD/MIT" because the Modified BSD is apparently compatible with GPL.

I must say it's a bit over my head but here is the thread talking about the change: https://groups.google.com/d/topic/requirejs/Ze3cnzXJ5M8/discussion

Now is someone could check that this allow requirejs to be commited in d.o repos, that would be nice.

alexweber’s picture

@nod_ according to the information here we are good to go.

The MIT license alone is acceptable and when multiple licenses exist we can choose the one that fits our needs best! :)

nod_’s picture

Sandbox here: https://drupal.org/sandbox/nod_/1545078, I should probably close this one…

nod_’s picture

Status: Needs work » Closed (duplicate)
nod_’s picture

Issue summary: View changes

add details

RobLoach’s picture

UMD provides compatibility across Globals (what we use now), AMD (require.js) and CommonJS (node.js). If we were to use a module-loader, we would want to use a UMD pattern. Best case is ES6, but that's not available across all browsers yet. UMD provides a compatibility-layer until ES6 is out.

nod_’s picture