Questions to resolove, see #55:

  1. How to aggregate all the js/css files used in the current page the answer belongs #1048316: CSS and Javascript aggregation improvement - meta issue (use this or that module as basis to put in core, etc.)
  2. How to load said scripts and CSS into the page so that they can be used on the front end, i've started a little something here #1423500: Use RequireJS to load all JS,
  3. And the most important one for last: How are we going to use theses script?

---

Drupal 7 improved the CSS and JavaScript aggregation methods so that we don't get dozens of different large aggregates for different pages.

However, this also means more js files loaded on each page, which has it's own issues, since browsers can only load and execute one javascript file at a time.

However, there is lovely, lovely http://labjs.com/ which allows for parallel loading and execution of js, just seen a contrib project for this http://drupal.org/project/labjs. - LABjs is being used on twitter and examiner.com amongst others.

It's MIT licensed so we'd have to speak to the author about dual licensing. I haven't reviewed the contrib project yet to see how well it works, however it's the sort of thing which could likely benefit from being built into core drupal_add_js()/#attached

Comments

jcisio’s picture

Subscribe. That could be great. Currently the labjs module must replace misc/drupal.js with its own version, thus I'd love to see it in core.

Vacilando’s picture

Subscribing.

alanburke’s picture

subscribe

droplet’s picture

sub+

there also have
ControlJS
http://stevesouders.com/controljs/

HeadJS
http://headjs.com/

RobLoach’s picture

Very nice...

Wim Leers’s picture

Issue tags: +WPO
Bevan’s picture

jcisio’s picture

If LABjs is in core, we won't need #784626: Default all JS to the footer, allow asset libraries to force their JS to the header any more. It will be equivalent whether a script is in header or in footer. It will be equivalent to put drupal.js, jquery.js... in footer. The only problem are scripts with document.write(). Even in that case, the temporary solution that I use in Google Ad Manager shows that it works perfectly with lazy loading mode.

mikeytown2’s picture

Subscribe... #1071368: Breaks Add This and Administration menu; integration with labjs
I'll be looking at various js loaders and seeing what ones I can put into AdvAgg. This is a 6.x module as there are a lot of sites out there still at 6.x

Shellingfox’s picture

subscribe

catch’s picture

Status: Needs work » Active

No patch here yet.

mrsinguyen’s picture

Status: Active » Needs work

subscribe

RobLoach’s picture

#1048316: [meta] CSS and Javascript aggregation improvement and #352951: Make JS & CSS Preprocessing Pluggable might be the first steps. Seems like this should be swappable before making it into Drupal core.

thehong’s picture

catch’s picture

Title: LABjs in core » Script loader support in core (LABjs etc.)

Re-titling.

RobLoach’s picture

Not sure which Script Loader to use? http://t.co/JpwIqN0

Dion posted the comparision on Twitter. Instead of settling with one for Drupal core, we have to make this pluggable.

mikeytown2’s picture

I think we should include 1 in core as a module. It will force all JavaScript added to be compatible with JS loaders as a result. And like Rob Loach said, this should be pluggable.

Not sure how many care, but I've made advagg fairly pluggable now. You can select a different function to render your script tags; what that means is js files added by drupal_add_js could be included in the JSON response and loaded on the client side. Same for css... actually js/css use almost the same codepath. Checkout readme.txt for a high level overview.

jcisio’s picture

Status: Needs work » Active

I agree that this should be pluggable. However, the required features should not be rich so that it can be pluggable. Being maintainer of labjs.module for a while, I think these features are required:
- Parallel loading (of course)
- Reserved execution order
- As much compatible as possible

Other features are not necessary, or we'll stick with one loader, like we're sticking with jQuery. Well, even in that case, it could be good. But I just don't see why we need other features, supposing that this module/subsystem is optional, they are just unnecessary and introduce more bugs.

bensnyder’s picture

+1

alexweber’s picture

+1 I like head.js

effulgentsia’s picture

Subscribing. Marking #1149770: Consider using RequireJS for loading JS files needed after an AJAX request a duplicate. Adding http://requirejs.org/ as a candidate for us to consider.

sun’s picture

  1. If this is pluggable, then I suppose the current "static" loading is the first implementation.
  2. There can only be one, so a simple OO class and interface as new subsystem is sufficient.
  3. Same applies to the client-side, modules and themes need a unique abstracted API to work with whatever implementation.
  4. Compatibility with Drupal core assumptions and requirements is crucial. The interface might allow us to enforce certain parts, but not everything; especially when it comes to the client-side.
  5. Debugging and handling of bogus bug reports will be hell for project maintainers. There must at least be an easy way to switch implementations.
Owen Barton’s picture

Some great ideas here - the getify blog post and spreadsheet are both interesting resources. Identifying what the API for a pluggable system might look like is going to be interesting.

One thing that has not been mentioned is that sufficiently small script loaders could (perhaps optionally) be placed inline on the page, eliminating an additional (high latency to data ratio) http request.

Owen Barton’s picture

One thing that just occurred to me, is that next generation JS aggregation would greatly benefit from the possibility to load (and cache) JS without executing it. This allows us to include commonly needed JS in a page where it is not needed (aggregated with other JS that is needed), without needing to execute it, preventing a likely additional http request on a subsequent page view. Drupal has several JS files that could benefit from this approach - they are tiny and make no measurable impact on the aggregate size (but a very measurable http latency impact), yet we are unable to aggregate them because doing so also executes them. Looking at the comparison grid (line 29) it looks like ControlJS, YepNope.js, curl.js, PINF JS Loader, BravoJS support this. Of these curl.js looks particularly interesting, although I haven't gone through the implications of using this in a Drupal context (using AMD seems like a benefit, but I think may imply a lot of work).

Another implication of using a loader with (very likely) parallel behavior is that to get the most benefit we _may_ want to split the aggregate file up into a small number of roughly equally sized sub-aggregates (perhaps ~3...would need research) to maximize throughput. This was discussed some in another issue, but I can't find it now...

mikeytown2’s picture

@Owen Barton
Advagg's bundler uses a default value of 4 aggregates; in firefox 4 the max is 6 on my box. Because of the ability to change the function that writes out the script tags, advagg should allow any script loader to be used in a 3rd party module.

Once 6.x-1.0 is out; I'll work on a 7.x version of advagg. Once 7.x is stable I would like take the lessons learned and bring the important parts into core for D8. Here is an example of a lesson learned #1148204: Setting JSON values to objects/callbacks via drupal_add_js extension (or similar)

Jeremy’s picture

Subscribing.

kurtzhong’s picture

Mark.

bryancasler’s picture

subscribe

stevetweeddale’s picture

Nice. I think a client side loader could be an ace little modernisation/optimisation of the Drupal front end. Certainly needs some careful thought though.

As for the question of which loader we use… might be worth noting #865536: drupal_add_js() is missing the 'browsers' option. The current patches assume bringing feature parity with drupal_add_css to be the best solution: that is, IE conditional comments. Conditional comments are massively limited, and it would be a whole heap better if there were some kind of clever client side conditional loading built into drupal_add_js.

With that in mind, this is a massive +1 for yep-nope, or if we go the pluggable route, that we include conditional loading in our api. Yep-nope does all the asynchronous preloading with decoupled (and thus ordered) execution, resource fallbacks and whatnot like the other libraries. I believe it was originally based off labjs, but was re-written with a really simple interface for making the loading of certain resources conditional.

It would open the door to replacing the icky/limited browsers option that's currently headed for drupal_add_js with a simple client side test option. Simple tests could just be included inline (like checking for HTML5 geolocation support with '!!navigator.geolocation;') or contrib code could even rely on external tests (such as 'Modernizr.geolocation'). Yep-nope even includes IE conditions built in as 'prefix' options... so we could still use drupal_add_js for 'just IE' if we wanted to (similarly to the 'browers' option issue).

I don't think it's much bigger than any of the others (I think it might be slimmer than labjs, at 1.6kb gzipped?), and I've heard no complaints in terms of speed, but that conditional loading functionality could be really useful. And by useful, I'm talking about improving Drupal as a framework here — I'm currently not aware of any core functionality that 'needs' it. Having said that, if the overlay were being written for D8/HTML5 it could've been a bit clever with using native hashchange events and only loading a polyfill where needed (though I fear the polyfill may be somewhat engrained in the BBQ plugin we used, so can't say how easy that would've been!). My point is core use-cases in D8 might come up; we'd have to see.

But for contrib, this would give modules that want to use, say, geolocation or web sockets a way to do so elegantly (providing fallbacks only as required, whilst not having to implement their own client side testing and script loading solution); at seemingly no additional cost to core over any of the other script loaders. There are some really cool things contrib could do with these new technologies; it'd be cool if the 'framework for innovation' was already there in core.

Another good reason for yep-nope is that it's wtfpl licensed, so we're good to use it as is, adapt it to fit more neatly into Drupal or whatever; we don't even need to worry about attribution other than as a courtesy.

I should also note, there is an issue in the html5 queue #1252178: Add Modernizr to core which expands the scope of this issue to bundling a feature test library such as Modernizr along with a script loader that might leverage those tests (yep-nope can come bundled with Modernizr). I'm yet to be convinced about bundling feature tests — but a script loader in core would certainly mean that feature test libraries such as Modernizr could live happily in contrib, should it not be deemed core-worthy.

Any thoughts? I may be entirely wrong :o)

RobW’s picture

Here from the modernizr thread -- from what I've read, yepnope is less robust than some of the other asynchronous/polyfill js loaders, but if it is good enough for the majority of use cases I think that its bundling/compatibility with modernizr should be taken into account. Whether or not modernizr is included in core, it is shaping up as a (very) widely used tool in front end dev and is likely to be included and included often on the module or theme level.

Planning for that integration would be a plus IMHO.

Jacine’s picture

Subscribing.

mfer’s picture

I'm going to break this down into a few thoughts and some recommendations.

Thoughts:

  • JavaScript file loading has changed a bit over the last few years. Two years ago one of the optimizations for performance was to use a script loader to async fetch files in browsers like ie6/7. In a few years the async flag on script tags could be in all major browser/versions changing how we optimize around this. My point is that how we optimize for browser/versions now will be different now than we do when Drupal 8 comes out and will be different from how we optimize JavaScript when Drupal 8 is 3 or 4 years old but still supported.
  • Modernizer is great but.... not everyone needs it, it adds additional stuff for a webpage, and causes a hit to performance. Performance is interesting right now. For example, JS that takes 100ms to run in a desktop browser will have about 1 second to execute on mobile. Mobile devices take about 10x as long to execute JS. In addition there is less memory for JS to use. I'm hesitant to add more opt out functionality by default that will have a negative performance hit. Maybe that won't be the case in a couple years. If so, lets connect on the performance issues then.

Recommendations:

  • Being smart about loading JS is a great idea. This is going to depend on use case, technology, and when you do it. We should make this system pluggable. Oh, there's already an issue for that in #352951: Make JS & CSS Preprocessing Pluggable. Lets make it happen.
  • I like modernizr. It can be very useful and we should teach people how to use it well. Someone should submit a session teaching it at DrupalCon Denver. But, to be smart Moderizr should be a custom build just for what someone needs in a site. Lets make using Modernizr easy, optimized, can be extended as what we look for changes, and opt in. Lets start by making the modernizr module useful and well used.
rupl’s picture

Totally in agreement, mfer. In fact, I recently signed on to co-maintain Modernizr, and have a couple fun projects in the works that will give it the love it needs. The community hasn't had the opportunity to generate real use patterns yet, which needs to happen before we decide on a use case for all of Drupal.

rupl’s picture

I have posted some additional thoughts over in the Modernizr issue queue - #1288248: [Meta] Develop a Modernizr API for other Drupal modules

gstout’s picture

+1

catch’s picture

Status: Active » Postponed

Let's at least postpone this on #352951: Make JS & CSS Preprocessing Pluggable.

There's so many options it'd be good to see things solidify and what goes on with async support too so I agree with mfer we may not want this in core after all (although we should look at the hacks that LABjs module had to do to drupal.js and similar to make it simpler).

ericduran’s picture

sub.

klonos’s picture

crashtest_’s picture

Sub

matthewv789’s picture

As with LabJS, there is already a Drupal module (6 and 7) for HeadJS:
http://drupal.org/project/headjs

Jacine’s picture

Here's a very detailed comparison of over 20 script loader libraries, including those named in this thread for reference:

https://docs.google.com/spreadsheet/ccc?key=0Aqln2akPWiMIdERkY3J2OXdOUVJ...

RobLoach’s picture

Issue tags: +JavaScript

nod_ brought this up on IRC, and I think we definitely need to push this forward. Having our JavaScript libraries load asynchronously will make our page loading/rendering performance extremely fast. RequireJS is another option. Whichever library we end up using will be infinitely better than what we bake ourselves.

Wim Leers’s picture

Yes. We need this for proper static analysis, and thus to allow for more efficient, fine-grained performance enhancements: it will enable smart bundling/aggregating of JS files.

At Facebook, they are currently overhauling all JavaScript to use CommonJS in favor of their own internal JS dependency/packaging system (which, in a nutshell, boiled down to a @requires foo bar comment at the top of the file to make it dependent on the foo and bar packages).

alexweber’s picture

Sounds good!

We should run some benchmarks and feature/stability comparisons to decide which one we should gun for and just do it!

I've had tons of problems implementing HeadJS and keeping it playing nice with contribs. Having this in core would definitely make things awesome! :)

mikeytown2’s picture

In regards to CommonJS, is this the async Loader Code? https://github.com/amdjs/amdjs-api/wiki/AMD Will it also load CSS async? If not we should look as being able to load JS and CSS async; gives us a lot more options for speed improvements when things like Embedded CSS are taken into account.

mfer’s picture

Let me start by saying I'm asking questions to drive out a solution good for a number of use cases. For example, we have the case of a site that someone interacts with and spends a bit of time on the site. Then we have the case of a publishing platform where someone visits a single page or a few pages of the same type. In the former cases you may have a number of cases different scripts used on different pages. In the latter case you have one set of scripts sent to a user. Optimizing for each of these cases is different.

An issue with performance (especially on mobile networks and devices – which will overtake desktop usage by the time D8 is popular) has to do with the number of files included, fetching them, and dealing with that. Sure, there will be improvements from what we have now but some of what we have now will still be used.

So, if we async individually load all the JS files on a page we will open up a bunch of network connections. The best browsers open up 6 concurrent connections to a single domain at a time (across all tabs and windows). This is 3 times the HTTP 1.1 spec. On each of these connections the current tcp spec has 3 packets transmitted before waiting for a response (though Google is working to change this). If we move from loading 2 or 3 aggregated JS files to a bunch (I've seen 15 or 20 on some sites) of individual files being loaded async the gains we get from async will be more than lost in additional network effects. This will really hit the site that has someone just reading a couple news articles or blog posts that have the same JS included.

So, when we talk about and architect a solution lets think about multiple use cases and the different parts involved in it.

Some other (random?) pieces of information to include in this conversation:

  • Newer browsers (Chrome, Safari, FF 3.6+, IE10) natively support async loading of JS via the async tag. See https://developer.mozilla.org/En/HTML/Element/Script for more details.
  • Let's not forget about the defer property on script tags. Even IE4+ supports this.
  • If there are 6 or fewer scripts to include in a page it can be better to not aggregate these into one script as better network utilization may make it faster. But, in Drupal is it common to have this case? Is there something we can do for this case knowing browsers and their networks?
  • Should we look at incorporating deferred execution as well? On an iphone 4 just executing jQuery to make it available for use on every page takes 320ms (You can learn more about execution at http://calendar.perfplanet.com/2011/lazy-evaluation-of-commonjs-modules/). This doesn't include the time to fetch it. Loaders like ControlJS try to push off execution until the time when the code is needed. This can have a nice impact on mobile.

This being said, I'm all for some good change here. I like CommonJS and RequireJS could be nicely used. If we go with something like RequireJS we should look at incorporating the optimization technique it can use for aggregation into our aggregate setup. Since we are not going to have a dependency on node/JS/Java on the server side we would need a PHP port of the aggregation (for now without the minification).

Thoughts? (@Wim, I'd appreciate anything you have for direction here as you are more up on this stuff than me.)

Wim Leers’s picture

I'm no expert in JS optimization, and definitely not in the JS "asynchronous modules" realm. I do know we can and should do better. Depending on what my job will entail, I may be able to work on this.

This answer probably belongs more in #1048316: [meta] CSS and Javascript aggregation improvement, but that issue has been silent for over 6 months.

Less bytes

To give an extreme example: on wimleers.com (which is on D7), I don't want *any* JS, except for Google Analytics. I don't need any JS, hence I don't want it. Yet, by using the Google Analytics module, all of drupal.js and all default jQuery stuff is loaded. Almost 100 KB, or ±65 KB gzipped. More than all HTML/CSS/images combined.
On the front page: "37.2% of CSS (estimated 24.5kB of 65.8kB) is not used".

My point is: we really need to move forward on this. Aggregation reduces the number of requests, which is good. But loading less data, and thus also needing less requests, is better. Of course, this is easier said than done in the case of Drupal, which must also work out-of-the-box, without deployment scripts and all that jazz.

Plugabble

Different use cases will alway require different solutions. Plus, this will keep evolving as browsers, webservers, standards and WPO techniques evolve. The single most important thing for Drupal 8 is to have all CSS/JS stuff pluggable, so that we can keep evolving in core, and not force outdated techniques upon Drupal users.

nod_’s picture

Status: Postponed » Active

I made a separate issue for the actual loading part #1423500: Use RequireJS to load all JS.

There is still lots to talk about here beside loading, let's get to it :)

mikeytown2’s picture

@mfer
AdvAgg's Bundler takes care of some of the concerns about aggregate size. AdvAgg also has hooks that make removing of JS files a fairly painless process.

AdvAgg makes most parts of it replaceable via variable function calls.

Wim Leers’s picture

I think cleaning up AdvAgg to make it core-worthy would be an interesting start. We'd definitely learn a lot from it, collectively, instead of just mikeytown2 :)

alexweber’s picture

Agreed, AdvAgg is amazing and standard on all our Drupal 6 sites and we're anxiously waiting on a D7 version! :)

mfer’s picture

As for minification, I'm working on that. Before DrupalCon I'll have something there and if my core conversation gets picked I'll be talking about it there. At the very least I'll be talking about that in my session Front End Performance Improvements. JS minification for Drupal has some GPL legal stuff surrounding it making it not as simple as just doing it.

RobLoach’s picture

Although this discussion is mainly client-side, I just wanted to point out that Assetic is probably the direction we want to go for handling all this.

effulgentsia’s picture

A lot of great discussion here, and I look forward to seeing and perhaps participating in some of the cool work that will come out of this discussion. In this comment, I just want to respond to Wim's #47:

I don't want *any* JS, except for Google Analytics. I don't need any JS, hence I don't want it. Yet, by using the Google Analytics module, all of drupal.js and all default jQuery stuff is loaded.

With hook_js_alter(), you can remove the JS files you don't want. Is there something about that that's not working for you, or are you just pointing out that D8 core should support this better without requiring an alter hook for fixing bad assumptions?

The single most important thing for Drupal 8 is to have all CSS/JS stuff pluggable

In D7, how the CSS is grouped, aggregated, and markup for it generated is pseudo-pluggable, since all of that is controlled via the rendering of a 'styles' render element, whose definition can be altered in hook_element_info_alter(). In D8, the JS is now also using the same pattern, with a 'scripts' render element. So, in the above statement, are you asking for something different than what we already have, or simply to retain pluggability as we add new features, like an async script loader?

nod_’s picture

I'd like to recenter a bit, there are three sides to this issue:

  1. How to aggregate all the js/css files used in the current page the answer belongs #1048316: [meta] CSS and Javascript aggregation improvement (use this or that module as basis to put in core, etc.)
  2. How to load said scripts and CSS into the page so that they can be used on the front end, i've started a little something here #1423500: Use RequireJS to load all JS,
  3. And the most important one for last: How are we going to use theses script?

First two issues will have patches but we should get "uses-cases" out of this one, like Wim explained about his website. There need to be a "decision framework" which will allow choices to be made for #1 and #2 or this could go off track.

You can see that for #2 it's actually pretty easy to have "script loader support in core" the hard part is knowing what to do with it. And I making the case that js shouldn't be managed in PHP that much, for example drupal_add_library() should go away and be managed as JS dependencies of AMD modules.

All in all, like effulgentsia said, D8 should support this better without having to use hooks :)

mikeytown2’s picture

@mfer
Here's my frontend performance presentation I gave in PDX: http://pnwdrupalsummit.org/sessions/front-end-performance (checkout the slides & notes in the slides). AdvAgg uses JSMin+ which is released under the GPL; allows for scripts to move from the header to the footer or to any other region of a theme; Aggregates are multi-process, each one is built in the background (code for this created HTTPRL).

@Rob Loach
assetic looks pretty cool! Thanks for the heads up, didn't know about it.

sun’s picture

Regarding 3):

Whatever script loader is being proposed, it needs to be able to cope with Drupal's #ajax framework, which means to dynamically lazy-load files and settings when needed, based on the previously existing state that performs a request, and even in the situation where all JS is aggregated into optimized groups, and even more so in the situation where individual files shall be cached.

Aside from that, I don't see a proposal à la "ditch PHP hooks/registries for JS" to fly for various reasons - it's essential for the Drupal backend to know which files it delivers, and also, that they can be altered, adjusted, or amended by modules. This information ties deeply into version control (hence, caching), but also contextual ESI injections.

nod_’s picture

"it needs to be able to cope with Drupal's #ajax framework", that's not much of a requirement, everything is done by ajax.js, so any loader able to load this file would work really :) the issue with requirejs has a working patch for it, it's NW only because I haven't fixed the tests I broke with my patch yet, otherwise it's working fine, try it out and break it so it can evolve towards a better solution :)

Concerning #3 I blamed the wrong function, I meant to get rid of drupal_get_library(), my apologies. I totally agree with you that drupal_add_library() is necessary, I mean it's from there that we can have paths to specific js/css files, so any loader would need this. I got a bit distracted while writing and got confused between the two of them.

I don't mean to get rid of hooks for js files, I'd just like to find a way where it's not necessary to use them to do simple tasks like not loading default drupal js files/settings. It's still fuzzy even for me, the only thing I'm pretty sure of is that js needs to be treated differently than PHP so the usual "Drupal way" is not necessarily the right one. I just want to make sure we have a go at another different approach more fitted to js.

mikeytown2’s picture

AdvAgg has a way to export the JS and CSS files that are loaded on the page: advagg_get_js_css_get_array(). Patch for QuickTabs is where the code came from #1172010-6: Quicktabs, AdvAgg - Loaded via AJAX do not pull advagg bundles which is my attempt at the #ajax issue.

Wim Leers’s picture

Also see #1279226: jQuery and Drupal JavaScript libraries and settings are output even when no JS is added to the page for why we need a JS dependency system.

#54: see #1279226: jQuery and Drupal JavaScript libraries and settings are output even when no JS is added to the page: this should work correctly out-of-the-box, without any altering. JS that's not used at all should not be loaded, period.

alexweber’s picture

Agree with Wim.

Despite jQuery being included in Drupal core and being essential to many administration aspects, it is not always used in the front-end (edge case IMO).

drupal_add_js() is also "dumb" in that it doesn't take dependencies into account or prevent the same script being added multiple times.

The fact that JS is alterable in D7/8 is a huge step forwards but we would really take this implementation further and if not add a script loader in core (which one is another debate), at least implement some form of dependency system to make implementing custom script loaders less of an uphill battle.

nod_’s picture

RobLoach’s picture

#1423500: Use RequireJS to load all JS gets the benefit of the package-based dependency tree.

alexweber’s picture

@nod_ and @Rob Loach, with the new RequireJS thread, can this be marked as a duplicate yet?

nod_’s picture

not really, I made another issue on purpose. Read up #55 to see what kind of things we have to agree on that have nothing to do in the RequireJS issue.

JohnAlbin’s picture

Trimming some redundant tags.

Wim Leers’s picture

xjm’s picture

Issue tags: +needs profiling
mikeytown2’s picture

Just a heads up. AdvAgg uses render arrays for JS (like CSS). It has these 3 patches included in some form #865536: drupal_add_js() is missing the 'browsers' option, #1140356: Add async, onload property to script tags, & #1664602: Allow attributes to be passed to drupal_add_[css|js] (SRI).

Long story short we can sorta testout the solution in D7. As an example, this D8 only module works if AdvAgg is installed (and the info file is hacked) http://drupal.org/project/async_script_shim

freblasty’s picture

subscribe

Wim Leers’s picture

#1279226: jQuery and Drupal JavaScript libraries and settings are output even when no JS is added to the page landed a long time ago in Drupal 8. Anonymous visitors on a default Drupal 8 installation won't get any JS on the front page anymore.

#352951: Make JS & CSS Preprocessing Pluggable landed today. Its change notice is at https://drupal.org/node/2034675 and specifically lists this:

override the JS collection renderer to use the http://labjs.com/ script loader

LabJS does not belong in core. Browsers evolve very quickly. HTTP/2.0 (aka SPDY) is coming. Once we have SPDY, having CSS or JS be aggregated is worse for performance! Today, LabJS is still very useful though:

<script async> and <script defer> are *almost* universally available, we just need IE8 and IE9 to die now: http://caniuse.com/script-async, http://caniuse.com/script-defer. SPDY support is going to be in the majority of browsers soon: last week it was announced that IE11 will support SPDY — http://caniuse.com/spdy.
Considering Drupal 8 is going to become mainstream somewhere in 2014/2015, it seems prudent to stick with what we have rather than trying to optimize for the current situation.

Therefor, I propose that we close this issue. It's been inactive for many months. #352951: Make JS & CSS Preprocessing Pluggable has enabled us to build this in D8 contrib very easily. That is in fact what we still need to do: validate that it is indeed possible to write this in Drupal 8 contrib.

droplet’s picture

Unfortunately, SPDY only available for HTTPS.

jcisio’s picture

Version: 8.x-dev » 9.x-dev

I agree that it's too late for D8. About not in core: we added many tiny JS tools in core, why not a 2 KB script loader?

But:

Once we have SPDY, having CSS or JS be aggregated is worse for performance.

Anything to back it up? SPDY vs http or SPDY vs https? Which version of SPDY? Also with mobile, SPDY = at least 3x RTT = more than 1s just for SSL negotiation.

Wim Leers’s picture

#74:

About not in core: we added many tiny JS tools in core, why not a 2 KB script loader?

Maybe because unlike many other additions, this impacts almost *everything* else?
Furthermore, I already explained that it could hurt performance in the future rather than help it.

Anything to back it up? SPDY vs http or SPDY vs https? Which version of SPDY? Also with mobile, SPDY = at least 3x RTT = more than 1s just for SSL negotiation.

Are you kidding me? 1) multiplexing (this is why many small files will be better than few huge aggregates), 2) resource prioritization. *Any* decent article on SPDY will explain this, so forgive me if I'm rather surprised you ask me to back that up. Reading the papers might also be useful.

jcisio’s picture

#75 Well, not really. Let me explain then ;) Ah, and there is an article to read before http://www.guypo.com/technical/not-as-spdy-as-you-thought/ (ok, I love SPDY, I'm using nginx with SPDY, but there are lots of work to be done).

0. SPDY (thus SSL, currently) adds 3x RTT for SSL negotiation. In a mobile world, it translates into more than 1000 ms delay before making any HTTP request.

1. Multiplexing: there are much more things in a web page (images, CSS) than just JS. With only 1 JS (even I didn't say just 1 JS), it is still able to multiplex.

2. Resource prioritization: browsers do it with HTTP already (https://insouciant.org/tech/resource-prioritization-in-chromium/), and usually it's better to do it browser-side rather than server-side because browsers know it better.

0-bis: Google engineers are trying (or tried) to ask OS developers to no avail to improve TCP slow start because it could help SPDY, but no help. As now SPDY uses only one (or two) TCP connection per domain, it's less "multiplexed" than e.g. using 4 TCP connections. Well, but I think it's a minor thing and could be changed.

Then here come script loaders (not all): resource prioritization. Without a script loader, how to you tell the browser load scripts as soon as possible, but execute them in a specific other (with parallelism), like: A1, A2, A3 in //, then B, then C1, C2 in //. I don't see how you can do it with script async/defer. And of course it is faster than executing A1, A2, A3, B, C1, C2 sequentially.

Furthermore, I already explained that it could hurt performance in the future rather than help it.

It is pluggable, so even if it were true, it's not a problem. The same thing for currently JS/CSS aggregation in core.

Wim Leers’s picture

#76: I think the fact that English is not your native tongue is significantly getting in the way of having a decent discussion.

The point I made in #72 was that we don't know how the web will evolve. The only thing we can mostly count on, is standards and pseudo standards. LabJS/script loaders are neither. Therefor it is risky to ship Drupal *core* with a script loader.
Conclusion: we should *not* ship Drupal core with a script loader, at least not Drupal 8. It should be trivial to make Drupal 8 use a script loader thanks to #352951: Make JS & CSS Preprocessing Pluggable, which means Drupal 8 contrib can cover that.

This is not about SPDY. SPDY is still an evolving proposal anyway. We don't know how it will evolve.

That is all.


If you agree, please mark this issue as postponed until D9 development starts. We need to focus on issues where we still can positively impact Drupal 8's performance impact.


Finally, @jciscio: would you like to port https://drupal.org/project/labjs to D8 now that you have #352951: Make JS & CSS Preprocessing Pluggable's asset.js.collection_renderer service that you can simply override? :) It'd be great to have validation that that is indeed possible with ease. If it is not, then we still have time to adjust the API if necessary!

jcisio’s picture

LABjs is quite stable: not a single bug since 2 years, even browsers are evolving. However, about LABjs in core: not in Drupal core BTW because:
- It will take 100 comments more to choose *which* JS loader to use.
- Script loader is an independant tool, it does not interact with other subsystem/JS tool, and it lives well enough in contrib. No one say "Google Analytics in core", even it has 300,000 installed and is used in virtually every website.

However, as catch opened this issue, I let him decide.

For porting LABjs to D8, I still have some other contrib to port to D8 first. But I'm quite confident that with the modular JS library work of nod_, it's much easier and LABjs will be much powerful.

Owen Barton’s picture

I agree with Wim's general assertion that things are changing - for example: AMD (discussed above) would affect JS loader selection and Drupal integration pretty significantly; a (exciting) potential longer term disruptor is QUIC
https://docs.google.com/document/d/1RNHkx_VvKWyWg6Lr8SZ-saqsQx7rFV-ev2jR... . The SPDY article above actually reinforces this point in my opinion - the 2 main reasons given for SPDY being slower are: "Too Many Domains", and "Blocking resources" - both of these are things (in the areas of infrastructure/embeds and browser/loader respectively) that will themselves be changing for performance reasons as SPDY gets deployed.

That said, I think making it as easy as possible to do "the kind of things" that we need to do to manage the load and execution order|dependencies|priority|parallelism of JS (heck, CSS too!) would be an excellent course of action, whatever the mechanism (loader scripts, async/defer, HTTP 3, HTML 6 etc). So I would suggest we close/postpone this one and open a new (meta-ish) issue that aims to exercise the wonderful plugable subsystem and identify any changes that would better enable this kind of function.

skyredwang’s picture

@Wim Leers said it right, The SDPY protocol supports prioritization. This prioritization can be taken advantage from browser/client level as well as content application level. In some cases, aggregation will neglect server prioritization. In some worse cases, aggregation will make the waiting time longer once the prioritization is introduced.

skyredwang’s picture

Issue summary: View changes

updated summary with 3 q's to resolve

mgifford’s picture

catch’s picture

Status: Active » Closed (won't fix)

Won't fixing this.

Version: 9.x-dev » 9.0.x-dev

The 9.0.x branch will open for development soon, and the placeholder 9.x branch should no longer be used. Only issues that require a new major version should be filed against 9.0.x (for example, removing deprecated code or updating dependency major versions). New developments and disruptive changes that are allowed in a minor version should be filed against 8.9.x, and significant new features will be moved to 9.1.x at committer discretion. For more information see the Allowed changes during the Drupal 8 and 9 release cycles and the Drupal 9.0.0 release plan.