JavaScript is typically used in two forms. There is the production and development version. The development version is used for development, troubleshooting, bug fixing, when you need to step through execution, etc. We should provide a toggle to allow switching between production/development JavaScript.

A simple implementation can be found in the speedy module.

This is a setup task to start shipping with minified JavaScript files in addition to the development version. The naming for production/development is taken because this is the naming convention used on jquery.com.

Crell, #18:

Note that we do now have a DrupalKernel class, extending Symfony's Kernel class, that has a dev/prod mechanism in it. We're just not using it at all yet, but it's there.

Comments

If we do this, it shouldn't be limited to just Javascript. It could easily apply to caching (page and block caching, or whatever that turns into in D8), theme caching, debug information etc.

The trick is figuring out how to do that without resulting in a global variable and 10,000 if statements throughout the system.

@crell - I think you are correct on this. Any suggestions?

Here is a possibility and I'm going to make 3 simple assumptions.

  1. This flag will be used by other systems and should be generalized. It's global and other systems will work out the implementation details. That means it can be a system wide variable.
  2. We need to describe a JS file as dev/prod/both. If statements are one way. A flag on drupal_add_js is another way. No matter which way we go we should default to both. You opt into describing it as being dev or prod. This is for the lazy develops to keep things working rather than exploding. I welcome challenges to this one.
  3. You might have some extra JS loaded just for development (think devel module) so it's not a one to one for every prod file there is a dev file and vice versa.

<?php
drupal_add_js
('foo.min.js', array('site_mode' => JS_PRODUCTION));
drupal_add_js('foo.js', array('site_mode' => JS_DEVELOPMENT));
?>

Thoughts? Refinements? Better ideas?

I do like the idea of passing this descriptive information a layer lower. If we make our JS handling pluggable it would be nice information for that system to know about.

I'm coming into the issue via http://engineeredweb.com/blog/why-hard-minify-on-the-fly/ so I haven't thought about what such a toggle should be limited to though it makes sense (though as you also say, having a huge number of if statements is going to be quite pain.

As such, I haven't thought the larger idea through but for the js (and I suppose for the css as well?), I can imagine doing something similar to how the css file detection for right-to-left exists (ie suffix of -min.js or -min.css? look at http://api.drupal.org/api/drupal/modules!locale!locale.module/function/l...) which would look for the optimized versions of those files and adds those if we're running in 'production' mode.

EDIT: I didn't see the nice post by @mfer above which sounds really nice.

@BTMash The difference between JS files when we deal with external libraries and drupal only css is huge. Not all libraries use .min.js for their shipped minified files. Should we ask Drupal devs to rename external libraries to our naming schemes? With Drupal css we create them and it's easier to set a naming convention for a feature.

I was referring specifically to drupal internal css/js. The flag that you have expands on the idea in that other modules are free to do work with the environment being dictated (so if a module is dealing with external js/css (or even internal) and it sees the production flag is up, it can add the necessary production version of the file(s) directly). And I was in agreement with your method - the way you have it (specify which type of js/css belongs in which environment) makes a lot of sense as well.

I agree with @Crell, but I think it's useful to be able to "set everything except X in production mode", i.e. don't use a single setting. But of course @Crell may have already implied that. I just wanted to make it explicit.

I didn't say that, but I can see the argument for it. :-)

mfer: It seems that there is a *.min.js standard at least de facto for minified JS files. Can't we just say "if in prod mode, look for a .min.js file and use that instead if found"? That seems like it would catch the majority case with no effort on the part of module devs.

Asking module devs to think about when certain JS files are appropriate is a losing proposition. We can't even get them to figure out head vs. footer for JS.

Also, drupal_add_js() needs to go away entirely. But that's another story. :-)

Can you elaborate on drupal_add_js please? I'm interested now.

@crell How about this. We create a way for drupal_add_js (or it's replacement) to be told what to include when and we try to automate the process with .min.js files. So, we can make it smart and help developers who want to work around the smart.

@_nod drupal_add_js/css need to go away to something with a better api and setup. That discussion is for other issues and people who want to carry that torch.

So, something like:

<?php
array(
 
'js' => 'foo.js',
 
'min' => 'foo.minified-version.js',
);
?>

Where the default for min if not specified is the value of the js key with ".min" inserted into it?

I could live with something like that.

That would be similar to Libraries API's registration of variants per library.

While I can see that working for pre-registered/predefined libraries, I don't think this will work with the current way of how independent JS/CSS files are added to a page.

FWIW, I'm not confident that we'll reach a point where each and every JS/CSS file will be registered/predefined as a library. That is, because there's still quite a difference between a frontend library delivering unique functionality/behavior/styling and arbitrary/specific frontend files provided by modules and themes. Doing so would also make theming significantly more complex.

Most themes I've seen don't have conditions in their CSS/JS usage. They just specify code in their .info file. Could we do the sort of logic discussed in #9/#10 in the info file, much as we do for media types for CSS? Or would that make the file too complex? (Or is that an indication we should start looking at using CMI-style XML instead of info in order to support more such use cases?)

I'm mostly just brainstorming here. In general I agree with having a "dev mode", but there's a lot to work out.

I suppose it would be like this:

<?php
$libraries
['keepitsimplestupid'] = array(
 
'js' => array('foo.js'),
 
'js-min' => array('foo.min.js'),
 
'css' => array(
   
'foo.css',
   
'foo-kungfoo.css',
  ),
 
'css-min' => array('foo.min.css'), // both foo.css and foo-kungfoo.css
);
?>

We simply would not support this for individual files being added. We should move the majority of those over to hook_libraries_info so we can eventually AMD them up.

Most themes I've seen don't have conditions in their CSS/JS usage. They just specify code in their .info file. Could we do the sort of logic discussed in #9/#10 in the info file, much as we do for media types for CSS?

The problem with declaring it in the .info file is that there is no context applied to when the file is added. This means that it's added to every page, whether or not it's needed. It's much better to use #attached or drupal_add_library when needed. This is something we'll be doing at #1541860: Reduce dependency on jQuery (use jQuery only when needed).

...coming from #1389058: Update html5shiv-printshiv to the latest (minified) version - currently 3.6.2 final and #1341792: Ship minified versions of jQuery and jQuery UI + other issues where it is debated which versions (minified vs non-minified) of libraries should be used.

I just want to point that there seems to be a common agreement that minifying on-the-fly is quite difficult at the moment for various reasons (postpone for D9 perhaps). One of the most important concerns with the current solutions is how to minify the actual code part of the files, but leave any license info comments intact. A fast and easy workaround is to include both minified and non-minified versions of the libs (so not to violate license) and come up with a way to switch between minified/full versions through the UI (à la the offline/online mode we currently have). The first part is only a matter of policy/decision and then doing the actual dual-file shipping with core. The second part will be handled in this issue here.

I guess what I'm trying to say is that implementing this feature request here has the pleasant side-effect of partially solving the license violation on minification issue I mentioned above as well. So +1 and I think that this should be mentioned in the issue's summary as an extra gain (besides the performance and dev/production gains I mean). If there's no objection, I'd like to add this info in the summary.

#12: in that case, I think it would make a lot of sense to let d.o's packaging scripts generate a minified version automatically for projects that don't have it already. We don't want to ask every single maintainer to manually minify all of their CSS/JS.

That being said, I *don't* think we should ship minified files at all. I think we should solve the minification in core issue. And it looks like we're getting very close to solving the licensing roadblock: #119441-160: Compress JS aggregation.

#15: There are a few problems with on-the-fly compression:

  1. It's expensive on the server
  2. All the PHP libraries out there won't get us as far as what Grunt and UglifyJS will give us
  3. It's already done for us in most cases. Most libraries out there already ship compressed versions of their assets (which is in most cases done by UglifyJS)

Taking that into account, compression is not the way we want to go for Drupal Core. It might be best left to contrib (Speedy, agrcache, etc). On-the-fly aggregation is a good though, as it means less http-requests and still is fast on the server.

This leads me to this question... What the difference is between a Production/Development Toggle for CSS/JS and the Aggregate JavaScript files/Aggregate and compress CSS files toggle in admin/config/development/performance?

Is this issue specific to JS, or does it encompass end-to-end? Environment and Mode being just a couple modules at my fingertips that use this concept to automate code & configuration change for deployment and testing.

Note that we do now have a DrupalKernel class, extending Symfony's Kernel class, that has a dev/prod mechanism in it. We're just not using it at all yet, but it's there.

I think there's a good use case for a system-wide dev/prod flag, provided we still have the option of doing more fine-grained control.

@Crell: Hey Larry, I've included your comment in the issue summary so it doesn't get lost if this issue becomes too big. Hope you don't mind.

@Grayside I took a look at the the Environment and Mode module. They appear to to solve a slightly different problem than what I'm proposing here. If I'm not right on this please correct me.

The environment module lets you specify differences in Drupal config per environment. This could be a production environment and a development environment. But, what if you have the case where you want to log some extra bits in development but not in production? It appears to be about switching from one environment to another automatically. For example, to disable the develop module.

The Mode module appears to operate around permissions which is a different space.

So, while I agree these modules can be useful I think they are solving a different problem.

A general switch to turn on / off debugging would be really helpful also for Twig. ( not only restricted to JS)

> The environment module lets you specify differences in Drupal config per environment. This could be a production environment and a development environment. But, what if you have the case where you want to log some extra bits in development but not in production? It appears to be about switching from one environment to another automatically. For example, to disable the develop module.

Environment defines a way to check what environment the site is

<?php
if (environment == development ) {
  
watchdog(..)
}
?>

Our use of environment modules involves having a three state -- production, development, readonly. However, the readonly is problematic as being part of those 3 -- read only mode should really be useable during production (e.g. there should be two separate sets: production, development; read, read only).

However, there is a case for a multiple state, I think: production, staging, development (, testing), so it'd be nice if core out of box supported this (even if it only came with 2 main states to begin with). Not sure if whatever system core has would find it useful to define

How I'd imagine it working, based on how used environment module:
Core would default to development mode to begin with, variables default to
Overriding configuration have a way to specific what mode/s they should apply to
Modules/etc. can react to a change in mode (if they need to enable/disable modules -- e.g. enable devel during development)
Modules/etc. can define new states

I'd also like to see this used in the usage statistics. 5000 sites use my module, great, but how many of those are dev sites? How many are actually in production? I'd like to know.

I don't know that is relevant, actually. The goal here is to have explicitly separate "I'm working on things" and "I care about performance" modes. There's lots of things we could build off of that. The number of people that are using workarounds now to get part way to that doesn't change that there's a lot more we could do if we had something like that baked in.

Just as services module is only used by a small number of sites, and deploy by even fewer, but we're rewriting our whole routing system to support web services anyway because it's the right thing to do, architecturally. :-)

Component:javascript» base system

Issue tags:+Twig

Adding Twig tag. I really *need* this.

The dev/production environment switch in Kernel doesn't seem to be documented much to what it does exactly (Didn't look too much though). It's a string, so not sure if it's restricted to production/development (prod/dev?)

It's set via the construction, for example index.php has

<?php
$kernel
= new DrupalKernel('prod', FALSE, NULL, cache('bootstrap'));
?>

and gotten via getEnvironment() function. It's current use is for determining the container class/building the container class.

It seems like having a separate issue for minification might be best where it's added via a configurable toggle like css/js/etc. aggregation, and not intertwine it with the issue of adding production/development toggle (other than the default, like with aggregation, is somehow based on environment).

Interesting thought is that production could remove the UI for changing variables like aggregation/minification.

The way Symfony fullstack uses it is to offer 3 environments: dev, prod, and test. test is only used during integration testing. The others are handled via different front controllers. Vis, there's app.php which has prod hard coded, and app_dev.php that has dev hard-coded but is otherwise identical. Since Symfony's request handling is more robust than Drupal 7's was (and we're now using Symfony's in Drupal 8), that works fine and you can go into "Dev mode" by just tweaking your URL. That results in slightly different config files getting loaded, different containers, etc.

I don't know that we can or want to handle different environments via alternate front controller files, but allowing arbitrary environments (with direct support only for dev and prod) that have slightly different config (mostly for things like "should we recompile Twig files on every page request" or "should we aggregate CSS files") makes total sense.

#28: I agree, I envision this currently like:

Current Environment: [Dev]
Select Environment --> Dev, Test, Prod
[ ] Aggregate CSS
[X] Recompile Twig files on each request
[ Save ] [ Switch Environment ]

Or similar ...

Then you can switch configs for environments via AJAX or whatever and switch between environments easily.

It is a little like presets and quite powerful ...

The problem with a UI toggle is where to save it. It has to be to a system that is readable pre-kernel. Right now, we don't have one of those. All we have is directly reading a YAML file or the PHP Storage system; The former is too slow, the latter is not guaranteed to exist. Everything else is only usable post-kernel (CMI, State, etc.).

A manual $conf variable in settings.php or separate front controllers seem like the only practical options.

I am happy with a $conf variable in settings.php to switch between different configs for certain admin screens.

Installer could ask if you want to enable development mode and write settings.php accordingly?

Speaking from experience, UI for environment toggling is buggy. If lots of operations are tied to it, it needs special memory handling. And yes, can have odd race conditions with different subsystems. drush integration can be nice though.

I'd like to second #27:

It seems like having a separate issue for minification might be best where it's added via a configurable toggle like css/js/etc. aggregation, and not intertwine it with the issue of adding production/development toggle (other than the default, like with aggregation, is somehow based on environment).

Items leaning on this environment toggle should have their own state, environment should be a meta-state. That way if developers need to take surgical actions, like test one "dev" component of the system in an otherwise production environment, it is possible.

Looking into writing to settings.php

Fully possible to write it during install.php. However, only form/steps that seems relavent to add to is the database configuration one, but that seems a bit misplaced. The last site configuration one seems a bit too late and it's after the "change settings.php permission back" message (that can be moved, but there's some comments above the message suggesting how it'd be bad if it was there).

Is there a system already in place for changing configuration like this? I believe there is, as domain and spaces type modules needed something like that (though this is different -- domain & spaces just need temporary overrides, this needs more permanent defaults), but don't know the details. If someone can point me in right direction, I'd like to try and implement it.

My guess is any IU outside of perhaps a settings.php toggle (or maybe just not have that and just have a settings.php variable) could be handled by contrib -- considering the timing, etc.

Another thing to note: currently things like css/js aggregation is turned off by default, but the current enviroment as set in index.php is production (which would likely have css/js aggregation on).

The problem with a settings.php variable is that it is then impossible to change mode without code/shell access, to a file that many people check into Git. A listener can't do it, because it happens after the container is active, but the container is one of the things we want to switch out.

For APC integration, why wouldn't you just detect if APC was available, then automatically use it? There's no need for a toggle at all. A checkbox for "would you like a slow site or a fast site?" is silly with regards to APC. It has absolutely no other affect on your site (unlike JS/CSS aggregation) other than making things go faster. If you're a developer and turn on xdebug, APC is automatically disabled. Or if you're running XHProf, combining with APC is totally allowed. Or systems like MAMP or WAMP allow toggling of APC in their UIs. There's no real question about it. If the server has APC, you use it. If you don't want Drupal to use APC, you turn off APC.

quicksketch: Restarting apache to clear your cache is bad DX. During development, I want nothing cached. NOTHING.

Developing with APC on is not bad. It's actually faster than not doing so. Don't force me to turn it off because Drupal can't not-cache something. (We're broken enough as is on that front.)

I haven't followed all of this closely or pondered all the implications, and I'm *way* out of the loop on D8 development. However, when I was re-writing the update manager to be included in D7 core, all that authorize.php plumbing that it uses to install new modules as the user that owns the files, not the user that the webserver is running as, was designed with the idea that it could be reused for stuff like the installer and/or something like this. In theory, we *could* write this toggle directly to settings.php, even without shell/code access, even from the UI. Not sure that's a good idea, but it's something to consider.

Developing with APC on is not bad. It's actually faster than not doing so. Don't force me to turn it off because Drupal can't not-cache something. (We're broken enough as is on that front.)

I mentioned this over in #1971198: [policy] Drupal and PSR-0/PSR-4 Class Loading, but if you wanted to turn off APC caching, that would be a logical setting for settings.php. Turn APC caching on by default if it's available, and allow it to manually be turned off in settings.php.

quicksketch: Restarting apache to clear your cache is bad DX. During development, I want nothing cached. NOTHING.

Well in that case you wouldn't be running with APC on in the first place. Not all developers share this sentiment. There are many, many "developers" that only build sites through the UI: Views, Panels, Rules, etc. For them, any additional time caused by not having caching on code they're not touching is a waste.

I was referring specifically to developers as "people who write codez", not to site builders. I thought that was rather obvious from context.

I am generally against a toggle. Which imply tests hock in the code. E.g. if (prod).

This should be avoided, instead using multisites capabilities such as a staging, dev and prod. They may share common modules but different configuration. Hence no need for a test hock.

Although I enjoy the idea of having One Toggle To Rule Them All, it is much easier to debug issues by having the control to enable/disable the individual toggles selectively. Maybe if we kept what is there currently, but had the Development/Production toggle just wizardize the Performance page using States...

  • [ ] Development
  • [x] Production
  • [ ] Custom

Without being aware of this proposal here, I came up with a very similar idea in #1925822-13: Stop locking me out of my own sites/default directory if Drupal is already insecure, some of the aspects might be interesting for this.

#41, what if you need more than Development, Production, and Custom ?

Such as Staging, Integration, Quality ?

Will you need tests hock in code (such as if (dev) {} blocks) with this solution ?

Why do people does not believe in the sites/ approach where you can have arbitrary any environments you need:

sites/all
sites/production
sites/integration
sites/qualification
sites/staging
sites/local

Where in sites all you have all the common modules to your application, and in each subsites, the configuration (through settings.php, features, code, ...) and modules which overrides default one for this specific site.

For instance, in the production folder, I would have an apc module, which is not in the local folder. I would have devel, coder module in the local folder but not in the staging folder. Additionally, I would put a secure module in qualification, integration and production, with different parameters dispatched respectively in settings.php files and/or features specific to these environments.

This would also mean, that the DI Container would be compiled and dumped on a site folder basis, hence they will be different between environments. To eventually optimize the solution, we could remove the logic which appear early in the bootstrap process (DrupalKernel) which is responsible for loading a class container based on $debug and $prod parameter and rely on the php file generated on a per site basis.

Issue summary:View changes

...adding Crell's comment in #18 about DrupalKernel and the dev/prod mechanism in Symfony's Kernel class

Issue summary:View changes

Note that #2226761: Make Drupal 8 fast by default: DEV/PROD toggle in settings.php, default to PROD contains a concrete proposal for a first baby step.