Adhere to licensing issues when minifying files

Related:
#1337022: [policy, no patch] Create/adopt JavaScript docs standards compatible with a JS documentation parser

@janusman While that would be great for our code, how do we handle libraries we drop into Drupal? Libraries we've included like jquery.ba-bbq.js (in core), jQuery UI (also in core), jQuery Cycle (commonly used in modules), and so many other JavaScript bits do not follow JSDoc.

If we start minifying all the JavaScript we need to cleanly handle all the cases of external libraries.

not lawyer and all that, but I don't think we have to worry about license text within the JS when it is minified. I think we would be fine if we kept a separate copy in a sources directory that is unminified. It would never be used, but it would solve our distribution issues since it would be distributed, with the license, with the rest of the source code. While it may increase our download size it realistically wouldn't be that much compared to the rest of the code.

We could also keep the sources directory as a separate download. Would complicate things a little bit more, but it would still solve the licensing issues.

here is FSF solution to the issue http://www.gnu.org/licenses/javascript-labels.html They feel that this suffices most licenses including the mit case.

We could add this to the powered by footer and then if a site removes it it's on them.

@redndahead I've found 2 problems (in the engineering sense) I'm not sure how to deal with.

1. Shipping a JavaScript file to the browser to execute there is distribution. Not all lawyers who work in this space agree with the FSF on how to handle this. Since it's not tested and there is no precedence here (that I'm aware of) we need to tread lightly and reasonably.
2. Now, the more difficult one. Not all the JS we send to the browser is GPLv2. For example, say I use twitter bootstrap and distribute that. It's under the Apache 2 license. A table on a site would need to properly identify each file in each contrib module and link to the proper license. How to we handle that cleanly?

Personally, I'm quite annoyed at all of this. I wish it were simple and easy. I sometimes miss the days when I was naive of all this performance and licensing stuff. Trying to find a solution to these problems is hard (in the non-technical sense). I like hard problems... but not this one.

@mfer whoops I followed a link to this and completely missed this was in the speedy queue. I'm thinking about core where things can be more defined.

That said. Apache 2 doesn't apply since we require people to manually download those libraries and it's up to those site owners to comply with the license.

So now onto contrib modules themselves since they are not functional programs by themselves and they are not distributing the javascript file they wouldn't have to provide links.

Distributions/install profiles: This is where the problems lie. We need a way to build these license texts from the make file. This could technically be added as an option to drush make. We could add a link on the powered by footer to a license file that's built when the distribution is built. Something like third-party-licenses.html

As far as not all lawyers agreeing with FSF that is fine, but I think we can reasonably say that FSF knows what they are talking about and if someone complains we can deal with it then. I think we as a project can sleep at night knowing we followed the FSF recommendations.

@redndahead I think you missed what I meant by Apache 2 license. I meant the Apache License version 2 which bootstrap and many other useful libraries are under. I was not referring to the web server software.

If a module (contrib or custom) includes a JavaScript file that JavaScript is distributed from the web server to the browser. The distribution we are talking about isn't free software packages (like Drupal) being distributed to developers to use or for one click installs. The issue of distribution is of JavaScript files being distributed from web servers to web browsers when an end user requests a page. This is where things get complicated.

Take a look at important comments. Minifiers like YUICompressor support them. Basically, JavaScript or CSS comments like /*! ... [content]... */ are not removed. This is a great way to preserve licenses.

http://developer.yahoo.com/yui/compressor/css.html

@mfer I was speaking about the license.

What I'm trying to get at is in d.o. land we do not have to worry about how files are distributed on a users website. That is not our responsibility. That's the responsibility of the website owner. That said I think we have a responsibility to do as much as we can with the software that we distribute from d.o. This includes installation profiles (with drupal core being one of them). We do not distribute apache 2 licensed code from d.o. So we don't have to worry about that. The owners of the site may have to, but d.o. doesn't.

Now should we provide tools so it makes it easier for site owners to comply? Maybe. I think it would be useful to either have an additional hook or an option in the .info file to incorporate this license information. Then a simple page is created through this method that is in a simple format like the FSF has specified.

That's about as far as I think we should go. I don't think we need to get into the business of making sure every site owner complies with the licenses because anything can happen once it leaves d.o.

Surely the same problem occurs when open source code is compiled before distribution?

What about adding something like this to the top of all minified JavaScript files:
"This JavaScript source has been minified. For the human-readable version, including licencing
information, please see http://www.example.com/files/js/my_script.js"

I think you'd need a very strict licence (or interpretation thereof) to say that the licence had to be in the same logical file as the source code, rather than just distributed with. Obviously this would mean the originals being hosted with the drupal site, but I don't think that's a big concern.

It is in fact sufficient to provide a link to unminified (source) versions of the distributed files, but we are free to distribute minified javascript (binaries). At least that was my understanding of open source licencing in general.

@Billy Hoffman YUI, Closure Compiler, and UglifyJS all handle license comments differently. Libraries in the wild use a number of methods including ones not supported by any of these for their licenses. We need a practical solution and asking site owners to make sure modules they use are including JavaScript that follows a certain pattern isn't practical. Most site owners just install a module and go. They are site builders and not developers. We need a solution that works for them without putting them in a place where they will violate licenses. It's the ethical thing to do.

@redndahead Our ethical responsibility is to produce stuff that won't likely cause the users to violate licenses. Most of the people who use these modules are site builders and not software developers. They often don't know about license issues of distributing JS from a server to the browser, how to modify modules (hacking), or any of that. We need simple solutions they can use that won't cause a majority of them to violate licenses unknowingly.

@thelem When you install a compiled version of LibreOffice, FireFox, or something else is the license shipped with it? The answer is yes. There is also a difference between a compiled script and a minified one. A minified script is not the compiled version. A closer representation might be obfuscation. Not to intentionally make it hard for people to read the code but for performance.

While I'm not trying to hold to a strict interpretation I'm also not trying to go after something loose.

@Robin Millette Minified code it's compiled from one language to another. After the process it is still the same language. We aren't distributing binaries. Instead we are distributing files that more closely represent an obfuscated version. It might be worth reading the minified entry on wikipedia.

While it wasn't a highly trafficked issue on Stack Overflow... even my lazy searching there found an answer - http://stackoverflow.com/a/3248962

What about adding something like this to the top of all minified JavaScript files:
"This JavaScript source has been minified. For the human-readable version, including licencing
information, please see http://www.example.com/files/js/my_script.js"

I like this solution, but it's probably not good enough for the legal stuff.
As for a regex, couldn't we preserve the first comment and/or comments that have "license" in them?

@FreekyMage Not all license comments are in the first comment. For example, jQuery BBQ has two license comments. The second one is halfway through the file. I use this as an example as I've seen other cases of this.

That's why I was thinking of leaving all comments with "license" in them. Surely we can contact those libraries that don't work after this and persuade them to fix this. After all, we aren't the only ones that will minify their code and other people might not be so friendly to notify them about their license getting ignored by things as uglifyjs.

@FreekyMage This isn't an issue of minifying code. It's a matter of how we are talking about minifying code.

Speedy currently walks through core and minifies JavaScript not on a blacklist. Already minified files (e.g., jQuery, jQuery UI) are blacklisted so they aren't processed twice. Extending this step to contrib is where it gets complicated. For any site you could have any number of permutations to deal with. Handing core is easy because we know what will be in it. Handling contrib in a fully automated for all cases manner is more difficult.

For many of these libraries you can download a minifed version with proper licensing. When you pass it through an automated step to minify all JavaScript these would get removed.

On Drupal sites we can drop in a lot of different libraries. Potentially thousands of them. Many of these (and we don't know all of them) have multiple comments for licensing. This isn't a bad thing either. Is asking all potential libraries to change so we can use them in a build step something we could politely pull off? I don't think we have the political capital on the web to pull something like that off.

Now, if we minified using build scripts for a specific site (knowing the JS being used and caring for it) or minified by hand (yikes) this isn't a problem. It's finding one implementation to deal with all sites that's hard.

Fixed typo.

Please see #2258313: Add license information to aggregated assets, it adds license metadata to asset libraries and exposes that information according to the JavaScript Web License Labels standard.

With regards regards to the Speedy module, I found that using uglify-save-license has been effective for maintaining license comments without having to handle it in PHP.

I do not have a patch/example for using this with speedy at present, but I have had it working. The rules it uses are pretty simple, per the description on it's NPM project page:

uglify-save-license checks each comment token of a JavaScript file. The comment will be regarded as a license statement and preserved after compression, if it meets at least one of the following requirements:

The comment is in the first line of a file.
The regexp for license statement matches the string of the comment. It matches, for example, MIT and Copyright.
There is a comment at the previous line, and it matches 1. 2. or 3.

Given that Speedy is a contrib module, and other methods mentioned may take a while to be available. Perhaps this would be a worthwhile implementation in Speedy?

duplicate of #2258313: Add license information to aggregated assets