Based on the dozen or so (it feels like) namespace organization threads floating around, I would like to offer the following over-arching proposal for Drupal namespace and file organization.

Executive Summary

  • /core/includes/ - Legacy procedural code.
  • /core/lib/Drupal/Component/ - Non-Drupal-specific libs we write.
  • /core/lib/Drupal/Core/ - Drupal-centric subsystems.
  • /core/vendor/ - 3rd party libraries like Symfony2 components.
  • /core/test/ - Test classes for stuff in the Core and Component namespaces.
  • sites/all/modules/$modulename/lib/ - classes in the Drupal\Module\$modulename namespace.

Namespace structure

Note: For the purposes of this explanation, one code block "depends" on another if it will generate a fatal error and not execute should another component not be available in memory or available to load into memory by the class loaded when needed.

Also note that if a system ships with optional, swappable components, dependency for those need not be met. That is, the existence of a database cache backend does not mean that the cache system as a whole depends on the database component.

Namespaces should be structured thusly:

Drupal - "Vendor" namespace. Everything Drupal-originated, including contrib, is in this namespace. No classes may live directly in this namespace, however. All classes must fall into one of the following categories:

Drupal\Component\$component - Drupal Components are Drupal-originated free-standing systems. They are code written for and by Drupal that is generally useful, not just to the Drupal project but to PHP projects in general. The intent is that we can split these off into free-standing repositories if we want to. Why would we do something like that? Several reasons.

  1. It encourages more loose coupling. Loose coupling makes for better code. Better code makes for a better Drupal.
  2. Marketing. If other developers can see awesome stand-alone tools that we build, and use a few of them, that's a "gateway drug" to turning them into full fledged Drupal developers.
  3. Less code in common.inc and bootstrap.inc.

A Drupal Component may NOT depend on any other code that is not also another Drupal Component, and depending on more than one other Component is discouraged. Components may not create a circular dependency. That is, a Component may not depend on being inside a Drupal install at all. Good examples here include the graph library, a locking mechanism, and most of the database layer. Components may have sub-namespaces if it makes sense to do so.

A Drupal Component MAY be a collection of related, self-contained stateless utilities. If so, then it is permissible for the component to include only static methods that may be called by other Components. However, an instantiatable object is preferred, and is required if there is any shared state to consider. Examples here include drupal_array_get_nested_value(), drupal_array_set_nested_value(), and drupal_array_nested_key_exists(), all of which could simply be static methods on a NestedArray class.

Drupal\Core\$subsystem - A Drupal-specific subsystem. Examples here include the Configuration system (CMI, once that gets built), Plugins, FileTransfer, Translation, the Drupal-specific bits of the database system, the Block/layout/routing system (WSCCI), the theme system, and modules system. A subsystem may depend on another subsystem or a component, but should try to limit the number of dependencies. Subsystems may not create a circular dependency. Subsystems may have sub-namespaces if it makes sense to do so.

When developing a new piece of code, building it such that it can be a Component rather than a subsystem is encouraged so that we can leverage all of the benefits of Components above.

Drupal\Module\$modulename - Classes that are provided by a module named $modulename. $modulename is the direct machine name of the module, with no case folding. E.g., "node", "views", "og", "entity_view_modes", etc. A module may have sub-namespaces if it makes sense to do so. Most classes in a module's namespace are up to it to define in whatever fashion it chooses; however, core may make certain documented class names magical for early bootstrap purposes. Modules are strongly discouraged from doing so, however.

A module may depend on any Drupal Subsystem or Component, and may depend on any other module provided it is declared in the module's info file. (That should be self-evident, but I include it here for completeness.)

Drupal\Module\$modulename - All test classes provided by a module go into the same namespace as the class they are testing, one class per file. They will *not*, however, be in the same directory. (See below.) Supporting classes that are not themselves active tests (eg, if several test classes extend from a common base test class) are permitted. Non-test-related code is not allowed in this namespace. As an exception to the dependency rule, modules that provide tests do not need to declare a dependency for them. (Again, self-evident but I include it for completeness.)

Drupal\Hook and Drupal\Module\$modulename\Hook - Reserved if we end up putting hooks in a namespace. Right now we're not, but if we do, they could go in one of these places. (Note: BOTH are reserved. We can decide which if either we want to use later, in a separate discussion. Please don't debate the merits of those locations for now.)

Disk organization of classes

/core/lib/Drupal - All core-provided classes under the Drupal namespace go here. That includes Components and Subsystems.

/core/vendor/ - All 3rd party libraries that ship with core live under this directory. At present the only one we have is selected Symfony2 Components, so those would live under /core/includes/vendor/Symfony2. (Yes, this means we'd move them from where they are now. There are no patches pending against them, of course, so no biggie.) Code in this directory MAY be under a non-GPL license, as long as it is in-bound compatible with the GPLv2. (MIT, BSD, and LGPL, mostly. Apache 2 is not allowed, as Apache 2 is only compatible with GPLv3.) All code anywhere else is GPLv2+.

The vendor/ prefix is not part of PSR-0, but is a de facto standard used by a number of projects, demos, and tutorials to indicate "3rd party stuff". Its use is mostly for consistency with trends in the PHP world at large, which in turn increases familiarity for incoming PHP developers.

$moduledir/lib - This directory, if it exists in a module, is mapped to Drupal\Module\$modulename automatically by the core system. No additional action by a module developer is required.

$moduledir/tests - This directory, if it exists in a module, is mapped to Drupal\Module\$modulename automatically by the core system. Test cases and supporting code should go here, mirroring the namespace tree of the code being tested. No additional action by a module developer is required.

$moduledir/+ - A module may specify additional namespaces to register via its info file. These may map to any directory within the module except the $moduledir itself, $moduledir/lib, and $moduledir/tests.

sites/[all|domain]/vendor - This is a special case of the "libraries" directory. It is reserved specifically for 3rd party PSR-0-compatible classes that a module may depend on. Ideally, these will be pulled in using composer.json (a PSR-0-based tool for managing independent but related libraries). Ideally, a module that requires a 3rd party PSR-0 library could specify that dependency in its info file or similar, or even just in README documentation, and require no additional PHP code or libraries.module to make it work. (Arguably this could also be at sites/all/libraries/vendor. I'm fine with that, too.)

Class loader configuration process

Mark Sonnabaum has pointed out that the Symfony2 Class Loader can, in fact, have new registrations added to it after it is activated. That is a very useful tool. It also has "fallback" namespaces, essentially registering namespace locations that are checked only if the others fail first. Based on that information, and the patch posted here and later comments, I would recommend the following:

  1. As early as humanly possible, we initialize the Universal class loader. We may even want to build in a super-basic hack to switch to the Apc class loader even before the config system proper initializes. We want it that early.
  2. We register all namespaces in /core/vendor as normal namespaces.
  3. We register the \Drupal namespace at /core/Drupal as a fallback namespace.
  4. We store the class loader object in a service container (or, for now, a collector function, as in Mark's patch).
  5. When the conf path system initializes, we register any sites/domain]/vendor namespaces needed. This may simply be through composer. TBD.
  6. When the module system initializes, iterate over all modules and their paths and register all Drupal\Module\$modulename namespaces.
  7. If a module specifies any additional namespaces to register, those will be registered as well. Because these are registered "normal", it means a module may override a core-provided class.

This overall approach provides the following benefits:

  1. It provides for autoloading of nearly all classes in the system; pretty much everything other than the class loader itself, and maybe Pimple if we use that, are lazy-loaded.
  2. It encourages decoupled design, which is generally good practice.
  3. Because we can see via "use" statements what Components and subsystems depend on what, we can easily build a dependency graph of our various moving parts. That means we can actually get a sense for what Drupal's architecture, you know, is, how it fits together, and thereby how we can improve it without guess-and-check.
  4. In the typical case, module-based classes just automatically work. No extra work (like the files[] array in Drupal 7) is required.
  5. No more registry. Yay!
  6. 3rd party libraries become easier to leverage.
  7. Modules can override core classes(!), provided those classes have not already been loaded yet. That allows for some degree of backporting, provided interfaces don't change, although it is not a panacea of a solution.

So what does this mean for module developers???

A typical module might then look something like this:

lib/Drupal/Module/kitten/Nermal.php
lib/Drupal/Module/kitten/Kitten.php
lib/Drupal/Module/kitten/FelineInterface.php
kitten.module
kitten.info

// kitten.info

name = Kitten
description = Provides Drupal with kittens.
core = 8.x
dependencies[] = entity
; Note the lack of files[] array.  Yay!

// includes/KittenInterface.php

namespace Drupal*Module*kitten;
use Drupal*Module*entity*EntityInterface;
use Drupal*Core*Cuteness*PettableInterface;

interface KittenInterface extends EntityInterface, PettableInterface { … }

// includes/Kitten.php

namespace Drupal*Module*kitten;

class Kitten implements KittenInterface { … }

// includes/Nermal.php

namespace Drupal*Module*kitten;

class Nermal extends Kitten { … }

// kitten.module

use Drupal*Module*kitten*Kitten;

function kitten_form_alter($form, &$form_state, &$form_values) {
 $k = new Kitten();
 // ...
}

Note that really the only changes from the status quo are:

  1. No more files[] array.
  2. You have to put classes in the right place.
  3. You need to "use" classes in the .module file (which is already the case for anything namespaced).

My thanks to Mark Sonnabaum, Kris Vanderwater, Pierre Rineau, Nathanel Catchpole, Daniel Kudwein, and others for their feedback on this proposal.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

catch’s picture

One thing that's missing from here, currently nothing in /includes is able to provide their own tests, it would nice to be able to do Drupal/Cache/Tests rather than having that all handled by system module.

Also I'm not sure where something like an alternate cache or database driver would live. At the moment it more or less de-facto goes in sites/all/modules but that's not the place for it (memcache project ships with a couple of modules but most sites don't use or need them).

pounard’s picture

I was quite fond of moving out the vendor dir from includes, such as:

/core/includes/Drupal/Component
/core/includes/Drupal/Core
/core/inculdes/some_procedural.inc
/core/vendor/Synfony
/core/vendor/SomeOtherThirdParty

This for two reasons:

  • Vendor third party inside includes/ can confuse some people and make them think those libraries are Drupal provided
  • Splitting this way makes all namespaces roots living at the same folder depth, seems natural

Aside of that, I'm fine with the overall proposal, I still don't like the module_name namespace (inside a CamelCased hierarchy) but I can live with it. I'm still pro having an optional namespaces[]=SomeNamespace in the modules .info files.

EDIT: catch++ with moving out tests from the simpletest module and make them live in /core/includes/Drupal/[*/]Tests.

donquixote’s picture

If this is going to become "the place" for this discussion, I am going to repeat my post from
#867772-78: Use PHP 5.3 namespaces
(slightly shortened)
I think it is better to start a discussion with a "design space", than with a "proposal". If you agree, we could change the summary accordingly.

Goal

Namespace and filesystem pattern for contrib, that works for
- class autoloading
- regular functions
- hooks
- If you like, auto-detected plugin classes.

Namespace per module

The one question i see is what is ucfirst, what is lowercase.
I would say, ucfirst looks better with classes, lowercase looks better with functions and literal module names.
IMO, if the module name is already lowercase, then drupal\modules can also be lowercase.

The rest is matter of taste.

I also think most of us agree that module namespaces should begin with "drupal\" or "Drupal\", and should be followed by something like "module\" or "modules\" etc, to distinguish it from core library classes or 3rd party code.

Possible patterns for namespace-per-module:

Drupal\Modules\admin_menu\regular_function()
drupal\modules\admin_menu\regular_function()
drupal\module\admin_menu\regular_function()
drupal\m\admin_menu\regular_function()
drupal\modules\admin_menu\SomeClass
Drupal\Modules\AdminMenu\regular_function()
drupal\modules\admin_menu\sub_namespace\another_function()

Hooks

Whatever we do with hooks, it should live within the module's namespace.
On the other hand, they should be distinguishable from regular functions, or regular functions living in arbitrary sub-namespaces.

Things to discuss:
- should the "inventor" of a hook be in the namespace or function name?
- should the namespace tell about the include file, where this hook is expected to be?
(I'd say no to both)
- Is it worth to have hooks in a sub-namespace, or do we rather want to avoid those extra curly brackets?

Possible patterns:

drupal\modules\mymodule\hook_views_api()  // no "inventor" sub-namespace
drupal\modules\mymodule\views\hook_views_api()  // "inventor" sub-namespace
drupal\modules\mymodule\hooks\views_api()  // "hooks" sub-namespace.
drupal\modules\mymodule\hooks\views\views_api()  // "hooks" and "inventor" sub-namespaces

The first one is easiest to type, because we will have only one namespace per file.
It is not 100% safe against hook nameclashes (two modules "inventing" the same hook name with different meaning), but probably good enough. After all, it is probably as likely as two authors making a module with the same name.

It does, however, prevent regular functions from being falsely interpreted as hook implementations. This is the main problem we want to solve.

Auto-detected plugin classes: postponed.

Whatever we do here, I think it can and should live within the module namespace.
And, auto-detected plugins should be distinguishable from regular classes.

Possible patterns:

drupal\modules\mymodule\plugins\views\SomeRowPlugin
drupal\modules\mymodule\views\Plugin\SomeRowPlugin
Drupal\Modules\MyModule\Views\Plugin\Row\NodeTeaser
etc

Imo, we can discuss these another day.

File locations: postponed.

I'd like to see classes in a /lib/ subfolder, and then some tricks to find hook implementations in files other than *.module.
In addition, we should have a mechanism that allows cross-namespace classes.
This said, all of this can be discussed another day.

Crell’s picture

donquixote: No, I do not agree. This is a complete proposal. Please discuss it as is. We've been kicking around "we could do X, or maybe Y, or perhaps Z in one of these forms" for months now. This is a concrete, actionable proposal. Please comment on it directly or do not comment in this thread. There are plenty of other threads for kicking ideas around.

Hooks are deliberately out of scope for this proposal beyond reserving the namespace for later discussion.

Others:

- I'd be OK with a subsystem having a Test sub-namespace as well, unless there's another existing standard we could adopt for that. We should see where Symfony puts its PHPUnit tests.

- I'm not sure where we put pre-module plugins (cache, database, session). That could arguably be a top-level vendor/ or lib/ directory that we scan and register. Or, arguably, if module_list() can run just off of CMI then we could still put them in modules, because their class loader directories would then only depend on CMI and conf_path(). Of course, CMI may in some cases depend on the database... Blargh. Not sure there.

- I'm OK with moving vendor/ out of includes/. I think it's good to have the vendor/ prefix so that we're consistent with emerging trends and documentation elsewhere in the PHP world, but I'm not particular about where it is.

sun’s picture

  1. I especially like /sites/$site/vendor/, because this means that Libraries API no longer has to care for PHP libraries (as long as they're PSR-0 compatible).
  2. Not amused by the additional includes/ sub-directories everywhere though. That adds one more empty directory you need to hop and click through, in addition to the namespace path component directories. I don't see a technical reason for why that is needed.

    Can we skip and remove the includes/ part from the proposal but keep the rest?

    Essentially:

    /core/includes/ - Procedural core code.
    /core/Drupal/Component/ - Non-Drupal-specific libs we write.
    /core/Drupal/Core/ - Drupal-centric subsystems.
    /core/vendor/ - 3rd party libraries shipped with Drupal core; e.g., Symfony2 components.
    /core/modules/$module/ - Classes in the Drupal\Module\$modulename namespace.
    /sites/all/modules/$module/ - ditto
    /sites/default/modules/$module/ - ditto
    

    Modules can still register additional namespaces in particular sub-directories, just like you pointed out in the summary.

  3. I understand the difference between "Core" and "Component", but the terminology is a bit nonsensical; especially for someone new who's trying to make sense of the code-base.

    We should look into better terms for those to clarify their meaning and their content.

    Note: I know that "Component" stems from Symfony, but other PSR-0 libraries do not use that term. We should stop treating Symfony as the golden standard, which cannot be questioned or challenged. That mantra and paradigm is a bit naive IMHO.

Crell’s picture

Re point 3, I am not aware of a broader convention for things like what I'm calling Components here. If there is one, I agree we should adopt it. If not, then parallelism with Symfony makes sense, I think, since that code is already in core.

sun’s picture

Forgot one:

  1. The literal under_score module name Drupal\Module\$modulename (e.g, Drupal\Module\admin_menu\AdminMenuWhatever) looks and feels very wonky.

    However, I understand that an alternate/CamelCase-adjusted name might be problematic, and I'm fine going with under_score names for now, in order to make progress.

    I'd like to see a separate issue to revisit this part later in the cycle (tag with "revisit before release").

pounard’s picture

I agree with #7.

Crell’s picture

Since with git changing that part later would not be catestrophic, just irritating, I am willing to revisit later once we have more experience.

donquixote’s picture

Some general points:

1. ucfirst vs lowercase:
Unlike Zend or symfony, Drupal has plenty of functions, which (imo) tend to look better with lowercase namespaces. Besides, module names are lowercase. So, we have to decide, what is more important: Looks nice with classes, or, looks nice with functions, or, looks nice with module names.
We can discuss this elsewhere, but please, don't just skip it.
Possibilities as in #3, not repeated here.

2. /includes/ vs /lib/
-> distinct places for classes and functions
This applies to core and contrib: /includes/ has traditionally been a place to put *.inc files filled with functions.
I would prefer if we had a different place for classes. My proposal would be /lib/.

3. overall naming conventions
E.g., Drupal\Modules\ vs Drupal\Module\ - the proposal itself has both variations :)
Can be discussed elsewhere, but should be done before anything is committed.

None of this kills your proposal.
But, changing these things later would be a pain for module maintainers.

----------------------

All of the following is with reservation to the points 1. - 3. above.

Namespaces:

Drupal\Component\$component -> great!
Drupal\Core\$subsystem -> yep
Drupal\Module\$modulename -> yep. +1 for literal module names. upper/lowercase as above.
Drupal\Hook and Drupal\Module\$modulename\Hook - Reserved -> yep (*)

(*) We could reserve more stuff for hooks, such as Drupal\Module\$modulename\$inventor\, or Drupal\Module\$modulename\$inventor\Hook, or Drupal\Module\$modulename\hook_
Anyway, this can be discussed later.

Disk organization:

/core/includes/Drupal - would prefer /lib/, otherwise ok.
/core/includes/vendor/ - would prefer /lib/, and/or move /vendor/ out of /includes/, as suggested.
$moduledir/includes for Drupal\Modules\$modulename - would prefer /lib/, otherwise ok.
$moduledir/+ - ok (*)
sites/[all|domain]/vendor - cool.
(**)

(*) We might want to reserve more stuff in the module dir for plugins or hooks or whatever, but this can be discussed another day.
(**) We might want additional folders for auto-generated code, file-based cache, etc. Anyway, can be postponed / discussed elsewhere.

Class loader configuration process:

In general, I am fine with anything that works :)
At least, fixing or improving this stuff does not cause any disruption for module maintainers.

Just one thing I want to add:
Afaik, Symfony maintains a cache of autoload classes. This cache is a big php file that defines an array. (or was it .ini? or .yml?) Array lookup is cheaper than file_exists().
For Drupal, we could decide to cache everything, or just some classes.
But again, this can be discussed elsewhere.

---------------

Overall:
I am generally happy with the proposal, except for naming conventions for. So, back to #3.
In fact, this is all within the limits of what I called a "design space" in #3.
With some additional ideas for core components and subsystems, which is cool.

donquixote’s picture

I was quite fond of moving out the vendor dir from includes

Yes..
I wonder, if we would have a separate folder for classes (so functions can continue to live in /includes/), then do we still want to move "vendor" into /core/vendor/ ?
And, if only /Drupal/ is left in /core/includes (or /core/lib), could we skip that step?
Or, should we rather have
/core/Drupal/.. - core classes in sub-namespaces of Drupal\..
/core/includes/.. - functions in namespace Drupal\..
/core/vendor/.. - 3rd party classes

Finally,
what should be the namespace for core functions? And, if a function lives in a sub-namespace of Drupal\, in a component or subsystem, where should the file be located?
Mixing function include files into the psr-0 folders seems like a bad idea.

pounard’s picture

Mixing function include files into the psr-0 folders seems like a bad idea.

I was against that too, but I changed my mind to: "I don't care". I give more importance to moving the vendor folder outside of includes to be sure there is no confusion about what's Drupal and what's third party. Even if I'd prefer to see Drupal outside of includes I could definitely live with it inside and be happy.

donquixote’s picture

Even if I'd prefer to see Drupal outside of includes I could definitely live with it inside and be happy.

So far we see those functions at Drupal\ root namespace.
However, what if you have a function at
Drupal\Component\Something\some_function()
in which filesystem location should that live?

neclimdul’s picture

@donquixote That's a lot of text... Don't know how to respond really but at least as far as the autoloader cache, we do support their APC cache with builds the list of the fly and doesn't do lookups if its already in APC. Its already in core so you can look at it there and handle a different issue for any improvements you think of. Not really relevant to the proposal in anyway though.

neclimdul’s picture

I honestly think functions and namespaces are a bit of a trap distracting from the bigger PSR-0 wins. PSR-0 doesn't really address functions so lets move that discussion elsewhere and see if we can nail this down to something easily actionable, specifically how we'll address PSR-0 namespaces.

Crell’s picture

Function namespaces are out of scope for this proposal, since no functions are in namespaces. Their disk location is also less relevant since they cannot be autoloaded anyway, no matter what they're called or where they live.

I am fine with lib or includes. I have no strong preference.

I'm also cool with moving vendor/ out of the includes directory.

Module vs. Modules: It's supposed to be singular. I made a few editing errors when writing this up. :-) I'll remove the plurals.

DjebbZ’s picture

I like the approach.

Just a question/suggestion : apart from the auto-loader, should the other vendor components be registered as fallbacks too ? It will allow contrib/custom modules to override non-core code, or update it without waiting a core update. If vendor code must wait a major release to be updated, I think we should definitely register them as fallback. Imagine a super interesting update from Symfony2's HttpKernel, or a better fork in Github, or just a very specific client that needs fancy stuff. We would lock Drupal to a specific version of a code we don't write nor control whereas we ship core with it, and at the same time offer the possibility to override the "real" Drupal code core ships with. Better have a safety net for vendor code too.

I understand from the issue where msonnabaum wrote the patch that the code that would override Symfony's code (or other vendor) couldn't be overridden because it wouldn't pass the strpos, because it's possible that these new namespace start with Symfony. But they could, if you want to update it by override. Is there a clean workaround or solution to this problem ? I think I made some valid points about vendor code "overridability".

pounard’s picture

I guess that when we override Symfony, we should avoid namespacing our own code as Symfony, it's plagiarism.

donquixote’s picture

Function namespaces are out of scope for this proposal, since no functions are in namespaces. Their disk location is also less relevant since they cannot be autoloaded anyway, no matter what they're called or where they live.

Sorry for being so pesky with this.
No, functions do currently not live in namespaces. But it is not a too ambitious guess that at least in D9 everything will be namespaced. For my taste, rather in D8.

What I would like to avoid is that some day we say, "haha, module namespaces have already been decided, now let us use the same namespaces for functions." if the year before we said "functions are out of scope, it does not matter if our namespace conventions look nice with functions".

All I want is, "this is how it would look like for functions, and we are ok with that. Noone can say we did not talk about it."
So,
Drupal\Module\mymo_dule\foo()
Drupal\Module\mymo_dule\hook_menu()
vs
drupal\module\mymo_dule\foo()
drupal\module\mymo_dule\hook_menu()

Now, looking at this again, the ucfirst version is not so bad. We will survive :)
If everyone else agrees, we can finish this argument, and gain peace of conscience.

sun’s picture

  1. There's no feedback on #5 2) Removing includes/ path component everywhere (and retain /core/includes/ for procedural/non-namespaced code).
  2. Apparently, Symfony Standard Edition packages its own components under /vendor/symfony (next to other vendors). Why do we structure it differently; i.e., why is Drupal not in /core/vendor/Drupal/?
donquixote’s picture

@sun,

"Removing includes/"

Not happy.
Imagine you have a module with plenty of classes, all living in the module namespace.
Without that /include/, you would have
$module_dir/cat.module
$module_dir/cat.info
$module_dir/WhiteCat.php
$module_dir/BlackCat.php
$module_dir/YellowCat.php
etc.

I'm all for renaming the /includes/ to /lib/, but not to drop it.

Maybe a different story for core, because the top-level only has /Drupal/.

Apparently, Symfony Standard Edition packages its own components under /vendor/symfony (next to other vendors).

Wow, good you had a look!
So.. why not
/core/lib/Drupal/..
/core/lib/Symfony/..

/lib/ would just indicate that this is a psr-0 folder, containing files with classes.
In core, this would all begin with the root \ namespace.
In contrib, /lib/ would begin with the module namespace.

Otherwise, replace /lib/ by /vendor/.

Imo, we do not gain much by the term "vendor". Esp, if the same namespace contains contrib and custom code, which is clearly written by other parties.

DjebbZ’s picture

  1. sun, about includes/ everywhere. I'm in favor of removing them in core/, but keep them in $moduledir as said donquixote.
  2. I prefer lib/ to includes/
  3. I agree with sun, Drupal code could live in core/vendor/Drupal. It's enough to differentiate for Drupal community contributed code from third party code.
  4. Thanks pounard, I didn't mean to be a plagiarist :) So how can you override Symfony's code if we wanted to ?

Edit : cited donquixote

sun’s picture

I prefer lib/

We have a general rule of not abbreviating names anywhere in Drupal, but using explicit names instead. Based on that, lib/ is a non-starter in the first place.

DjebbZ’s picture

Didn't know that, thanks for the info. I'm fine with that.

donquixote’s picture

We have a general rule of not abbreviating names anywhere in Drupal, but using explicit names instead. Based on that, lib/ is a non-starter in the first place.

Hmm.. /javascript/ .. /cascadingstylesheets/ .. /miscellaneous/ .. (:
Anyway, the main point was to keep classes separate from procedural include files.
"lib" for me is just a meaningless folder name. "libraries" makes me think of the libraries module, which is misleading. If it can't be lib, then rather not libraries.
Curious where this goes.

Crell’s picture

OK, I just spent my Saturday night poking through a number of other projects to see what they do. :-) Details here:

http://www.garfieldtech.com/blog/php-project-structure

Executive summary: There is no clear industry-wide standard as far as I can tell.

Symfony-family projects seem to use /src/ for their own namespaced code and /vendor/ for 3rd party code. CakePHP uses /lib/ and /vendor/ for the same purpose. Symfony bundles are self-rooting, but are not allowed to have non-PHP code in them the way Drupal modules frequently do.

Zend uses /library/ for its own namespaced code, and doesn't seem to have anywhere to put non-native-namespaced code. (Not surprising, since it assumes that it is the non-native code being used.)

CodeIgnighter is not namespaced, but uses /system and /application/third_party in roughtly a parallel fashion, I think.

Composer enforces a vendor/ directory, but the organization below that will vary by project.

For our case, then:

- If we don't like /core/includes as a place for Drupal-namespaced classes (fair point), then /core/src and /core/library would be the non-unique alternatives. Between the two I think I prefer /core/src for the reason donquixote mentions; "library" makes me think of libraries.module. I can live either way, though.

- For non-Drupal-namespaced code, the only consistency seems to be that it is NOT put into the same directory as native namespaced code. /core/vendor/ would be consistent with Symfony-based apps. I far prefer that to /core/third_party. Blech. :-)

- Composer uses /vendor, but not in the nicest way. I suggest we punt that question to #1398772: Replace .info.yml with composer.json for extensions.

- For modules, Symfony bundles are entirely outside the Symfony namespace completely. Zend modules appear to be in the Zend namespace (if I understood them correctly), but the directory has a complete namespace/path tree. Neither of those look like what's in the proposal above. Given that Drupal is far more centralized in its code libraries and modules, I'd rather keep everything in a Drupal namespace by default.

- As far as tests go, a top-level test or tests directory seems to be most comon. That is, a /core/tests directory containing test classes for core, which then have a similar structure to the namespace of the classes they're testing, would be reasonabley consistent. I like that approach.

I would therefore propose the following changes to the above proposal:

- core/includes - Procedural code (as now, no change).
- /core/src/Drupal - Drupal-namespaced code.
- /core/vendor/* - Non-Drupal-namespaced code that ships by default.
- $modulename/src - => Drupal\Module\$modulename code.
- Non-Drupal-namespaced code that is a 3rd party dependency: Punt for now.

Thoughts?

Michelle’s picture

Comment withdrawn. My thoughts on this don't matter.

pounard’s picture

I like that proposal, there are still details I don't like, but it's a good one that sounds like a reasonable compromise. I say let's go for it!

EDIT: I'm all for including a core/tests too.

donquixote’s picture

Overall ok.
Whether it is src or lib or something else, we just need to get used to it.

#23 (sun) argued that we don't want abbreviations.. would that stop us?

yched’s picture

For someone grabbing its 1st drupal tarball and going through the folder structure, I'd find core/src and [module]/src highly misleading. As if the stuff outside src was not source code too.

We will have a "transition" phase where not all our code (core and modules) lives in psr-0 classes (unlike most frameworks listed above), and that phase might last 2-3 releases - or possibly forever. We should accept that mixed nature, and adopt a folder structure that embraces it, not one that tries to hide it under the rug.

So if we have to chose, 'lib' or 'library' sounds less restrictive than 'src'.

[edit: fixed typo]

pounard’s picture

I agree with #30 about this one, this is a very valid point "library" is less confusing, but I still could live any of "src" or "library".

donquixote’s picture

#30

As if the stuff outside src was not source code too.

Valid point.

We will have a "transition" phase where not all our code (core and modules) lives in psr-0 classes (unloke most frameworks listed above), and that phase might last 2-3 releases - or possibly forever. We should accept that mixed nature, and adopt a folder structure that embraces it, not one that tries to hide it under the rug.

Very much agree!

Now, imo, "src" is as much an abbreviation as "lib" (or "misc").
Any meaning that would be added by saying "library" or "source" instead of "lib" or "src" would be more misleading than it would help. With "lib" or "src" it is easier to accept that this is just a stupid policy that someone agreed on, without assuming any philosophical depth.

webchick’s picture

Is there any more compelling -1 argument against "library" as the directory name, beyond "'library' makes me think of libraries.module"? Because if not, I don't see a reason for us not to call these things what everyone else calls them just because we have a legacy mental hurdle to overcome.

Furthermore, isn't this essentially an equivalent of Libraries module we're putting in core, focused on the management of PHP libraries..?

Michelle’s picture

It's a moot point... Found out on IRC last night that classes have to go in a directory named "drupal" because that's how the rest of the world does it and having people outside of Drupal see what they are used to when they look at Drupal is more important than naming it in a way that makes sense.

Michelle

pounard’s picture

@#34 Putting files in folders that are mapped on namespaces also make much more sense than any actual Drupal OOP code organisation (there is none, actually): the folder structure will be the exact same tree than the code organisation, flow, naming conventions etc. What you say is true for the "Drupal" folder but isn't for what we put in it.

Michelle’s picture

@pounard: Sorry, I haven't the foggiest idea what you're saying. Don't worry about it, though. I was only posting to let them know that they are bikeshedding over an already painted bikeshed to save them wasting their time. That's all the more I intend to be involved in this.

Michelle

Crell’s picture

We usually avoid abbreviations, but as #25 notes there are exceptions, usually when an abbreviation is an obvious standard (eg, "js"). So I don't think src or lib are problematic strictly for that reason, since both are very old and well-known standard abbreviations. That doesn't make them the right choice, but it means they're not ruled out automatically on those grounds.

At some point this is a pure bikeshed, though, and a decision just needs to be made. Any thoughts on any of the other points from #26?

Crell’s picture

To clarify:

Michelle's comment relates to msonnabaum's observation in IRC last night that it appears that we cannot, in fact, make Drupal\Module\foo map to simply sites/all/modules/foo/src (or whatever). The class Drupal\Module\foo\Thingie would have to live at sites/all/modules/foo/src/Drupal/Module/foo/Thingie.php. That is, you can't map a folder to a portion of a namespace so there would still be multiple directories below whatever directory we bikeshed our way to. Michelle saw that and went "OMGWTFBBQ!" ("That's how the rest of the world does it" being "that's what PSR-0 seems to require".)

I'd like to verify that conclusion before we assume it's true. I would find it highly annoying. :-/

donquixote’s picture

I'd like to verify that conclusion before we assume it's true. I would find it highly annoying. :-/

Me too.
In threads like this and other, we have seen this statement, and its opposite.
This also was the number one reason for my own doubts about following PSR-0 100%. Having those three additional levels on module directories would be super painful.

bojanz’s picture

Seriously, "sites/all/modules/foo/src/Drupal/Module/foo/Thingie.php" sounds horrible.
Either we find a way to follow the standard without stabbing contrib authors in the eye with a fork, or we don't follow the standard.

pounard’s picture

It's not the best we could imagine, yes, but it doesn't sound *that* horrible. At least it forces code to have a logical and strict physical organization: IMHO it worth the pain opening 2 more folder if it makes the whole code being structured. But I totally understand why it sounds horrible to some people.

Michelle’s picture

The pain isn't just in opening a couple folders. It's in having one bit of your code in this crazy messy thing. So you have, say:

mymodule/includes
mymodule/css
mymodule/js

Nice, neat, understandable. You know what is where just by looking at it. And then instead of something sensible like mymodule/classes, which I've been told is now old fashioned, you get this fancy new mymodule/src/Drupal/Module/mymodule to store your class files in.

To me, that's just plain icky. I was patted on the head and told that I just don't understand because all I know is Drupal and evidentially this structure makes sense to more worldly people. But for simple folks like me who just know Drupal, it's a serious WTF.

Michelle

donquixote’s picture

Hell yeah, "classes" is actually the most honest folder name that I have seen in any of those discussions!
Nothing you will see in other frameworks, but.. it is very to the point.

And I totally agree with #40 and #42, those 3 extra levels would be just plain horrible.
I've been saying it over and over, but it seems only now is it actually "on the table". Ok, whatever.

@pounard,
yes, it is +3 levels, not +2.
(lib|classes|whatever)/Drupal/Module/mymodule/MyClass.php
vs
(lib|classes|whatever)/MyClass.php

So, what is really wrong about those +3 extra levels:
1) 3 extra clicks or navigation steps (think of cd .. in the terminal, etc).
2) Longer paths in your terminal, in your file browser, etc.
3) More places that could contain a file that is from a different namespace, even though in 99% of modules it is not the case.
4) More chances to do it wrong, move stuff into the wrong folder, etc.
5) Much more work to inspect 10 modules and see which classes they have.
6) It will be tempting to use more shallow namespaces, outside of Drupal\.

(numbering only for quotability)

------------

Could anyone actually look this up somewhere and post a link? So we don't discuss for nothing. How does symfony do it, or what does the "standard" say? Atm I don't know where to look.

Mark Trapp’s picture

The relevant code is in Symfony\Component\ClassLoader\UniversalClassLoader::findFile(). UniversalClassLoader works on the assumption that what you're feeding it is the base path to a PSR-0 compliant directory structure, which I don't believe is the case here.

Let's say you register your namespace with Symfony\Component\ClassLoader\UniversalClassLoader::registerNamespaces(), and you try to do something like this:

$loader = new Symfony*Component*ClassLoader*UniversalClassLoader();
$loader->registerNamespaces(array(
  'Drupal\Modules\my_module' => $module_path . '/includes',
));
$loader->register(); 

When you attempt to use Drupal\Modules\my_module\Foo, the registered autoloader won't ignore the Drupal\Modules\my_module part just because you registered it as linking to a specific directory (i.e., what people are expecting here).

Instead, it'll attempt to generate the path to the class off of the full namespace (i.e. Drupal\Modules\my_module\Foo.php) (line 250), and it'll just use what you registered in UniversalClassLoader::registerNamespaces() as the place to look for the full file path: e.g., /sites/all/modules/my_module/incudes/Drupal/Modules/my_module/Foo.php (line 257).

My understanding of the convention is that what is trying to be done here, where modules are outside the directory structure of the Drupal namespace, isn't PSR-0 compliant. That is, if Drupal is mapped to /core/vendor/Drupal, Drupal\Modules\my_module really ought to be in /code/vendor/Drupal/Modules/my_module in order to be PSR-0, not /sites/all/modules/my_module/includes.

Crell’s picture

Well, forcing a module to get split up on install and go various places is not really an option. Plus, if a single project having more than one root for a given namespace is a PSR-0 violation then Symfony2 encourages such violation as well, since there are lots of FOS bundles (Friends Of Symfony, sort of the generic namespace, kinda like contrib, I think) that use the FOS namespace but are in different download bundles, so in different places in the file tree.

Based on the conversations on the standards list back when PSR-0 first came out, I don't think it invalidates PSR-0 because the unified class loader still works.

Mark Trapp’s picture

FOS uses a singular vendor/bundles directory structure mapped to the FOS namespace. For example, FOSUserBundle would be found in vendor/bundles/FOS/UserBundle and called via FOS\UserBundle.

Not exactly the same as what Drupal would want to do, which is map Drupal\Modules to either /core/modules, /sites/all/modules, et al and ignore the includes directory when building a module's namespace (so, path/to/modules/my_module/includesDrupal\Modules\my_module).

I agree and it's a fair point that using the DrupalWTF structure (e.g., path/to/module/my_module/includes/Drupal/Modules/my_module/Foo.php) would be compliant, although it's a bit of a non-Drupal WTF that there are two different places Drupal\* namespaces are provided: one would expect the code for Drupal\Modules to be in a subdirectory under Drupal.

However, path/to/module/my_module/includes/Foo.php would not be compliant if you wanted to use the Drupal\Modules\my_module namespace: Drupal/Modules/my_module must exist in the path.

sun’s picture

  1. It looks like no one here wants to go with a full namespace path structure within a module directory, so let's stop discussing that.
    (there's no point in blindly following a standard that wasn't designed for the purpose in the first place, and trying to bend it just to make any sense.)
  2. So can we extend the autoloader to support a ->registerModuleNamespaces()?
    (sounds like this would be a base starting point for a PSR-1 proposal to be discussed with the larger PHP community)
  3. AFAIK, we still want to allow modules to put classes into other namespaces (e.g., extending or even overriding core stuff). For that, we likely need an additional mechanism for registering custom namespaces within a module directory, and possibly also a different sub-directory.

Aside from that, I like the proposal in #26, with the exception of 'src'/'library':

  • for modules: As others mentioned, a name like 'classes' would be a lot more clear and intuitive, especially if you consider 'js', 'css', 'includes', and possibly more directories next to it -- 'src' or 'library' would be very ambiguous, since JS, CSS, and whatnot is also source code, or potentially also contains other kinds of libraries.
  • for core: /core/src/Drupal does not show the full picture in terms of empty directories to click through; the current proposal actually foresees another Component|Core sub-directory within. Given that the Drupal/Component|Core part is part of the namespace and there seems to be no way around for that to be empty, the additional (empty) 'src' directory seems entirely superfluous and unnecessary. The 'classes' proposal for modules doesn't really work here either. Since we need a different layout for modules anyway, I'd simply drop the 'src' directory component from the core layout proposal, so we have /core/Drupal and /core/vendor/*. (most of the time we'll only work in the /core/Drupal stuff anyway)
pounard’s picture

So classes folder would also contain interfaces and may be traits in the future? Classes seems broken to me and refers to a really old time now where PHP did only support classes, which is not the case anymore.

core/src/Drupal has a meaning, for example, if somehow we port some core classes as PSR-0 code without namespacing it, we could end up with:

core/src/Drupal/Core/..
core/src/Drupal/Component/..
core/src/SomePoorlyPortedClass.php

If we don't have the src/ (or library) folder we can, even if we don't want, but logically can, have .php files directly into core/ which would break all encapsulation we are doing right now.

Besides, nothing says that we would have only one core namespace:

src/Drupal/...
src/DbTng/...

DbTng is almost a standalone library, I don't see it be forever namespaced under Drupal vendor.

yched’s picture

A 'classes' folder containing interfaces in addition to classes doesn't strike me a as a real issue.
I'm much more reluctant to see a 'src' folder with core source code *outside* of it.

pounard’s picture

I'm not saying it's an issue, if it comes to this name I will accept it; I'm just saying that historically, a lot of frameworks and applications used to name their OOP code folder "classes", but this is not really the case anymore (at least a lot less seem to do that). "src" or "library" or "lib" are names for which usage tends to raise now, and I guess this is not only for pure semantics, but also a sign of rupture between old PHP <=4 legacy style and new PHP >=5 modern framework orientation. What's modern now may not be modern anymore within a few years, but at least using the same samantics would make Drupal seamlessly integrate its change within the era in which it did change. This seem important -in term of pure semantics- because someone that will see this code in a few years from now will be able to tell when it did happen and what he will find in it.

donquixote’s picture

Probably this is because in most of these frameworks you don't find any procedural code.
Naming this stuff "classes" indicates "you know, not everything is classes, we also have procedural". Naming it "src" or whatever, indicates "we don't need a place for procedural, because procedural stinks".

Michelle’s picture

When the day comes that Drupal drops the last line of procedural code, then naming it "src" will make sense (aside from abbreviations being icky). Until then, putting the classes in a directory named "classes" makes more sense to me than putting them in a directory named "src" and having 80% (guestimate) of our "src code" be outside of that directory.

Maybe the pro OOP folks can celebrate that day by having a ribbon cutting for the newly renamed "src" directory. ;)

Michelle

Crell’s picture

At the risk of bikeshedding even further off topic, "classes" is the most misleading (as yched notes) and self-antiquating (as pounard notes) name we could go with. Even includes is better than that. :-) "Other projects are doing it" *is* a valid argument, for the simple reason that we want to be able to absorb (read, steal) developers from those other projects. The more comfortable it is for them to jump ship to Drupal, the more likely they are to do so. So if src is what the de facto convention is, that carries weight even if we may have "source" code that is not in that directory. (For any modern project, there is virtually no procedural code to deal with.)

That said, does anyone besides sun want to comment on anything else in #26? Specifically:

- core/includes - Procedural code (as now, no change).
- /core/X/Drupal - Drupal-namespaced code.
- /core/vendor/* - Non-Drupal-namespaced code that ships by default.
- $modulename/X - => Drupal\Module\$modulename code.
- Non-Drupal-namespaced code that is a 3rd party dependency: Punt for now.

Also note there have been some additional enlightening comments on my blog post:
Zend 2: http://www.garfieldtech.com/blog/php-project-structure#comment-2108
Symfony2: http://www.garfieldtech.com/blog/php-project-structure#comment-2102

donquixote’s picture

I don't have any more strong points to make about #26 / #53. We agree on one of these things, and I am happy with it.

But, something I'd like to say about modules.
One important thing in the "enlightening comments" on Crell's blog was that "modules" should be created reusable across frameworks.

For Drupal, I think the solution is to have modules consist of two parts: "pieces" and "glue".
"pieces" is the reusable stuff. Some of this can be shipped with the module, other parts might be pulled in via libraries module.
"glue" is what lets the reusable components talk to Drupal: Hooks, plugins, anything that actively calls Drupal functions and APIs.

Probably a lot of the "pieces" will be OOP, while much of the "glue" will be procedural.

Maybe some modules will not even be designed specifically for Drupal, but rather have a huge collection of reusable code, and then some framework-specific glue code. Similar to what CiviCRM does (but maybe more beautiful in code).

What does this mean for our namespacing?
I think the namespace-per-module, starting with Drupal\Module\$module_name, is still the correct way to go.
We just need to get used to more stuff being in sites/all/libraries, and more out-of-namespace code within Drupal modules.
For stuff like CiviCRM, the module directory could simply be a symlink into a subfolder of that project.

catch’s picture

With classes/lib/src (for classes currently in /includes).

classes does look a bit old fashioned to me.

src - that is going to be strange compared to the amount of source code that'd be outside of it, while other modules might use it, they also don't have such a large code base outside of that directory so it's a different context.

lib - doesn't really mean anything, but is also the least offensive and not inaccurate.

Unless a dramatically better new option shows up, I'd go for lib. Remember this is only one directory, and we won't even be including these classes directly it'll just be a namespace registered for the autoloader, so if we wanted to change it at some point, that should be a 2-3 line patch (+ docs).

I am much more concerned about the subdirectories inside module folders - that's a much bigger change, seems really, really awkward to me, and is not going to be easy to change at a later date.

So I am wondering a bit:

1. Could we use 'lib' for now, and tag that decision with 'revisit before release' (or revisit before beta). There are several PSR-0 conversion patches which currently target /includes, it'd be good to only have to move those classes around once.

2. Is there anything else in here that's controversial?

3. If not, could we move this to RTBC, then continue the module discussion at #1290658: Move all module-provided classes to PHP namespaces (PSR-0 or similar), and autoload them without the registry [policy, no patch]?

webchick’s picture

Status: Active » Reviewed & tested by the community

Yes, please.

plach’s picture

Just to be clear: is the RTBC proposal #53 with X = 'lib'?

neclimdul’s picture

Status: Reviewed & tested by the community » Active

Thanks catch! I was going to avoid addressing the folder name discussion because I think its the least important point we have to deal with but you summed it up so well and I agree with every statement.

I would like to add as we move to the module specific questions in the other issue, I think we should be consistent with that name across Drupal unless there's a enormously strong reason to use something else.

The rest of this comment will carry over into #1290658: Move all module-provided classes to PHP namespaces (PSR-0 or similar), and autoload them without the registry [policy, no patch] but I want to put them here since they follow up #47. TL;DNR I think we absolutely need the full PSR-0 path structure.

1) I for one do feel that we should have the full namespace path structure. Not doing so would be a serious mistake. I think there's been a great push in Drupal recently to accept the PHP community as part of our own not accepting PSR-0 whole heatedly and instead mixing it with Drupal specific quirks greatly diminishes the benefits we're gaining from using it.

  • Well designed Drupal Libraries can't be used by PSR-0 frameworks and the like without moving things around and doing weirdness which makes it un-maintainable on their end.
  • Drupal Modules using PSR-0 namespaced libraries have to use a completely different system for registering and autoloading classes. More work for module developers and something else they have to keep straight. For example, Michelle might like to use phpBB4 libraries someday or allow phpBB4 components which are suppose to be built on Symfony to interact with Artesian Forum but that would be a seperate folder structure and registration method and would likely have to be maintained outside of core or as a secondary core registration system.

2) First, I think exceptions should be handled by PSR-1. If something like "registerModuleNamespaces()" is what the module PSR is, I disagree. First you're requesting registering sections of namespace directly to a folder and you should call it that not modules. Module organization is a different problem which Drupal has very novel ideas which allow us to do some truely awesome things but aren't strictly compatible with what everyone else is doing in the area. If we can just support it with what's really a fairly small consolation I don't see why we should make a big thing out of it.

3) Exactly, this we agree on but not the conclusion being reached. Why would we have more then one method when one will do? Do we want to maintain 2 code paths instead of just pointing an autoloader at these roots?

neclimdul’s picture

Status: Active » Reviewed & tested by the community

X-post

donquixote’s picture

Well designed Drupal Libraries can't be used by PSR-0 frameworks and the like without moving things around and doing weirdness which makes it un-maintainable on their end.

You need either a separate autoloader (which can be shared across anything from Drupal), or create a few symlinks.
Both of this is doable.

Drupal Modules using PSR-0 namespaced libraries have to use a completely different system for registering and autoloading classes.

That would be one line in the .info file saying

; Class phpBB4\SubNamespace\SomeClass.php sitting in [path to module]/phpBB4/SubNamespace/SomeClass.php
autoload[phpBB4] = phpBB4

or

; Class phpBB4\SubNamespace\SomeClass.php sitting in [path to module]/external/phpBB4/SubNamespace/SomeClass.php
; external mapped to root namespace
autoload[] = external

Or something equivalent using a hook or whatever.
The cost: not really anything. (I wonder if you can come up with something that needs zero lines of configuration)
The great benefit: By looking at a module folder, you immediately see what is Drupal-namespaced code, and what is outside the Drupal namespace, without diving into subfolders.

donquixote’s picture

Issue summary: View changes

Remove accidental plurals.

Crell’s picture

I just updated the summary with lib, as everyone else was talking, apparently, and someone else was editing the summary. BAH!

I agree with neclimdul in #58 regarding the benefits of accepting PSR-0 as is.

webchick’s picture

Title: Proposal for unified namespace organization » Proposal for unified namespace organization [policy, no patch]

All of the comments since #55 seem to be about module-provided class names, correct? If so, those should go in #1290658: Move all module-provided classes to PHP namespaces (PSR-0 or similar), and autoload them without the registry [policy, no patch].

If there are compelling reasons not to use /lib/ as the baseline, which are really worth further holding up the entire PSR-0 conversion of core classes movement, speak now. (Note: I don't consider "we never abbreviate words" to be such a reason, since as various folks have pointed out, we actually do do that, especially with directory names (css, js, misc).)

If not, let's hash out the rest over in #1290658: Move all module-provided classes to PHP namespaces (PSR-0 or similar), and autoload them without the registry [policy, no patch] and close this one out with a "Revisit before release" tag.

webchick’s picture

And in the future? I'd suggest we timebox these bikeshed discussions to two weeks or similar.

neclimdul’s picture

I think /lib is great. The abbreviation is an accepted standard from FHS that most of us are familiar with so its at the very least good enough for now. Lets start discussing all the module related concerns on the other issue. That's progress and narrows the scope of the discussion.

Crell’s picture

Title: Proposal for unified namespace organization [policy, no patch] » Proposal for unified namespace organization

We would need a patch to adjust the autoloader config. I will take that on, along with moving the agreed upon parts of the summary (Core, Component, vendor, tests) to the handbooks somewhere.

pounard’s picture

I'm really happy to see this particular issue going out of bikesheding, thanks catch and webchick.

Michelle’s picture

You must not be on IRC, then...

Michelle

webchick’s picture

pounard’s picture

Yeah, decide whatever you want, anyway, I just want this issue to be sorted out, and I don't really care it makes people happy or angry, we need it for a lot of other dependent tasks.

EDIT: I'm all for the +revisit later tag as catch said, we need a working solution we can still step back anytime before the code freeze.

Crell’s picture

Status: Reviewed & tested by the community » Needs review
FileSize
451.62 KB

Patch to reorganize the existing Drupal and Symfony-namespaced code as above. Also adds README files to document the various directories and their purpose. It does not touch modules yet.

Note: The patch looks huge, but that's because 99% of it is just moving code around. The actual changes are tiny. I recommend NOT committing it, but merging from this branch:

http://drupalcode.org/sandbox/Crell/1384132.git/shortlog/refs/heads/1400...

Status: Needs review » Needs work

The last submitted patch, 1400748-namespaces.patch, failed testing.

Crell’s picture

Status: Needs work » Needs review
FileSize
453.04 KB

Grumble grumble installer grumble grumble.

tstoeckler’s picture

Status: Needs review » Needs work
+++ b/core/includes/bootstrap.inc
@@ -2293,7 +2293,7 @@ function _drupal_bootstrap_configuration() {
     // All Symfony-borrowed code lives in /core/includes/Symfony.
-    'Symfony' => DRUPAL_ROOT . '/core/includes',
+    'Symfony' => DRUPAL_ROOT . '/core/vendor',

@@ -2303,7 +2303,7 @@ function _drupal_bootstrap_configuration() {
     // All Drupal-namespaced code in core lives in /core/includes/Drupal.
-    'Drupal' => DRUPAL_ROOT . '/core/includes',
+    'Drupal' => DRUPAL_ROOT . '/core/lib',

The comments need to updated as well.

-22 days to next Drupal core point release.

jbrown’s picture

I am extremely happy with the decisions reached in this thread. Thanks everyone!

Couple of ideas - feel free to shoot them down.

What about calling the 'lib' directory 'psr-0'? As soon as new-to-Drupal developers look at it they will know exactly how the directory is organised.

I'm not keen on having separate Drupal\Core and Drupal\Component. If a system is "freestanding" then it should be in its own top-level namespace, e.g.

/core/lib/Drupal/Cache/CacheBackendInterface.php
/core/lib/DbTng/DatabaseConnection.php

This simplifies the hierarchy for the Drupal namespace.

Thoughts?

pounard’s picture

I wouldn't mind removing the Core and Component separation, but I still like them, it clearly decouples Core heavy business and almost standalone pieces of API, it seems a nice separation to me. I agree with DbTng having its own namespace thought, since it has a Drupal-free existing copy over a github project it makes real sense to me.

'psr-0' is weirder than all the previous suggestions, but I understand the argument, but still it wouldn't be the only 'psr-0' container since vendor is supposedly containing PSR-0 code too.

donquixote’s picture

I also thought of naming it "psr-0" for a short moment, but then discarded that idea. It just looks far too awkward. And it is awkward to speak, peeassarezero vs just "lib".

--------

For the Core vs Component, I generally agree having those two sub-namespaces.
I wonder, if we want to extend this with 3rd party or contrib components, where will these live, and how will they be namespaced? Will they ship with modules, or be downloaded into sites/all/libraries?

pounard’s picture

"vendor" is for 3rd party components (just like Symfony components): the fact that you need to download them or not would probably be licence driven and not technically driven

donquixote’s picture

My point or question is, should we expect to see stuff in Drupal\Component namespace, that lives outside the core/lib/Drupal/Component folder?

pounard’s picture

Following the proposed standard, my guess is no: in core/ lives all the Drupal core stuff, and Drupal\[Core|Component]\* is reserved to core: seing code namespaced as one of the two outside of this folder would probably be wrong module or 3rd party code.

donquixote’s picture

A module is something that has an entry in the system table, that can be enabled or disabled, that interacts with the rest of the world via hook implementations.

A "component" is just a bunch of classes, that can be used one or another way, even outside of Drupal.
I don't see how this concept has to be restricted to core.

pounard’s picture

Drupal namespace is owned by Drupal any 3rd party code naming itself Drupal would be rude. Proposal also states that modules should use a sub namespace of their own and not live in Core reserved sub namespaces. Convention is convention, it's arbitrary, if we start violate it even before it got commited, we could just abandon it right now because it wouldn't serve any purpose.

donquixote’s picture

what what..
Drupal\Module\abc is ok for contrib modules
Drupal\Component\xyz is not ok for contrib components?

Crell’s picture

The proposal was that anything provided by a module *should* go in Drupal\Module\$modulename. That the code is self-contained is a good thing in and of itself, as any module should be. Drupal\Components were specifically for core-provided free-standing bits.

That said, if someone puts code in $moduledir/lib/Drupal/Components, it will probably work just fine. (Assuming we use the PSR-0-based structure for module lib directories).

I'll reroll for #73 tonight.

jbrown: I prefer to not pollute the top level vendor namespace too much if we can avoid it. Originally I wanted to have DbTng (or renamed) as its own top-level thing in order to encourage it to be non-Drupal-specific, but I realized that there are other pieces of code we have in core that could/should be as well. Separating those off into a Drupal/Components namespace is an pattern stolen from Symfony2.

jbrown’s picture

Okay - I'm onboard with Drupal\Components.

But there is another problem - the proposal is currently

core/modules
core/vender

which contradicts

sites/all/modules
sites/all/libraries

We can't have both

core/lib
core/libraries

Do we change sites/all/libraries to sites/all/vendor?

Or why not just leave out the lib dir altogether?

core/Drupal - core PSR-0 components that are not in a core module
core/libraries - stuff in any top level namespace other than Drupal (e.g. core/libraries/Symfony)
core/tests/Drupal - tests for non-module core

This would give us:

core/Drupal/Core/Cache/CacheBackendInterface.php
core/libraries/Symfony/Component/ClassLoader/UniversalClassLoader.php
core/modules/entity/Drupal/Module/entity/EntityInterface.php
sites/all/modules/kitten/Drupal/Module/kitten/KittenInterface.php

Modules (unlike core) are not supposed to ship with vendor libraries - these should be under sites/all/libraries (in their own namespace).

Crell’s picture

jbrown: We went over leaving out /lib above. Let's please not rehash that.

"libraries" is the more generic term of 3rd party "stuff". "vendor" is specifically for PSR-0-loading 3rd party PHP stuff. There will still be things in libraries, like TinyMCE or jQuery Cycle, that have nothing to do with PSR-0 at all.

Crell’s picture

Status: Needs work » Needs review
FileSize
453.25 KB

Now with better comments.

jbrown’s picture

This looks really great.

There is some inconsistent capitalisation of "component" in core/lib/Drupal/Component/README.txt:

Drupal Components are independent libraries that do not depend on the rest of
Drupal in order to function. Components MAY depend on other Components, but
that is discouraged. Components MAY NOT depend on any code that is not part of
PHP itself or another Drupal component.

Does "component" and "subsystem" need to be capitalised?

Should there be information in the README.txt files saying that these directories should follow PSR-0?

Is the intention that tests for core/includes should be in core/tests instead of core/modules/simpletest/tests ?

In core/lib/Drupal/Core/README.txt

Code in the Core namespace represents

should be

Code in the Drupal\Core namespace represents
sun’s picture

@Crell: Please consider to add this to your .gitconfig:

[diff]
  renames = copies
jbrown’s picture

FileSize
28.94 KB

Rerolled #86 with renames = copies for easier reviewing.

Crell’s picture

sun: Oh wow, I never heard of that option before. Sweet! (Thanks for the reroll, jbrown!)

If someone wants to reroll with the comments from #87 before I do, feel free.

Not sure about tests for /includes at the moment, other than I hope most of /includes goes away and moves into /lib/Drupal anyway.

Crell’s picture

FileSize
29.44 KB

Minor language tweaks, added a new readme file in core/lib to specify PSR-0, and rerolling with renames=copies.

Can haz RTBC plz?

tstoeckler’s picture

jbrown:

We can't have both

core/lib
core/libraries

and then

Crell:

jbrown: We went over leaving out /lib above. Let's please not rehash that.

Actually, I think jbrown is right. Core is currently not following its own practices by lumping jquery and friends in /misc instead of /libraries, which is a well-accepted standard in contrib. In theory we would already have had a /core/libraries folder for a while.
And having /core/lib and /core/libraries is a super-major WTF!!
So I think this does needs some more discussion/thought.

pounard’s picture

Then let's rename "libraries" "assets" or "resources" or "public_resources".

I agree with Crell, let's RTBC and put the "revisit later" tag.

tstoeckler’s picture

Well this patch is 50% (/insert variable large number here) moving things around, so if there's a good chance, we'll be moving the same things around again...
...but I digress. It wouldn't be the end of the world either. :)

Crell’s picture

Status: Needs review » Needs work

Resetting testbot. Testbot, you suck.

Crell’s picture

Status: Needs work » Needs review
FileSize
29.44 KB
msonnabaum’s picture

Status: Needs review » Reviewed & tested by the community

All of this look great to me.

pounard’s picture

Same for me.

plach’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/core/includes/cache.inc
@@ -8,20 +8,20 @@
+ * Classes implementing Drupal\Core\Cache\CacheBackendInterface can register themselves

Wrong comment wrapping.

+++ b/core/tests/README.txt
@@ -0,0 +1,3 @@
+This directory contains test case code for Drupal core Components and Subsystems.

Wrong comment wrapping.

+++ b/core/tests/README.txt
@@ -0,0 +1,3 @@
+Test classes should mirror the namespace of the code being tested. Supporting
+code for test classes is allowed.

This point in not completely clear wrt to the issue summary: supporting code == test modules?

+++ b/core/vendor/README.txt
@@ -0,0 +1,6 @@
+Code in this directory MAY be licensed under a GPL-compatible non-GPL license. If

Wrong comment wrapping.

-25 days to next Drupal core point release.

pounard’s picture

Wrong comment wrapping.

Oh god only knows how much I hate those automatic reviews (and yet again there is no proof to me she exists). It doesn't make anysense to just put those automatically formatted reviews without an once of any human common sense in it.

This point in not completely clear wrt to the issue summary: supporting code == test modules?

For that I agree, but I don't think it is a commit blocker. Still RTBC in my opinion.

plach’s picture

Oh god only knows how much I hate those automatic reviews (and yet again there is no proof to me she exists). It doesn't make anysense to just put those automatically formatted reviews without an once of any human common sense in it.

It is not automatic: it's me reading the code, willing to understand it, spending time on it. If the only thing I cand find is unrespected coding standards kudos to the author, but the patch needs work. It will be a much greater waste of time to open a minor issue to clean the code up. We have coding standards for a reason.

pounard’s picture

Yes, true, I'm sorry sometime some of them are just annoying, yours are real after reading it again. Just as a note: "comment wrapping" is as much clear to me as the sentence you don't fully understand in the comments themselves. It's overlapping the 80 chars limit (I took me to copy paste it into an editor to understand, really).

EDIT: Oh my.. I'm bikeshedding again, sorry.

plach’s picture

Just as a note: "comment wrapping" is as much clear to me as the sentence you don't fully understand

I'll keep it in mind :)

Crell’s picture

plach: Seriously? I have my IDE set to mark the 80 character point. I didn't see anything going past by more than a pixel. If it's not lining up for you, can you reroll it?

Also, supporting code is extra temp classes and such. If any code in Components or Core requires a "real" module in order to be tested, it means someone screwed up their architecture and they need to fix that first.

plach’s picture

Status: Needs work » Needs review
FileSize
1.76 KB
29.5 KB

plach: Seriously? I have my IDE set to mark the 80 character point. I didn't see anything going past by more than a pixel. If it's not lining up for you, can you reroll it?

The last two were off-by-one errors ;)

plach’s picture

Status: Needs review » Reviewed & tested by the community

Just rerolled, back to RTBC. No commit credits please.

pounard’s picture

Thanks platch, RTBC now for me too :)

Crell’s picture

Thanks guys!

catch’s picture

Title: Proposal for unified namespace organization » Change notification: Proposal for unified namespace organization
Priority: Major » Critical
Status: Reviewed & tested by the community » Active

Alright, committed this. Hopefully this will unblock the dbtng patch and others.

Needs a change notification (we can probably update the PSR-0 one).

sun’s picture

+++ b/core/lib/Drupal/Core/README.txt
@@ -0,0 +1,6 @@
+Code in the Drupal\Core namespace represents Drupal Subsystems provided by the
+base system.  These subsystems MAY depend on Drupal Components and other
+Subsystems, but MAY NOT depend on any code in a module.

As a Next Best Step™ I'd like to see how we're going to inject the database subsystem, variable subsystem, and user/session subsystem dependencies into Drupal\Core\Cache\DatabaseBackend in order to actually fulfill the promise stated here.

Those dependencies only exist for Drupal\Core\Cache\DatabaseBackend.

Drupal\Core\Cache\InstallBackend extends DatabaseBackend, but is a bit more tricky in terms of dependencies, as it tests directly for a hard-coded class_exists('Database').

Lastly, FWIW, my existing D8 dev site survived for some reason. I expected a total breakage and fatal error, so this is good news. That said, I don't really understand why it survived and how it was able to cope with this major change.

Crell’s picture

It survived because right now the Symfony class loader does no caching. So classes moved, the root locations in the class loaded moved, poof, nothing to worry about. PSR-0 is much less fragile than the D7 registry.

I want to refactor the DB layer to remove, or at least reduce the use of, of the Database static class. First we get it into PSR-0, then we do refactoring. There's a separate issue for that which needs reviews. :-) That should help with pulling apart the various dependencies and injecting them cleanly. We will likely need a proper Service Locator or Dependency Injector to do it "right". We were discussing that in this morning's WSCCI meeting with Fabien as well. The log should get posted soon.

RobLoach’s picture

Somewhat unrelated to this discussion, but I've uploaded a patch at #1085590: Update to jQuery UI 1.9 which moves jQuery UI into core/vendor/jquery/jquery-ui.

Crell’s picture

I'm not sure if we want to do that for non-PHP code. Maybe we do, but we shouldn't assume it as a given, especially as there is talk of using Composer to download stuff in /vendor rather than having it in the repository.

RobLoach’s picture

I'm not sure if we want to do that for non-PHP code.

I've talked with the Composer folk and they said it's fine to use on non-PHP code. We just obviously arn't using the PHP autoloading stuff for it.

There is talk of using Composer to download stuff in /vendor rather than having it in the repository.

Using /vendor would work. Composer lets you change the vendor-dir, so either one isn't a problem. As for downloading Symfony stuff via Composer, we might like to move that to #1424924: Use Composer for updating Symfony components (without removing Symfony code from repo).

tstoeckler’s picture

Cross-referencing my comment over there: #1085590-125: Update to jQuery UI 1.9

klonos’s picture

Let me also state that having /core/lib/, /core/vendor/ is not in tandem at all with what's been going on for quite some time in contrib when it comes to 3rd party libraries. While we now have for core:

...
/core/lib/
/core/modules/
/core/themes/
/core/vendor

...for contrib we have:

/sites/all/libraries/
/sites/all/modules/
/sites/all/themes/
/sites/*/libraries/
/sites/*/modules/
/sites/*/themes/

I did read through the issue and saw that at some point there was mention of a discussion where it was decided to leave /core/lib/ alone (?) and and add a "revisit later" tag here about perhaps considering /core/libraries/, but I see no such tag added.

I don't see why since we went through the whole trouble to move all core stuff under /core/ we can't also change /core/lib/ to /core/libraries/.

I understand the reason why we went with "vendor" since we are not dropping/changing /core/lib/. I also understand having both that and /core/libraries/ is a WTF. What I honestly still cannot grasp is why there's need for both /core/libraries/ (or /core/lib/ or whatever) and /core/vendor/(?) when we can simply have one of them.

  • As a core or contrib maintainer or even simply a long-time drupalist, I wouldn't have much trouble seeing the new /core/libraries/ instead of /core/lib/.
  • As a newbie, I would simply not understand why jQuery isn't under either one of these directories (whichever we end up choosing). It is a js library after all.
  • As a newbie, "vendor" would surely strike me as yet another Drupal-specific thing (this term being used in a couple of other not-so popular open source projects still wouldn't change this fact).

I guess what I'm trying to say is that I believe that the goal here is to find a common directory for all 3rd party stuff (IOW non-Drupal things) - not one for 3rd party libraries, one for 3rd party tools, one for 3rd party utilities ...you get the point. I also believe that people would be more confused by the existence/purpose of the "vendor" directory rather than by a "libraries" directory holding other things than only libraries.

Since "libraries" is a commonly accepted term and one that's been accepted by most contrib (the ones using Libraries API) for such things, I think we should go with that. Plain and simple.

RobLoach’s picture

@klonos Since "libraries" is a commonly accepted term and one that's been accepted by most contrib (the ones using Libraries API) for such things, I think we should go with that. Plain and simple.

This is actually a Drupalism, and is only used within the Drupal community. The most commonly accepted term by the rest of the world is "vendor". Libraries API for Drupal 8 will most likely evolve to adopt that standard.

donquixote’s picture

@klonos,
libraries has a different purpose than lib/vendor.
sites/all/libraries is for a mix of prededural, javascript, images, whatever downloaded from external sources.
lib/vendor is for classes in psr-0, shipped with core or with contrib.

If we want to mix these two, that would be a new discussion.

pounard’s picture

actually the legacy "libraries" folder should be named something like "resources" or "assets" probably.

klonos’s picture

@Rob Loach:

The most commonly accepted term by the rest of the world is "vendor"

Care to explain what exactly you mean by "the rest of the world"? Any *popular* projects/software using that? The reason I'm emphasizing on "popular" here is because if we go with something like "vendor", it needs to make sense for newcomers looking at the code and file structure we use from the first glance. I for one am not aware of any project using this in their directory structure. That's why I say that it strikes me as odd.

@donquixote:

...libraries has a different purpose than lib/vendor.

So we are still keeping js libraries under /core/lib/?

@pounard: "assets" sounds like a place where images could be kept - actually under /assets/images/. If the images are from different, specific "vendors", then we should use their names as subdirectories:

/core/assets/images/*.* <- non-specific vendor images
/core/assets/images/[vendor_name_1]/*.*
/core/assets/images/[vendor_name_2]/*.*
...

Pretty much like we do with Libraries API now:

/sites/all/libraries/*.* <- non-specific vendor libraries (custom code for example)
/sites/all/libraries/[vendor_name_1]/*.*
/sites/all/libraries/[vendor_name_2]/*.*
...

RobLoach’s picture

@klonos Care to explain what exactly you mean by "the rest of the world"? Any *popular* projects/software using that?

@donquixote sites/all/libraries is for a mix of prededural, javascript, images, whatever downloaded from external sources.

I've talked with the Composer people and a couple Symfony folk and they said that "vendor" can be used for PHP, JavaScript, or anything. As long as there is package information available (composer.json), then we don't have to worry.

@klonos As a newbie, "vendor" would surely strike me as yet another Drupal-specific thing

Users shouldn't mess with /core/vendor as that's in the /core folder and should not be touched. In order for the user to update a vendor, they should rely on Drupal core to do it for them, or use a supporting module. An example is jQuery UI being updated via jQuery Update or the jQuery module. The talk of sticking jQuery UI in /core/vendor/jquery/jquery-ui though should be over here.

@klonos So we are still keeping js libraries under /core/lib/?

/core/lib is meant for Drupal 8 core's PSR-0 supported autoloading PHP classes.

@pounard actually the legacy "libraries" folder should be named something like "resources" or "assets" probably.

Yeah, Libraries are different than vendor packages. All vendor packages provide version information, dependencies, and a means of upgrading itself. Libraries, however, arn't required to provide that information. Vendors are also usually placed in vendor/[vendor]/[package] too, rather then Libraries' libraries/[package].

sun’s picture

Sorry, but let me repeat it once again, and hopefully for the last time:

/vendor/jquery (or /lib/jquery) and /libraries/jquery have zero in common.

The former is a hard dependency. A library shipped with the code that you're using. The dependency not only applies to the mere library, it also applies to its version, variant, and API. The library cannot be replaced, updated, or changed in any way without performing full regression tests for all consuming code, and potentially adjusting all consuming code to be compatible with the changed API. Trivial example: Change a public property in a Symfony component into a protected or private. Your system blows up.

The latter is Libraries API's multi-dimensional concept of introducing a single dependency for potentially multiple consumers. Every single consumer has to care for the exact version and variant of the actual library in place. As such, multiple consumers are able to depend on the same library, regardless of its exact incarnation. Each consumer may continue to work after replacing, updating, or changing the library (API). Three may continue to work, but one may not. In that case, the particular consumer has to be updated to account for the new version/incarnation in its specific integration files. In any case, the system actively allows and encourages to replace, update, and downgrade "external" libraries whenever necessary. As long as all currently available library versions/incarnations are supported by every consumer, the system does not blow up.

These two are completely different things and MUST NOT be mixed.

klonos’s picture

Daniel & Rob, thank you both for taking the time to explain things. I have a much better idea of what's involved.

Crell’s picture

klonos: Also see comment #26 above, and the referenced blog post.

tstoeckler’s picture

Expecting a bashing from sun here, but I absolutely disagree with:

/vendor/jquery (or /lib/jquery) and /libraries/jquery have zero in common.

There are two competing facts that we somehow have to get by with:

  1. Drupal core must ship with a certain version of jQuery and Drupal will ship with exactly that version for the entirety of its major version cycle.
  2. Contrib projects may need a later version than the one shipped by core.

So in a perfect world, it would be fine to simply drop a new version of jQuery in sites/all/libraries/jquery as long as you also provide compatible versions of core scripts, i.e. sites/all/libraries/tabledrag etc. (whether the core "libraries" should go into the same folder as true, external libraries or not is OT for this issue).

That's why a core/libraries makes sense for CSS/JS conceptually.

It's the same as if someone writes a node-7.x-2.x module in contrib. You just put it in sites/all/modules and also put a new version of comment.module in sites/all/modules.
Or if, in theory, only node.module would depend on Symfony and none of our base system would rely ont it, then you could drop a new version of Symfony in sites/all/vendor (again, bikeshed vendor/libraries/...) and provide a new node.module as well. And everything should work fine.

Conceptually it's the same thing.

xjm’s picture

Issue tags: +Needs change record

Can someone clarify the current status of this issue? The title indicates it needs a change notification, but there's lots of discussion after. Does the subsequent discussion change what would go in the change notification? And, if not, can we spin the followups into followups? Thanks!

tstoeckler’s picture

I think the issue summary is still pretty accurate. I wasn't involved in this patch, so there might be something missing, but from glancing over I think it's fine.

The later discussion really does deserve it's own issue, as I had said above, so I went ahead and opened #1431804: Bikeshed 'lib' vs. 'libraries' directory. See you there!

Crell’s picture

The next steps on this issue are for me to get around to updating d.o documentation with the content of the summary, and then finishing up the issue for the location of classes in modules. (Which, really, boils down to "do we use PSR-0 for modules or not?") I don't think there's anything else to do here.

sun’s picture

On the last weekend, we discussed this once more in person, and also had @fabpot and @Dries with us, who both were a little confused about the current incarnation. Apparently, none of both participated in this issue. ;)

With regard to framework classes shipped with Drupal core, the additional /lib and /vendor directories were perceived as "nonsensical." That is, because Drupal is a vendor, and Symfony is a vendor, too. And the mere vendor name as top-level directory is more than sufficient to differentiate between vendors.

In turn, the concrete change proposal was to simply drop the additional, outermost "category" directories; i.e.:

/core/lib/Drupal/...     becomes /core/Drupal
/core/vendor/Symfony/... becomes /core/Symfony

KISS :)

Crell’s picture

That was a last minute proposal as we were heading out the door. We didn't actually have consensus on that. I am still not a fan myself. :-)

On the subject of modules, we listed 4 physically possible approaches (with their main criticism):

0) $moduledir/ClassName.php => \Drupal\$modulename\ClassName [This is not PSR-0 compatible.]
1) $moduledir/Drupal/$modulename/ClassName.php => Drupal\$modulename\ClassName [This mixes procedural code and CSS/JS files in with PSR-0 class files.]
2) $moduledir/lib/Drupal/$modulename/ClassName.php => Drupal\$modulename\ClassName [This has an extra directory.]
3) sites/all/DrupalModules/$modulename/ClassName.php => DrupalModules\$modulename\ClassName [This is the most radical change, mixes procedural code and CSS/JS in with PSR-0 class files, and has no way to include out-of-namespace code. It's also the most likely to blow up in our faces.]

I favor Option 2 (which is what's in the summary of this post right now), but could live with option 1 if people are really that opposed to the extra organization directory.

I would also note that I am not sure what happens to tests directories and such, since we want to move those to PSR-0 classes too. (Again, see the summary.)

Fabien suggested we look at Flow3/Typo3 as well, since they're more application-y while the other things we looked at before are framework-y.

[Edit: Fix rather important typo...]

tstoeckler’s picture

I like 2) as well, also because it is in-line with #129, too. And the downside, that it mixes PSR-0 with procedural, JS and CSS, is technically correct, but in reality, once people get used to it, they will know that "Drupal" is the dedicated directory for PSR-0 classes. Just like many contribs already have the standard of putting all their classes in an own /includes.
I do see a theoretical benefit of the extra 'lib' directory in that respect (because 'lib' is clearer than 'Drupal'), but that is definitely outweighed by the downside of an extra directory / extra nesting, IMO. And again, it's just a matter of habit.

Crell’s picture

tstoeckler: I think you are talking about 1). 2) is the one with the lib directory, which core has currently.

tstoeckler’s picture

Oh yes, sorry. Even though I'm a pretty big nerd, I still count starting at 1, sorry! *stupidme*

donquixote’s picture

#130
What happened to
4) sites/(all|default|*)/modules/(contrib|custom|*)/$modulename/lib/ClassName.php,
for Drupal\Module\$modulename\ClassName.php ?

This is the best we can get for module DX, imo.
(for core, I don't care atm)
Maybe not strictly PSR-0, but I think we had a significant number of people tell us that they don't care. And should be easy enough to implement on top of the PSR-0 engine.

This mixes procedural code and CSS/JS files in with PSR-0 class files.

Not cool, imo (for any proposal).

---------

I want my module directory structure to look like this,
http://drupalcode.org/project/crumbs.git/tree/716b9a424997638460f2a6e402...
just with namespaces.
Any "Drupal/Module/$modulename" directories added in between, or the "lib" directory removed, would hurt my DX big time, much more than PSR-0 will ever help me.

----

EDIT:
Looks like 0) is pretty close to 4), just that the lib folder is dropped. Why??

pounard’s picture

Any of 1 or 2 is good IMHO. About 1, the fact the namespace root is the module folder indeed mixes it up with procedural and other files is true, and that's why I like more the option 2. I'm not afraid having huge folder hierarchies, that's already what I do with my own modules, and that makes sense more that's annoying me.

2++

donquixote’s picture

Drupal\$modulename

Now we need reserved subnamespaces for anything in Drupal\* that is not a module.
I don't like this.

pounard’s picture

Reserved subnamespaces are Core and Component, I guess we can live with that...

donquixote’s picture

Btw, for 1) and 3), big problem for projects that ship with submodules in nested subfolders.
Also, we probably still want a separation into contrib and custom.

Of anything proposed in #130, I'd rather choose 2), although I'd much much much prefer #134 4).

neclimdul’s picture

@donquixote We're trying to provide PSR-0 and none of your suggestions are PSR-0 compatible. This is the fundamental problem with 0) and your lib\Classname.php suggestion. PSR-0 is a standard and like it or not we should support it and not maintain some goofy NIH autoload solution. That said if you want to implement your own or port the current autoloader in contrib have at. The complexity just doesn't make sense in core and has been a constant headache.

As for 3), if it wasn't clear this option sets all the modules in a PSR vendor root of sites/X/ and the namespace is DrupalModules\modulename. This is actually the system that has the biggest weirdness for submodules and non-Drupal namespaces IMHO. Also as you mentioned, it is a bit weird in its mixing of PSR and unrelated static resources. Its novel but I think we are still too procedural and frameworkless to commit to this even with all we want to do in d8.

As for 1) and 2) there really isn't that big of a burden on modules with submodules. Discussing with merlinofchaos who has a unique perspective on maintaining projects with tons of objects with lots of submodules, the fact of the matter is that you can mostly maintain all your objects in the parent module PSR-0 space because that's relevant to the project as a whole. Even so, the relatively short depth we've narrowed this to mean it really is not that big an imposition on DX.

And if you're comfortable with 2) let me make it clear the real difference with 1) is negligible and boils down to emulating what we choose to do with /core. If we keep /core/lib as the drupal core vendor root then sites/X/modules/module/lib should be the vendor root. if /core becomes our core vendor root, then sites/X/modules/module should be our vendor root. Its not really anything to get religious about.

donquixote’s picture

@donquixote We're trying to provide PSR-0 and none of your suggestions are PSR-0 compatible. This is the fundamental problem with 0) and your lib\Classname.php suggestion. PSR-0 is a standard and like it or not we should support it and not maintain some goofy NIH autoload solution.

PSR-0 was designed with a specific use case / scenario in mind. Drupal is far away from that.
PSR-0 has some really good ideas, and we should follow these as much as possible.

But I have yet to see an explanation why PSR-0 is important enough to screw our nice folder structure and DX.

Everyone says "interoperability", but this is not convincing for me. If anyone wants to use non-PSR-0 Drupal code within a PSR-0 compliant framework, there are still plenty of ways to do so: Extend the autoloader (or add one), create symlinks, download only the /lib/ part of the module (and the rest somewhere else), etc. Imo, it is the job of a framework to provide a hookable autoloader, where custom autoload implementations can be registered for specific prefixes or namespaces.

And the specific variation of PSR-0 that is implied in 4) just makes so much sense, that framework providers should consider to implement it by default.

And if you're comfortable with 2) let me make it clear the real difference with 1) is negligible and boils down to emulating what we choose to do with /core.

Ah.. sorry. I was wrong on that one!
Actually I prefer (1) then :)
Anything that does not impose on a module's filesystem location, and does not mix psr-0 with procedural and css/js/images. The "Drupal" subdirectory is sufficient, we don't need to hide that in /lib/.

---------

Btw, we could easily combine options (1) and (4).
This way, module maintainers can choose to write "interoperable" and cross-namespace code and put it in $moduledir/Drupal/, or 3rd party libraries and put them in $moduledir/HTMLPurifier/, or they can write drupal-specific code and put it in /lib/.

so,
every $moduledir/ maps to the root namespace
every $moduledir/lib maps to Drupal\Module\$modulename, or Drupal\$modulename
Additional mappings for a specific module can be added in the info file. Such as
$moduledir/3rdparty/stuff/HTMLPurifier/..
Or.. should we have a $moduledir/vendor/.. for 3rd party stuff?

donquixote’s picture

That said if you want to implement your own or port the current autoloader in contrib have at. The complexity just doesn't make sense in core and has been a constant headache.

http://drupal.org/project/xautoload
Done that for D7 already (w/o namespaces).
(the autoload guys didn't reply, so I pulled my own)
If the /lib/ClassName.php thing doesn't make it into D8, then I will produce a D8 version of xautoload, doing exactly that.

Obviously, a D8 version would need support for namespaces, and reuse symfony's autoload cache. I can't imagine what is so difficult about that - but maybe I'm wrong.

tstoeckler’s picture

Well, PSR-0 or not, the idea is to have an Autoloader that doesn't rely on the DB.
(Note that any way you put it, you always need to know if a module is enabled or not, and we currently store that in the DB, so no DB here really means almost no DB.)
And with a file structure like that:

lib/SomeViewsPlugin.php
lib/SomeEntityController.php
lib/SomePanelsContentType.php

There really is no way for any autoloader in the world to know that the actual namespaces are

\Drupal\Views\DisplayPlugin\SomeViewsPlugin
\Drupal\Entity\Controller\SomeEntityController
\Drupal\Ctools\ContentTypes\SomeContentType

So given that we want to organize our code into neat little namespaces, you really have to somehow nest your files according to your them, whether you do PSR-0 or not.* But since PSR-0 does just that and is somewhat of an industry standard, it would be super weird to use something other than PSR-0.

I think if it were possible no one (or at least not me) would argue against the above file structure if we could have a mean, lean, DB-less autoloader at the same time. But we can't. So it's not some foreign standard that we force onto our code and hurt DX simply for the standard's sake, PSR-0 or not is a technical debate. And if we can have a mean, lean, and DB-less autoloader for it, I'll happily wade through a bunch of nested directories for that.

(Disclaimer: I'm not involved with WSCCI nor a PHP hacker, so the above might be complete BS. But I think I got it right. :) )

< for completeness' sake >
*Instead of nesting directories you could of course pre-prend the namespaces in some fashion to the filename. In the above example, that might not actually look too bad, but not only would that be a bad case of NIH, more importantly it would be even worse DX if you have 10s and in some cases 100s (if core is converted more and more) of files with almost the same filename in the same directory. I mean, that is why PSR-0 isn't just that.
< / for completeness' sake >

donquixote’s picture

@tstoeckler (#142),
I doubt that PSR-0, or the nesting within $moduledir/Drupal/$modulename/ instead of just $moduledir/lib/ does help any bit with avoiding database lookups.

If a class is provided by a module, but that module is disabled, the class should not be available. (Or should it?) Thus, we still need to know which modules are enabled. Also, if a module is in two locations, we need to know which of them is the "active" one.

Also, starting with a class name, it would be dead inefficient to look in each module folder separately. Thus, we need some way to tell from the namespace, which module folder we should look in. So again, we need to know about modules (and ideally, also know which of them are enabled or not).

There are actually two ways to avoid database lookups, but these have nothing to do with PSR-0.

  1. A file-based cache for autoload locations. This is what symfony 1 does, afaik (and probably symfony 2 too). This can cache all autoload locations, or just the most frequently used ones. The file could be stored somewhere in sites/$sitename/cache. This cache would be flushed whenever a module is enabled or disabled or updated, or whenever someone requests it.
  2. A file-based cache for module locations and module enabled status. Same story, we skip the database.

These two things work perfectly fine with any of the proposals in #130 and #134, including non-psr-0 (4) and (0).

EDIT:
I should add, imo it is quite important to "hide" the classes provided by inactive modules, providing a safe way to avoid broken code from being included.

donquixote’s picture

Actually that is a question worth discussing:
Should classes defined by disabled modules still be available?
Or, more precisely,
a) Should classes defined in the filesystem location of a disabled module still be available?
b) Should classes namespaced with the module name still be available?

(important distinction for cross-namespace or 3rd party namespaced classes)

EDIT:
My answer would be:
- By default, module-provided classes should only be available if the module is enabled.
- However, a module/project should have the possibility to provide some classes that are always available. E.g. database plugins or maybe some 3rd party stuff. These might be namespaced with something other than a module name.

EDIT II:
E.g.
- Drupal\Module\$modulename - only available if module is enabled.
- Drupal\Boot\$stuff - always available.
("Boot" is for "boot extensions". Name to be discussed.)

neclimdul’s picture

PSR-0 was designed with a specific use case / scenario in mind. Drupal is far away from that.

Thanks donquixote but we've talked to the PSR-0 and our usage as proposed in 1 and 2 was considered and is the entire reason for the multiple vendor root clause.

every $moduledir/lib maps to Drupal\Module\$modulename, or Drupal\$modulename

This either is a fundamental lack of understanding of PSR-0 or that we do not want to have our own Drupalism autoloader in core. You are required to have the entire namespace path on disk. Also, multiple registration methods are far more WTFy and a DX burden then a couple directories.

And re all the discussion around avoiding DB looks, that's a intended side effect but I think its more important we have a consistent namespace strategy. But, since there are serious pushes for module lists prior to database and with a variable you can turn on symfony's APC variable cached autoloader that will bypass both of these needs there is a serious gain in not hitting the database because even on non APC sites in D8 its likely with the PSR-0 models proposed that classes can be loaded without touching the database without any Drupalisms at all. And if we want some magic cache thing(which I think is a terrible idea and pre-optimizing something before there's a problem) we can do that within the confines of 1) and 2) by extending symfony's autloader instead of designing our own standard.

Actually that is a question worth discussing:
Should classes defined by disabled modules still be available?

No this is entirely off topic.

sun’s picture

And if you're comfortable with 2) let me make it clear the real difference with 1) is negligible and boils down to emulating what we choose to do with /core. If we keep /core/lib as the drupal core vendor root then sites/X/modules/module/lib should be the vendor root. if /core becomes our core vendor root, then sites/X/modules/module should be our vendor root. Its not really anything to get religious about.

@neclimdul is spot-on with that, and this is also what it boils down to for me. In other words: consistency++

Personally, I'm heavily annoyed by the additional /lib subdirectories after merely working a couple of hours with them and constantly having to cd/click through them.

I'd be more than happy to close this chapter of epic discussion by getting a compromise on 1) of #130, which would then also mean to do #129 for core components.

The structure seems to make most people/perspectives happy, is PSR-0 compatible, and merely has one minor itch with regard to procedural files and static assets being on the same level as the namespace mount point, but thus far, no one was able to actually come up with an example that would render the proposal invalid or would be huge show-stopper, and which couldn't be easily resolved differently.

Crell’s picture

There are at least 3 possible "magic cache things" we wouldn't have to invent ourselves that are already in use by other PHP projects: APC indexing, phar, and file compilation. All of those someone has already tried using, are in use, and we wouldn't have to invent a Drupalism for. And all of those are straightforward to investigate and leverage if we're using PSR-0.

donquixote: Really, from your comments here it sounds like you do not understand PSR-0, nor the reasons for it. Instead, you're just taking this thread off topic yet again. We need this issue resolved, not bikeshedded even further. Please, for the sake of Drupal, stop babbling and taking issues off topic.

As far as I see it, there are only two viable options: We have a lib directory in core and in a contrib module, or we have it in neither place. I'd prefer the lib, as it separates from /tests/ and procedural code, and the original summary covers all of the various moving parts although it seems people are no longer reading it. It WAS a thought-through holistic proposal.

As far as reserved namespaces, I originally had a Modules namespace to separate off module code, but people objected to the "extraneous" directory there. Instead, modules and themes are \Drupal\lower_case_string, while Core-reserved stuff is Drupal\UpperCaseThing. That's nice and simple and avoids any new collisions.

donquixote’s picture

Thanks donquixote but we've talked to the PSR-0 and our usage as proposed in 1 and 2 was considered and is the entire reason for the multiple vendor root clause.

I have questioned this repeatedly, and noone has ever given an explanation other than "it is standard" or "it has been decided", or a vague statement of "interoperability". If someone would care to explain, or point to another discussion, I would be much less annoying.

Besides, my post in #143 was a rebuttal of the argument in #142, and nothing else. It can still be PSR-0, just that has nothing to do with avoiding database lookups (which I think you agree).

donquixote’s picture

Ok then, let it be #130 (1).

I still have plenty of time to decide whether there should be an xautoload D8 in contrib to implement the not-really-PSR-0 $moduledir/lib/ClassName.php thing, and how that would play together with the symfony autoloader + cache.

EDIT:
https://groups.google.com/forum/#!topic/php-standards/PhgdC7djii8

I do not like any of these options. :-(

(Side note: I do recall objecting to PSR-0 when it was first published
on the grounds that it ignored the needs of CMSes and pretty much
anything but PEAR/Zend-style component frameworks. We're now seeing
why. )

This lets me sympathize a lot more with what is being discussed here. At least someone (Larry) did try.
(though I would still rather break psr-0, or make a psr-1, but this is not up to me)

RobLoach’s picture

I updated the patch at #1424924: Use Composer for updating Symfony components (without removing Symfony code from repo) to use Composer's ClassLoader to load our vendor packages. From there, we can register the namespaces and paths of contrib modules as we wish (probably a follow-up patch).

effulgentsia’s picture

Sorry to be jumping in late here, but if I'm understanding #121.5 and #122 correctly, then we're going to end up wanting core/libraries at some point for some of the stuff currently in core/misc. And I'm not too keen on having a core/lib meaning something substantially different from core/libraries. Why do we need a core/lib folder? From what I can tell, the only valid subfolder that can possibly ever be in it is Drupal. Any reason not to move that directly to core/Drupal?

Also, why is Drupal/Component in core/lib? Unless I'm misunderstanding the comments about Symfony and ZF2, shouldn't that be in core/vendor? Note, I'm just talking about Drupal/Component here; Drupal/Core should remain in core/lib, or if you agree with my first paragraph, in core.

Crell’s picture

Drupal Components are still part of Drupal proper. They're still produced BY Drupal. They're just the bits that are designed to function outside of Drupal should one so desire. From a Drupal site's point of view, they're not 3rd party code.

effulgentsia’s picture

Ok, I think I get it. With Symfony, there's a clearer separation between the framework (https://github.com/symfony/symfony) and the distribution (https://github.com/symfony/symfony-standard). So the idea is you build your web application starting from the distribution, and adding the framework's "src" contents into a "vendor" folder within your copy of the distribution, which makes sense since in this case Symfony is a vendor for your custom web application. With Drupal, however, core includes both the components and the CMS application itself, so calling Drupal a vendor of itself would make less sense.

What about #151.1? I'm still not clear what advantage core/lib/Drupal has over core/Drupal. According to #26, the modern PHP community has not yet standardized this, so why not go with what's shorter and doesn't require someone to understand the difference between lib and libraries?

donquixote’s picture

Sorry for yet another note.
I hope that talk about autoloading performance is not off-topic.

I did have a look at the autoload implementation used in D8, provided by Symfony.
http://drupalcode.org/project/drupal.git/blob/78d1b495c7cf192e2e140eaf02...

One detail that I am unhappy about: (line 213 ff)

<?php
    public function findFile($class)
    {
        [..]
            foreach ($this->namespaces as $ns => $dirs) {
                [..]
                foreach ($dirs as $dir) {
?>

I assume that we are going to register one namespace per module, with one directory each.
This is going to be a really big loop, that will run for every class that is not in the APC cache (which might not be available at all).
There are alternative implementations imaginable, that perform a lot better with a big number of namespaces.

pounard’s picture

We need to provide an autoload cache at some point, sure.

donquixote’s picture

My point is:
The task of "look up one class that is not (yet) in the cache", can be done a lot faster than Symfony does it. So then the question is: Why are we so obsessed about using the Symfony class loader?

The Symfony class loader is inspiring, but does not provide the performance that we should aim for (imo).
Writing an autoloader is not really that complicated, especially if we have one to adapt from. (We could subclass the Symfony one instead, but the inheritance of UniversalClassLoader and ApcUniversalClassLoader makes this more difficult). In fact, I wrote one last night.

effulgentsia’s picture

Re #147:

As far as I see it, there are only two viable options: We have a lib directory in core and in a contrib module, or we have it in neither place.

I agree with the need for consistency in core and modules, which is why I think we need to get this worked out.

I'd prefer the lib, as it separates from /tests/ and procedural code

How does lib separate from procedural code any better than Drupal? #131 suggests that because 'lib' is clearer than 'Drupal', but is it? To whom? As far as /tests/, those could still be in a /tests/ folder either way, couldn't they?

effulgentsia’s picture

Re #129:

While I am suggesting that we remove /lib/, I'm still leaning towards keeping /vendor/. From what I can tell, that word actually is somewhat standardized at this point for meaning "code maintained in another project that our project is merely using", and according to #121, also implies some packaging standardization. As we grow to using components from more projects, I can see the value in having all those top-level vendor names inside a /vendor/ directory.

webchick’s picture

Title: Change notification: Proposal for unified namespace organization » Proposal for unified namespace organization
Status: Active » Needs review
Issue tags: -Needs change record

Given we've had another 50+ comments since #109 marked this effectively "fixed" it seems like we still have more work to do here. Worst-case, we can assign to Dries to make a call, but Alex's proposal in #157 & 158 makes me think we might be able to converge on a solution here that would bring the following benefits:

1) Helps assuage concerns from contributed module developers about the level of nesting of directories. Removes one level of nesting, which is about the best we can do and still conform to PSR-0.
2) Helps assuage concerns up above about abbreviating words (lib) in directory names. It simply removes it. Done.
3) Helps assuage concerns about the mental model conflict between the lib directory and the contributed module Libraries API's concept of libraries. Core has no such directory. #1431804: Bikeshed 'lib' vs. 'libraries' directory can be closed, and we're short one fewer soul-crushing bikeshed discussion.
4) Everyone who was in favour of keeping lib said they could live with not keeping it. Smells like an opportunity for consensus reaching to me. ;)

The one remaining concern that I could suss out above is that /lib/ would provide an additional hint that what we're looking at is a PSR-0 folder, but IMO a capital D on "Drupal" already does that. Nowhere else do we use ProperCasedDirectoryNames, so biz-am. There's your clue.

If there are no strongly heart-felt objections to this proposal expressed by Tuesday, I propose we Make It So™. Get your ya-yas out.

Crell’s picture

Title: Proposal for unified namespace organization » Change notification: Proposal for unified namespace organization
Status: Needs review » Active
Issue tags: +Needs change record

The separation is because with $moduledir as the PSR-root, the procedural files, JS files, CSS files, etc. are inside the PSR root. Now, in practice most modules that have only Drupal-namespaced classes will have all of those inside the Drupal directory, but that won't be all modules. Wheter should they put any other classes (that don't belong in a global vendor instead)?

As for tests, there's still ongoing discussion about what the proper namespace for tests should be. The Symfony convention, IIRC, would be Symfony/Tests/*mirror of the main namespace*. And those files are then placed outside of the production class root, so that when Components are split off they are *not* included, because the test frequently assume that multiple components are present for optional dependencies. (For instance, some parts of HttpKernel depend on EventDispather, but not all.)

If we want tests to remain in /tests/ inside a module, and we use $moduledir as the class root, then that implies all test classes are in a "tests" top-level vendor namespace. That is not desirable. If we do not, and put classes inside /tests/ anyway, then we have a class root inside a class root. The odds of that blowing up in our faces in unpredictable ways is high.

Splitting the roots to lib and tests neatly avoids both issues.

donquixote: You're prematurely optimizing. There's another issue that's already been linked to many times where we discuss optimizations such as precompiled files, phar, etc. Please stay on topic.

Crell’s picture

Title: Change notification: Proposal for unified namespace organization » Proposal for unified namespace organization
Status: Active » Needs review
Issue tags: -Needs change record

Bah, crosspost. My points above still stand.

(Also, lib vs. libraries is only an issue because we went with lib instead of src above. Just pointing that out.)

tstoeckler’s picture

Wheter should they put any other classes (that don't belong in a global vendor instead)? [sic]

Just like we (will) put external code outside of module directories in core, I think contrib modules should not ship with external code. It's basically the argument as with JS libraries. If that directory for contrib ends up being sites/all/vendors, sites/all/libraries or sites/all/src I don't really care.

If we want tests to remain in /tests/ inside a module, and we use $moduledir as the class root, then that implies all test classes are in a "tests" top-level vendor namespace.

While the above discussion about "lib or no lib" has been solely about taste and theoretical purity etc. this seems to be the first actual technical argument. (I might have overlooked something; this is not supposed to be judgemental in case there actually is a technical argument above. Regarding that, I personally don't view procedural code/JS/CSS in the class root as a technical argument, but I realize that is arguable.) Based on that I might withdraw my endorsement of removing lib. Hmm... (Also not saying that my endorsement means anything, I just wanted to point this fact out.)

effulgentsia’s picture

Status: Needs review » Active

Now, in practice most modules that have only Drupal-namespaced classes will have all of those inside the Drupal directory, but that won't be all modules. Wheter should they put any other classes (that don't belong in a global vendor instead)?

Ah, ok, this is what I was missing. I thought we were planning on registering *only* the Drupal namespace (or even Drupal\$modulename) to the module directory, and that any class outside that namespace would have to go into $moduledir/vendor or $moduledir/tests. However, if there need to be classes outside the module namespace that are neither tests, nor fulfill the requirements of a "vendor", then I now understand the purpose of $moduledir/lib. Unless someone can argue for why we shouldn't ever support non-vendor, non-test, non-module-namespaced classes, or can propose a better way to support them, I'm tentatively sold on $moduledir/lib.

effulgentsia’s picture

Status: Active » Needs review
pounard’s picture

this seems to be the first actual technical argument

What?!

tstoeckler’s picture

OK, if my comment is more controversial than I thought it would be, then please disregard it. My intent was not to polarize this discussion. Sorry, I guess...
I agree with #163, which is pretty much in the same direction as what I was trying to say. Either way, I'll let you guys figure this out. Unsubscribe.

effulgentsia’s picture

My last thoughts on this for the night. One more thing I don't like about a /lib/ folder name is that not everything (maybe not even most things) that will go into it is a library, in any conventional sense of the term. For example, while we don't yet know whether block implementations will be classes or not, suppose for the moment that they will be, at least for some modules. So let's say the user module has a class Drupal\user\LoginBlock. This is application code, not a library, so it appearing in modules/user/lib/Drupal/user/LoginBlock.php is a bit of a WTF to me.

This is probably why Symfony and ZF2 use a folder named /src/ instead of /lib/. However, as per #30, /src/ wouldn't work for Drupal, because procedural code (e.g., user.module) is also part of the source code, and to a front-end developer, user.js is part of the source code too.

Brainstorming some alternate folder names: /oo/, /oosrc/, /oophp/, /psr/, /autoload/.

Alternatively, we could use no folder name for the Drupal namespace, as per #159, but address #160 by using a /lib/ folder for what's not in the Drupal namespace (but for whatever reason doesn't belong in /vendor/), since anything not in the Drupal namespace will probably actually be a library in some sense of the term. Seems to me that non-Drupal, non-vendor classes will be a fairly small edge case, so the few modules that need it can worry about the existential question of what /lib/ means, while the rest of us don't have to.

Finally, in #159.3, webchick states that core doesn't have a /libraries/ folder to worry about. It doesn't currently, but I don't want to rule out the possibility that we'll want to move, say, the vertical-tabs js and css files into a core/libraries/vertical-tabs folder. Or, if we end up adding a rich text editor (wysiwyg) to core (please, let's not start a debate here on whether we should, just consider it an example), maybe that will make sense to put into core/libraries/whatever (if the editor chosen doesn't comply with the packaging requirements needed to belong in core/vendor).

msonnabaum’s picture

I assume that we are going to register one namespace per module, with one directory each.
This is going to be a really big loop, that will run for every class that is not in the APC cache (which might not be available at all).
There are alternative implementations imaginable, that perform a lot better with a big number of namespaces.

No, it won't. You should step through the code to see what it really does. I addressed performance concerns with the autoloader here: http://drupal.org/node/1290658#comment-5444932.

effulgentsia’s picture

Re #167: I added my latest suggestion to #1290658-112: Move all module-provided classes to PHP namespaces (PSR-0 or similar), and autoload them without the registry [policy, no patch]. In short, we keep core/lib, since those really are library-level classes, but for modules, we auto-register $moduledir (not $moduledir/lib), so that module namespaced code can go in as shallow a folder as PSR-0 allows (and also to not force application-level classes into a "lib" folder), and we allow the module's .info file to add prefix/namespace mappings to "lib", "vendor", or wherever.

If people are in agreement with at least the "keep core/lib" portion of this, then I think we can close this issue, and continue the module-level discussion in the other issue.

Status: Needs review » Needs work

The last submitted patch, core_namespaces-1400748-105.interdiff.D0.patch, failed testing.

cosmicdreams’s picture

+1 for #169

hinikato’s picture

Hi, I would like to propose to use the core/libs/ instead of core/lib/ and core/libraries/ directories. And to use the vendors/ instead of the using vendor/ directory.

If the lib/ and the vendor/ names will be used let's rename also the following directories:
- modules/
- includes/
- themes/
- scripts/
- tests/

It seems like that using of plural names for the lib/ and vendor/ directories introduces some inconsistence. Please also notice that other frameworks can have some inconsistence too, so a wise decision should be used here.

hinikato’s picture

Please consider also of the issue http://drupal.org/node/1451602, because they can be related.

pounard’s picture

I don't see any good reason to put plural here, most other frameworks never put "vendors" but only "vendor", and "lib" is actually an abbreviation, and I don't know for english, but in french, we never do plural on abbreviations (except for minor execptions, also is true for acronyms and such). Plus, it depends on how you actually consider it to be, but I consider the Drupal library (without 's') being one single unified library, not a set of libraries.

hinikato’s picture

@pounard, I think it is not problem to use plural or singular names, I propose to introduce just one convention for all modules directories. If the singular names will be used, then the other directories should be renamed too, for example:
- module/
- misc/
- theme/
- script/
- module/
- lib/
- include/
- vendor/

What you think about that?

Please see also the Occam's razor page.

neclimdul’s picture

@hinakato please try to stay on topic. we're 170+ comments in. Can we not bikeshed the name and agree on how to build the reactor?

hinikato’s picture

@neclimdul, I have tried to create a new issue to describe my suggestions, but it was marked as duplicate: http://drupal.org/node/1436384. So I describe my suggestions here.

pounard’s picture

Changing all folders names of core right now would severely disturb backward compatibility, let's only talk about new ones.

effulgentsia’s picture

Crell’s picture

Status: Needs work » Fixed

No, we're done here. Plenty of paint has been spilled.

pounard’s picture

Yes, we have a magnificient bikeshed now!

Automatically closed -- issue fixed for 2 weeks with no activity.

Anonymous’s picture

Issue summary: View changes

Revise based on discussion.

davidwbarratt’s picture

I've created #2513388: Create (and maintain) a subtree split of each Drupal Component to create a subtree split of every Drupal Component.