Move all module-provided classes to PHP namespaces (PSR-0 or similar), and autoload them without the registry [policy, no patch]

We should also consider that just because a class lives in a given module on disk does not mean it must be part of that module's namespace. Eg, if module Foo provides a Views style plugin, which of these is the right namespace:

class \Drupal\Views\MyStyle
class \Drupal\MyModule\MyStyle
class \Drupal\MyModule\Views\MyStyle
class \Drupal\Views\MyModule\MyStyle

I'm not entirely sure how the Symfony autoloader (which we're using for core for now) handles multi-root namespaces, and if it does so performantly. Eg, it would be sweet if we could just say that $module_dir/includes/Drupal is a root of a Drupal namespace tree, and everything just logically folds together. That is, $module_dir/includes/Drupal/Cache/CustomCacheImplementation.php just magically works for loading a new caching implementation. Whether or not we can do that in anything resembling a sane performance cost, I do not know.

well the Symfony ClassLoader just cycles through everything you configure. You can configure as many different namespaces for a single dir as you want. In terms of performance the more dirs you configure the longer it will take .. unless you cache .. or better yet use a build script to generate a static map.

At the moment we're not dealing with procedural namespaces, just classes, and at the moment there are no plans to make hooks OOP. Namespaces for hooks is in another thread and stalled, because it's actually quite difficult. We're only concerned with classes for now.

Even if every module includes only one class, a common site these days can easily contain 100 modules. That's 100 possible places to search for classes if we don't have some faster mechanism.

The key question, I think, is how one provides a class to a subsystem not your own. Cache implementations, Views, Blocks (when we get those rewritten as OOP plugins), etc. are all things that modules will be providing all the time to plug into systems not their own. That's a use case we must support and support well.

The key question is whether the subsystem namespace should be used for classification, or if we use namespaces purely for autoloading.

Vis:

1) All cache backend implementations live in the \Drupal\Cache namespace. You can then specify a cache class by just its short name, and the cache system will know to prepend \Drupal\Cache to it.

2) A given module's cache implementation lives in its own namespace, and only coincidentally relates to caching. You must then explicitly reference your class as \Drupal\MyModule\SomeCacheImpl so that the autoloader can find it.

In either approach we're still looking at 100 autoload roots in the degenerate case (sans APC caching or a built map, et al), so we're going to have a performance problem either way.

#6 is currently true, but only for legacy reasons relating to finding procedural code. Moving forward, I'd argue that we should be minimizing module "ownership" as a critical concept.

OK, finally catching up on this, I hope...

So, we need to figure out how we want to organize our namespaces. The key question is performance, but API usage is also a question.

I see two possible approaches:

1) Unified namespace tree with fragments.

In this model, there is a single global "tree" of classes, rooted at \Drupal. Every module contains a /includes subdirectory in which all namespaced classes live, with a full path. Namespaces are based on subsystem, which may or may not have any relation to module names. So for instance, a module providing a new Caching plugin would have:

sites/all/modules/memcache/includes/Drupal/Core/Plugin/Cache/Memcache.php
=> namespace \Drupal\Core\Plugins\Cache

For that to work, we would need to register multiple base paths for the Drupal namespace in the class loader. The Symfony loader can do that, but I'm not sure of the performance implications when we get to 100 modules.

Pros:
- Very predictable namespaces for things. You can tell what something is just by its namespace.
- You can add to a namespace from anywhere. Very simple.
- Some systems become very easily extensible. For instance, if we can move the module_list() function to depend on configuration, not on the database, and the DB layer drops the fancy magic naming of classes in favor of namespaces, we can very easily add database drivers as "real" contrib modules.
- Class names are totally divorced from modules, which makes them trivially portable if we want to refactor.
- We can potentially use namespaces as a registration mechanism. if(class_exists('\\Namespace\\We\\Expect\\Thingie')) then we know we can use that and don't need a definition hook of some sort (hopefully).

Cons:
- I worry about the non-APC-cached performance of the autoloader if we have 100 possible roots for a namespace.
- Possibly annoyingly deep paths to get to a class (as above), especially if a given module only provides one class.
- We would have to decide what constituted a subsystem and therefore a new namespace. Sometimes that may correspond to a module, but not always.

2) Module-as-namespace.

The alternative is to say that every module is its own component namespace. That is:

sites/all/modules/memcache/includes/Memcache.php
=> namespace \Drupal\memcache

In this case, we'd hook up the autoloader with a pattern for each module, in the form:
drupal_get_path('module', $module_name) . '/includes' => '\Drupal\' . $module_name

Core-provided classes would live in either a Core sub-namespace, or use a subsystem namespace as they are doing now for both CMI and WSCCI.

In this case we'd have one autloader wired up with a whole crapload of namespaces, each with a single root path. Again, we're looking at 100 different namespace bases.

Pros:
- Probably a simpler and therefore more performant autoload time. By how much, I do not know. It will likely vary widely depending on the OS and kernel version, as newer kernels have much better disk caching.
- You know exactly what module a class comes from just from its name. (Whether this is a pro or con is a matter of opinion.)

Cons:
- No ability to slip a class into a namespace. All classes get namespaced just to the module and that's it. That means we could not rely on the namespaces for any functionality. (No adding DB drivers from contrib, at least unless we use some alternate method there.)
- No way to tell what a given class is from its name.
- Module names with underscores could get tricky, since that has special meaning in PSR-0 for PEAR-compatibility.
- Namespaces are then variable case. Some are Leadingcaps, like "Drupal" or the core systems like "Context" or "Configuration". Others would have a module name, which is always lowercase. That could be confusing. Doing a ucfirst() on the module name is possible, but could also be confusing, especially for module names with underscores.

Another consideration is that we should support non-module-provided classes, like from 3rd party systems. That is, the libraries API should just automagically support PSR-0-compatible 3rd party libraries thrown into sites/$whatever/libraries (or sites/$whatever/libraries/includes, or whatnot). That would be useful for an increasing number of 3rd party libraries, such as Symfony2 components we don't ship in core, anything from Zend Framework 2, the Buzz HTTP client, etc. That's probably doable by just scanning for top-level directories in sites/$whatever/libraries/includes and treating each of those as a namespace to be registered. That's probably its own patch.

I don't really see a 3rd approach that's viable that isn't just a slight variation on one of those two.

From the list above, it looks to me like approach 1 has more Pros. My preference would be approach 1, certainly. However, the major outstanding question is performance. We therefore need someone to benchmark both approaches.

The main question, I suppose, is which is faster: 100 roots for one namespace, or one root each for 100 namespaces? (I'm using the number 100 as it's a not unreasonable number of modules to have on a site these days, and is a reasonable target for benchmarking.) We need to ask this question both with the Apc-caching class loader and the non-caching one, and probably on a fairly recent Linux kernel to get the benefits of modern disk caching. (I think by the time Drupal 8 is released, with a PHP 5.3 dependency, we can count on such recent kernels being typical.)

Another wrinkle, that we should probably benchmark while we're at it: In practice, I'm assuming that we will have the option to:

1) Compile all or most code into a single file that gets loaded early, skipping autoloading for all of those classes. (This is popular with some frameworks, and is of course APC-only.)
2) Compile all or most code into a phar archive that gets loaded early, skipping autoloading for all of those classes. (I don't know if this would be APC-only or not.)
3) Ship either 1 or 2 as part of core by default, for core's classes at least.

(In all of those cases we'd need a kill switch for debugging purposes.)

So we probably want to also compare loading 100 classes with the class loader vs having them all in one file vs. having them all in one phar archive, so that we have a baseline to compare to.

OK, that's a lot of benchmarking. :-) But I think these questions are rather important, and will impact how we organize code throughout all of Drupal.

Zend has ZendX for "framework extensions" and it seems that in those posts, nobody really tell there is the possibility of doing something like:

• DrupalX\MyModule\TheRestDoesNtMatter.php
• DrupalContrib\MyModule\The\Rest\DeosNt\Matter.php

This has almost only pros:

• It's so much clearer of what's core and what's not
• With this standard, autoloading is fast because of "subprefix include path" feature
• People using an IDE needs to see the module name in the namespace
• Moreover, if they type random stuff and they don't know all the modul names, the X-ternal namespace tells them it's not core but a contrib
• Leverage core quality insurance, at least visibility: Drupal is stable while DrupalX may not be
• What's inside contrib doesn't matter for core, and what's inside the module namespace I don't care if it is DrupalX\ViewsCsv\Views\Plugin\Display\Csv.php or DrupalX\ViewsCsv\ViewsCsvDisplayPlugin.php both are equally valid and they will probably go deeper on hierarchy by themselves only if the module contains thousands of class
• Contrib stays encapsulated in their own namespace

I don't see cons, really.

For the rest, I don't like magic, and I definitely do not want that my modules having a forced namespace. For what it worth, I'm totally ok with a "namespace = [DrupalX\]MyModule" inside my info file.

Really, about the
[Whatever\]MyModule\Core\Cache\MyPlugin
Or
[Whatever\]MyModule\MyPluginCache

This issue doesn't matter and doesn't belong to this thread. For some modules, I would naturally unfold the various core namespace parts, depending on my module complexity, but for some others I would not because they only carry one class or are really simple. This doesn't really matter as soon as the word "Cache" appears somewhere in the class name or namespace name.

Module names in Drupal are lowercase with underscore - that's a historical design choice.
Making subsystem and "Drupal" namespaces ucfirst is a recent design choice, trying to be aesthetically compatible with other frameworks.
Giving up either of these has a price.
Mixing both has an aesthetical price, but one we might choose to pay in the end.

I tend to say that modules should register explicitely their namespaces. Give up magic is definitely a good thing, explicit registration gives us predictability and easier debugging. If we follow my reasonning here, having ViewsAttach namespace part for views_attach module is no longer a problem.

@donquixote
It seems you are all about registration mecanism: for pure PSR-0 class namespace this is off-topic. Module just needs to tell core they have a namespace for autoloading, end of story, this goes way out of scope of coding standard/convention itself.

PS: I'm still pro DrupalX (where X can stand for anything else) root namespace for modules. Core is core, and must not be merged with modules. Modules can use the core API the way they want it's not our own problem.

To go further about a bit off-topic, anyway, plugins should be at some point explicitely registered, it doesn't matter how right now, and plugin API will gives us the way to do that, but namespacing and component registration must not be coupled. Coupling those two in order to provide magic is definitely a refactor/flexibility/debugging blocker.

Now we have "Give up magic" vs the good-old "convention over configuration".

Giving up magic doesn't imply that we should not have a convention. This "versus" is a nonsense IMHO.

We need a convention, but we must not dictate this convention because potential magic we would want to do, it's non efficient and will end-up with something confusing for people that don't understand/know the magic.

All the pros I see are like "performance" and "plugins" and stuff, but that's not the question. The real question is having a "readable" convention for people to code efficiently. All the rest doesn't matter. Right since the beginning we all agreed to do PSR-0 code so far, and for what it worth, autoloader performance is no longer our problem, Symfony autoloader itself has various optimized versions of PSR-0 autoloaders.

Plugins and stuff are pure business stuff that doesn't matter about this namespacing standard either.

The real concern is encapsulation and code readability.

don and pounard: Uh, I think you guys missed the point. Nothing in this thread matters in the slightest until we get some benchmark data for #9. Until then we're just bikeshedding needlessly. Performance is absolutely still our problem, until benchmarks say otherwise.

None of what anyone has suggested above is not "just a slight variation on one of those two." The key question is unified namespace or namespace-per-module, because that's what affects the way the class loader is configured and how it will perform.

Nothing precludes us from having a portion of the namespace tree in option 1 that is used for module-specific stuff, or having a \Module\ part of the namespace if it makes sense, etc. There are valid arguments for that, but it does not relate to performance.

Also, I don't see us using PEAR classes much but the Symfony class loader supports _ => directory mapping, because that's what the PSR-0 standard says to do. That means we DO have to deal with it.

Just to clarify the special underscore handling. Copying the standard:
"Underscores in namespaces and class names:

\namespace\package\Class_Name => /path/to/project/lib/vendor/namespace/package/Class/Name.php
\namespace\package_name\Class_Name => /path/to/project/lib/vendor/namespace/package_name/Class/Name.php"

https://github.com/php-fig/fig-standards/blob/master/accepted/PSR-0.md

So the special handling is only relevant for class names but not namespace names. So I think there is nothing to worry about in this regard for the topic at hand.

I'm definitely for the "Module-as-a-namespace" approach.

Using namespaces as "a way to classify" classes is the wrong approach. Having a foreign Views handler in Drupal\Views\Handlers is gross and error-prone. You would have to namespace those again manually, losing the benefit that namespaces where giving you in the first place.

How to automatically detect implementations of a given interface remains in question, but it is a question independent of the one of the autoloader. Using namespaces as a way to classify classes doesn't help you here: there is nothing in PHP that allows you to get "all the classes of a given namespace (even those that are not loaded yet)".

@donquixote: it's sadly absolutely not for you to chose. PSR-0 is a strict, one-class-per-file autoloader.

Deviating from PSR-0 is not on the table for consideration. The whole point of using PSR-0 is interoperability. If we break PSR-0, then we break interoperability.

Both PSR-0 and the Symfony2 class loader in particular allow for a namespace to be "rooted" in multiple places, which takes care of the "modules can live in 4 places" problem.

With most of the namespace being resolved to a module dir, I don't see much left of the original idea of PSR-0

That is not guaranteed, as we are still waiting on benchmarks.

Rather than bikeshedding further, can someone put together a test script or zip file that can be tested to answer that question? msonnabaum has said he can run the tests if someone puts together the scripts TO test.

I'm still convinced than DX is more important than performance when it comes to autoloading. Whatever you are doing, if autoloading itself become a performance problem you have to solve, it's because your framework is not well designed. I'd say that this is micro-optimization, when you are too slow, it's the overall software design you have to re-think, not the autoloading part.

Damien Tournoud +1 about the Module as namespace note.

A nice person on Twitter pointed me toward this set of benchmarks, which show Phar being really good for performance:

http://cweiske.de/tagebuch/php-phar-files.htm#benchmark

@donquixote
Following an half PSR-0 would just be an awkward yet another Drupal WTF. It would probably force us to rewrite our own autoloaders instead of being able to use already existing one.

Standard is not only important for code re-usability, but also for code comprehension; 4 years ago Zend code was already PSR-0 compatible (yet I don't know if PSR-0 acronym was invented, but the code was); Now since a lot of framework are switching. It's not only for performances and code readability, but it also make people that are used to code with this kind of naming conventions to be able to understand, broswe into, find and write the code in an efficient way. Drupal has developers from everywhere, including those that are used to Zend of Symfony or both.

I started with Zend before Drupal, more than 4 years ago when it was 1.0.0 then, I am one of those people. Please think about developers. Just another half reinvented wheel in Drupal would definitely kill kitten.

Agreed. We are not implementing a custom one-off pseudo PSR-0 implementation. The entire point of PSR-0 is to do away with Drupalisms and play nicer with the wider PHP world: http://www.garfieldtech.com/drupal-symfony2.

Let's focus on getting those benchmarks, please. Crell or Mark, would you be able to stub out some pseudocode of what you're looking for?

Fixing title.

Removed old and incorrect descriptions of options that were never under consideration. Replace with the descriptions from in the thread.

I actually agree with having a pseudo-PSR-0 for modules.

Related: #1299424: Allow one module per directory and move system tests to core/modules/system

@damz Well is some respect we have to psuedo-PSR-0 at some level because PSR-0 does not not allow for our use case. Its mostly designed for vendor namespaces in a single directory. That said, we can have multiple PSR-0 roots and implement PSR-0 within the modules to keep things as consistent as possible.

As I see it, allowing "_" is out of the question though because it breaks in the PSR-0 we are using for core, its not compatible with PEAR and frankly as it stands is not compatible with out class naming standards. Its just something we have to live with.

--

I think the real question of this issue boils down to how we handle default supported function our new Drupal namespace wrt modules.
1) We allow modules to interact with the root PSR-0 namespace.
2) We don't allow modules to interact with the root PSR-0 namespace.

To that end, we should setup some framework for testing this. Maybe a sandbox with a bunch of mock modules roughly falling into the 2 setups that we can then build some autoload strategies for? We can then attack the benchmarks and possibly expose weakness or strengths that can then give context to discussing the variations everyone has gotten so caught up in.

edit: fix typo

@#36 neclimdul+++ (one + for each statement).

I'm still convinced than debate around magic module stuff autoloading should be shut off and we should consider that module should register their namespaces.

If one module embed a GPL compatible PHP PSR-0 library it needs to be able to arbitrary register it. My guess is that module namespaces should not be linked at all from the real module name.

PSR-0 does not assume that you have a single root tree. As was explained to me when I raised that exact issue on the FIG list a few years ago, the ability to register multiple class loaders at different roots is a deliberate design feature, so having a separate root in each module does not break PSR-0. Not having the /Drupal/whatever part of the namespace represented in the path might; I'm not 100% sure there.

neclimdul: It's more whether our namespace hierarchy is subsystem-based or package-based. (Module usually means package, but sometimes means subsystem.) A module could always inject a class somewhere it didn't mean to, and I don't think making that impossible is wise.

We don't need converted Drupal for benchmarking. We just need a simplified test case that can be run in a vacuum.

Core modules can be in core namespace, they are core modules. I don't see any problem in that (they are core components). Where modules should not be namespaced upon the system is as soon as they are not part of core distribution: they can provide any component pretty much like core does, but in their own Vendor namespace (even if the namespace root is something such as DrupalContrib\MyModuleVendor).

pointless

Nothing tells that nobody will never want to write generic code in there. I don't see the point of forcing the Vendor name of contrib modules.

Why not follow Symfony to the fullest:

Namespace of any project hosted on d.o. should start with "Drupal" (being the vendor). Drupal modules, install profiles, etc, hosted somewhere else may use their own vendor namespace.

[drupal]

core/
modules/
themes/

[drupal]/core

Drupal/
Drupal/Component/
Drupal/Component/Module/
Drupal/Component/Module/Module.php
Drupal/Component/Form/
Drupal/Component/Form/Form.php
Drupal/Module/
Drupal/Module/Node/
Drupal/Module/Node/NodeModule.php
Drupal/Module/Node/Form/NodeEditForm.php
Drupal/Theme/

[drupal]/modules/Drupal/Module/Views

ViewsModule.php
Views/ (contains code regarding the views API)
Views/Plugin/
Views/Plugin/Plugin.php
Views/Plugin/DisplayPlugin.php
Views/Plugin/PageDisplayPlugin.php
Views/Handler/
Page/
Page/ViewsPage.php

[drupal]/modules/Drupal/Module/ViewsBulkOperations

ViewsBulkOperationsModule.php
Views/
Views/Handler/OperationsFieldHandler.php

If we don't rely on magic names (but instead use registers, like is being done more and more), tiny modules don't have to add several nested folders to implement a certain api.
[drupal]/modules/Drupal/Module/TinyViewsThingy

TinyViewsThingyModule.php
Views/ThingyFieldHandler.php
or
ThingyViewsFieldHandler.php

[site]

[drupal/]    (could live somewhere else)
sites/
index.php

goal title is now "namespaces and autoloading"

After watching several times this presentation about using several components from different PHP frameworks in a single project (thanks Crell for the link !), the procedure for adding code in a project is pretty clear and composed of 3 steps :

1. Download library code in the proper directory : in our case a contrib module, or a custom written one that would go in sites/[all|$site_name]/modules
2. Manually register its namespace with the autoloader : only the top level namespace part matters
3. Profit !

So basically I agree completely with pounard. No need to enforce anything. Just let modules or external libraries register themselves.

Waiting for the benchmarks.

@#46

Drupal\m\[module name]\[class name (*)]. The latter is easier to type, gives us shorter window captions in gedit, etc.

I do not agree with editor captions being smaller, this is not IMHO a good point: having the word Module explicitely fully written is much more explicit.

Deep vs flat

This is a valid point, but I'd go for deep, because the modules could embed third party libraries, so if we need a standard deep would allow them to put those libraries in the lib/ folder as well, e.g.:

mymodule/lib/SomeThirdParty/
mymodule/lib/Drupal/Modules/MyModule/

Subfolder name for class files. [...] We should keep in mind that we might want to keep the other names for other things. For instance, we might still want places to put include files with functions, or classes that don't follow module-per-namespace

I definitely agree about this, and I'm sad that core chose to put all inside includes.

If we go through the declarative path (i.e. modules must register their namespaces) having a camel cased module name in namespaces doesn't cause any problem. We can transform og_access in OgAccess too quite easily too, which allows us to skip the declaration per default.

I tend to say that code must be pretty, that's really personal and subjective of course, but Drupal\m\og_access\SomeClass isn't pretty, while Drupal\Module\OgAccess\SomeClass surely is too my eyes.

EDIT: Code must like a melody, you must have no wrong notes in it, and mixing naming convention in what is supposed to be a coding convention seems like weird to me.

I believe info files were mentioned in the Context API issue. catch was reluctant because those currently hold a lot of crap that becomes a memory issue. Personally I'm fine with it, provided we denormalize that information somehow so that we can load it up pre-bootstrap_full without blowing our memory. It does seem like the only viable out-of-default-namespace approach (whatever the default namespace structure ends up being).

Yes the hardest is to provide an efficient denormalization mecanism. Because module list still rely on database, we can safely put this denormalization in the database as well, maybe denormalizing it as a field in the system table directly would be quite efficient (loading it at the same time as the system_list() cache).

PSR-0 was never designed for extensions/components/plugins, trying to use it for modules is just wrong.

I think PHP needs a PSR-1, that would focus on extensions and include a proper mechanism for metadata (ala Ruby's gemspec, CommonJS' manifest.json, Drupal's .info, etc.). This metadata could include the registration of namespaces that the extension provides.

Mapped to Drupal, we could have a default of:

namespaces[Drupal\Modules\<modulename>] = /

Which means "the files in the root of the extension belong to the Drupal\Modules\<modulename> namespace".

If the extension ships other namespaces, it can register it manually:

namespaces[Symfony\Component\HttpFoundation] = /lib/Symfony/Component/HttpFoundation

PSR-0 was never designed for extensions/components/plugins, trying to use it for modules is just wrong.

Still, this is the standard we are using here; And there is no other for us right now. Even Symfony uses it for its Bundles (which are more or less as our modules): I don't see anything wrong with it.

What you are refering as PSR-1 is one of the solution proposed upper: most of the frameworks' autoloaders (Symfony's one is our concern) allow us to set specific pathes for sub-namespaces (e.g. /some/arbitrary/Path => Vendor\System\SubNamespace), we can use this as well, this has not been declined in this thread: it's in fact being discussed a few comments upper and the question is still opened.

, maybe denormalizing it as a field in the system table directly would be quite efficient (loading it at the same time as the system_list() cache).

That's what we currently do with every single thing specified in .info files and it leads to a massive cache entry being loaded every request once you have some contrib modules installed, because all kinds of crap is put into .info files that's never, ever going to be needed for the vast majority of requests. This happened with the scripts and styles addition in Drupal 7 (where we went from not loading anything from .info to loading the whole lot), there's a patch at #1061924: system_list() memory usage but it's only a band-aid on this.

In this case, we'd be exchanging this for the files[] declaration for the registry, so at least it's likely to mean less in .info overall.

@catch I was saying denormalize this unique information, and do not load the info field content: the impact on memory would be reduced because we would be loading only the this field content more.

Right, but at the moment we're loading everything anyway, so doing it like that now would mean loading it twice (in fact we don't only load the .info file but we already load the entire contents of each row of the system table right now).

I like the syntax proposed by Damien #54, it matches exactly the info we need : namespace and location, so that we could hand them to the autoloader and we're done. I agree with pounard's idea #56, loading only this information. Of course only for the activated modules.

Further thinking, a bit OT : if all modules become classes, all we need is the namespace info and the status (activated or not), right ?

catch: Yes, the system table right now is a performance suck. But we should define the developer API for how they specify what non-standard namespaces they provide on its own; if the result of that decision is we have to restructure how we store data in the system table or replace the system table with something from CMI, well, that's on the table. Let's not dismiss a DX proposal on "our current implementation already sucks".

I was under the impression that once the details of the namespaces were sorted out, there'd be a patch here to support it, so I've been replying on that basis.

If this is only a policy issue, let's re-title it then.

And yes in principle I'm not opposed to this being a .info key (especially since files[] is already and we can ditch that in the same process), but we really need to not tack it onto an already broken implementation if we go with that, and fixing the whole mess that was introduced in D7 isn't a job for this issue (although I wouldn't want to make it depend on CMI either).

No, I'm assuming that the result here will be a patch to auto-register namespaces for classes provided by modules, whatever convention that ends up being. My point is that:

1) The way we handle info file data now sucks.
2) We already know we need to overhaul that to make it not suck.
3) Info files seem like the best DX for out-of-convention namespace registration.
4) Given 2, there's no reason that 1 should block us from doing 3.

I think we all agree about using the .info file, we also all agree that its managing sucks but can be changed later, ever more once CMI would have came out with something über cool for configuration managment. Let's continue the thread about naming/namespacing?

OK fine, let's add the .info property here if we have to, can't really object since the same patch will be removing the requirement for files[], no matter how much I dislike .info file abuse this isn't making it any worse. Since the current proposal allows a default location without puting crap in .info then in practice it may not get used that much anyway.

So, ignoring everything in this thread past #9…I don't think we really need to benchmark these approaches. The behavior of each should be pretty easy to predict.

First though, I think we need to adjust how we register our namespaces. The attached patch moves the initial autoloader include into the drupal_get_autoloader() singleton. This holds one copy of the autoloader and registers it immediately. Both core and contrib can then add whatever namespaces they need during the request.

I've also moved the main Drupal namespace to a fallback namespace. The way it is now, Drupal => core/includes/ is registered first so that path has precedence over any other registered namespaces that come after. This means that if we do this in views.module:

$loader = drupal_get_autoloader();
$loader->registerNamespaces(array(
  'Drupal\Views' => __DIR__ . '/includes',
));


$view = new Drupal\Views\View();

The autoloader will first try to load core/includes/Drupal/Views/View.php before it tries the correct location, which would be something like sites/all/modules/views/includes/Drupal/Views/View.php.

When we're talking about performance here, we're talking about reducing the number of failed file_exist calls. If we register the main Drupal namespace as a fallback namespace, this won't ever happen for contribs. It will cause more namespaces to be iterated through for core classes, but they will fail at strpos and never get to a file_exists.

This also has the added side effect of allowing a contrib module to override any core class by registering Drupal\Core.

If we can make this change, I think it makes sense to just do option 2. We can predict that it will perform decently and it makes the most sense from a DX perspective as well.

+++ b/core/includes/bootstrap.inc
@@ -2288,39 +2319,16 @@ function _drupal_bootstrap_configuration() {
   // Register classes with namespaces.
   $loader->registerNamespaces(array(
     // All Symfony-borrowed code lives in /core/includes/Symfony.
     'Symfony' => DRUPAL_ROOT . '/core/includes',
-    // All Drupal-namespaced code in core lives in /core/includes/Drupal.
+  ));
+  $loader->registerNamespaceFallbacks(array(
+     // All Drupal-namespaced code in core lives in /core/includes/Drupal.
     'Drupal' => DRUPAL_ROOT . '/core/includes',

Is there a reason to not put Symfony in a fallback, too?

There are two issues I have with #67:

1) It's going to conflict with what I had to do with the autoload setup in #1321540: Convert DBTNG to namespaces; separate Drupal bits to leverage the autoloader in the installer, too. Mark, can we cut to the chase and bring in that part of that patch so that the installer will work? That at least reduces further conflict between those patches.

2) The word "singleton". Eek. :-) That said, we don't have a better mechanism than that in core right now, so we can convert to something better than a global function once we have a something better.

Also, should we put the baseline registrations into the factory routine in drupal_get_autoloader()? I'm not sure why those need to be separate since they're so basic and universal.

I've been warming to approach 2 from #9, based on the WSCCI experimentation I've been doing. If it would actually be more performant, that would help that case.

Bah, Dreditor.

About #68, issue 2). Singleton. Does it make sense to register the autoloader in Pimple ? My understand is that Pimple is a service container, and the autoloader can be considered a service. It's just a suggestion out of my mind, not even sure it's possible.

DjebbZ: When/if we have Pimple in core, yes. That's the "something better later" I was thinking of (and mentioned to msonnabaum in IRC when we were discussing this).

Is there a reason to not put Symfony in a fallback, too?

Yes, none of our namespaces will start with Symfony, so it'll never get past strpos. Vendor libs should be registered normally.

1) It's going to conflict with what I had to do with the autoload setup in #1321540: Convert DBTNG to namespaces; separate Drupal bits to leverage the autoloader in the installer, too. Mark, can we cut to the chase and bring in that part of that patch so that the installer will work? That at least reduces further conflict between those patches.

New patch attached incorporates some of what you did there, but I'm not seeing what I can do with the installer bits. If we can get this in soon though, the changes to the DBTNG patch will be trivial.

2) The word "singleton". Eek. :-) That said, we don't have a better mechanism than that in core right now, so we can convert to something better than a global function once we have a something better.

Indeed. It's fine for now. Gotta move forward.

Also, should we put the baseline registrations into the factory routine in drupal_get_autoloader()? I'm not sure why those need to be separate since they're so basic and universal.

I don't think we should. There'd be no functional difference, but I like that it serves as an example of how it would be used in contrib the way it is.

I agree with msonnabaum, we should go ahead and implement this since we know more or less what it'll look like in terms of the performance characteristics of the options, and benchmarking it is about as much work as implementing it. The main concern here should be avoiding file i/o and this appears to be fine for that - we should never be looking for a file anywhere other than where we very much expect it to be located.

Once we have actual code using it, we can then profile that to confirm what it looks like and see more what the general performance of the Symfony autoloader starts to look like when it's in use. As long as we do that soon-ish, it gives us time to deal with it if there turns out to be a problem.

Various code clean-ups. Looks good to go for me.

Why do you use registerNamespaceFallbacks for core? Even the method names yells it's wrong, because core is trying to slowly switch every bit of its code to PSR-0 code in /includes: it's not a fallback. It sounds semantically incorrect, even if it technically is correct. So unless there is a real technical concern about this, in which case it needs to be documented because it sounds semantically incorrect, it should be reverted to the original statement with both namespaces registered at once.

@pounard: That was explained in detail in #67 already.

That said, it should also be explained in code. Adjusted the patch.

Oh right I missed that point upper, thanks. Patch looks good.

I like the patch too. Thanks sun for the clarification in comments.

+++ b/core/includes/bootstrap.inc
@@ -3025,6 +3013,45 @@ function drupal_get_complete_schema($rebuild = FALSE) {
+ * Initializes and returns the class loader.
+ *
+ * The class loader is responsible for lazy-loading all PSR-0 compatible
+ * classes, interfaces, and traits (PHP 5.4 and later). Its only dependencies
+ * are DRUPAL_ROOT and variable_get(). Otherwise it may be called as early as
+ * possible.
+ */

What about adding a @return line with an explicit type hint? Eclipse uses PHPdocs as helpers when providing code completion, other IDEs might do it too.

If I understand the patch correctly we are implying that the returned value is an instance of UniversalClassLoader. Unless we want to support any class implementing the registerNamespaces and registerNamespaceFallbacks methods, in which case we should have an interface in place.

-9 days to next Drupal core point release.

yeah @return is needed anyway

Apparently, and this is very odd (for Symfony), there is no ClassLoaderInterface.

This should be fixed upstream, and when that is done, we can add a @return ClassLoaderInterface. Without an interface, we can't really specify anything aside from a description. So until that happens, I'd rather leave this alone. The phpDoc summary already states that it returns a class loader.

What's wrong with the UniversalClassLoader type hint? If Symonfy wontfix this for any reason we are stuck with an unspecfied return value for a factory, which is kinda unusual (to use a polite word).

Agree too, let's type hint the most we can, even in PHP doc: it makes IDEs and self-documentation way more efficient and precise; it's also a good helper for later documentation generation.

Added @return to the phpDoc.

Ok looks good for everyone now

Yes nice, thanks.

Thanks :)

There's no code here to register the module namespaces, which makes this just clean-up of the existing code at the moment.

I'm fine committing that, but then we need a follow-up to actually register the default namespaces for modules (and possibly support the .info key that was proposed above), if we're not doing that there.

#1399718: Register the deault namespace for modules

@catch:

I don't think it's sane to move over there since all the relevent discussion is here. Can't we have a commit and go on as nothing happened? ;)

We should commit #85 and keep talking. It would hardly be the first time we committed a patch without ending an issue... The original namespace patch was the same way.

Well we should keep talking anyway if we're doing everything in this issue, since I leave all patches at RTBC for 2-3 days before committing anyway, fine with doing it that way though.

Related: #1400748: Proposal for unified namespace organization

Oops.

I like #85 a lot. Should make registering module namespaces a lot easier when we start getting into implementing whatever we decide the policy is.

I don't like having the "use" statements 3k lines above the only time we'll ever use the class but this is the standard we've chosen. This is sort of a poster child for why I was uncomfortable with how strict we made it the guidelines for procedural code but now that I've said my piece about something mostly unrelated to the actual patch, it looks great.

Committed/pushed #85 to 8.x, back to active for the discussion.

I am hoping this isn't off topic for this discussion, but I was directed here from git blame and my pondering over at http://drupal.org/node/1417320#comment-5613882.

This isn't a question about namespace exactly, but rather the committed patch and why drupal_classloader is called so 'late' in the very early bootstrap? It seems it should happen at the very beginning of _drupal_bootstrap_configuration() (if not earlier). There is a lot of code that could use autoloading before that e.g. the request object.

I don't recall exactly what broke, but when we moved the class loader before the config step something else exploded. Hopefully we can move it earlier once we've cleaned things up further. Or maybe we could push it earlier now that it allows for multi-step setup. Not sure.

In that case.... Here's a patch with autoloading available sooner. Lets see if testbot spots a problem.

@donxiquote Using APC cache based autoloader is the same as using a map based autoloader: the resolution would happen only when attempting to fetch a non already fetched class name. What's disturbing with the Symfony APC implementation is that they don't store a class map, but one user cache entry per class name, but this is another matter, the fact is that it's an incremental cache that will actually lookup only on the few first hits. On a stable production system, once this incremental cache is built (i.e. probably really quickly) it is at its maximum efficienty because the code is stable, so theorically, the pragmatic lookup would never happen except when trying to request a non existing class, case in which a WSOD will happen: in that particular case, you have bigger flies to worry about because your site is broken, not the loader. So, stop worrying about the loader internals, because we can, later, if we want to, implement our own front loader proxy instance to cache a map if we want to, that's not the concern of this issue and this is radically off-topic right now EDIT: OK it might be some of the concerns of this issue.

EDIT: PSR-0 implies the file_exists() calls, whatever happen. The best we can do to minimize them is ordering namespaces puting the most used ones first, once that done, I don't see any other good way to proceed except by getting out module namespaces outside of root Drupal namespace, but once again it doesn't sound loader related to me, but more an organizational problem.

indeed .. for those willing to "pre-generate" the class map, then for 2.1 we have the necessary tools to generate the class map as well. modules could also ship with class maps. so i agree .. its a non issue. there are various ways to address performance issues.

donquixote: Enough. This thread is about how we organize code for modules, NOT for micro-optimizing the loop count within the class loader. You are, once again, grossly off topic.

foreach() and strpos() are pretty damned fast. You're micro-optimizing and wasting everyone's time.

No, do not open a new issue to discuss it. There is already an issue to discuss it: #1241190: Possible approaches to bootstrap/front loading of classes

And no, please do not go discuss it there either. Squeezing every ounce of speed out of the class loader is a complete and utter waste of time, because the IO cost of reading the file off disk will be bigger than any foreach() loop you can possibly write. Map-based loaders, precompiled files, phar, and other useful options are already being discussed, but really, it's not worth discussing until after December 1st. We have 2 months of "feature but not code freeze" that are ideal for sorting out this sort of optimization. That's most of a year away.

Let. It. Go.

Following up on #100
I initially moved the call to drupal_classloader earlier in the bootstrap_configuration - but it turns out even that is a tad bit late in installer because of its logic.
So I've created a new bootstrap phase for bootstrapping the autoloader. This will certainly ensure that having the classloader available is one of the very first things we do.

If this should be spun into its own issue I can do that. But since this classloader and namespacing issue isn't yet closed, I thought I would continue with patches here.

Lets try this again.

Other than a few minor gripes, it looks good to me! Having the autoloader initialize earlier on in the bootstrap means that we could move some of the bootstrap to Drupal\Core.

+++ b/core/includes/bootstrap.incundefined
@@ -105,44 +105,49 @@ const WATCHDOG_DEBUG = 7;
- * First bootstrap phase: initialize configuration.
+ * First bootstrap phase: initialize auotloader.

Minor spelling typo.

+++ b/core/includes/bootstrap.incundefined
@@ -2181,6 +2187,10 @@ function drupal_bootstrap($phase = NULL, $new_phase = TRUE) {
+          break;
+        ¶
         case DRUPAL_BOOTSTRAP_CONFIGURATION:

Have some trailing empty whitespace.

+++ b/core/includes/bootstrap.incundefined
@@ -2318,6 +2318,22 @@ function _drupal_bootstrap_configuration() {
+  drupal_settings_initialize();
+
 }

Probably don't need that extra line-break at the end of the function there.

7 days to next Drupal core point release.

[EDIT]....... Think we should open up a new issue for this rather than having it on this seemingly meta one?

Thanks for the review Rob - I created a new issue specifically for the new bootstrap phase as suggested

http://drupal.org/node/1451524 along with the latest patch fixing the minor items from the review.

I just read through this full issue, and from what I can tell, we still need to decide on a default folder layout for modules. @Damien proposes a non-PSR-0-compliant option in #54. Here's my counter suggestion to that, based on #1400748-167: Proposal for unified namespace organization:

Without a module needing to add anything to its .info file, early in bootstrap (TBD how early and how to deal with a non-db-dependent system_list()), on behalf of all enabled modules, we call

$loader->registerNamespace('Drupal\\' . $modulename, $moduledir);

Which means, if I want to add a Drupal\user\LoginBlock class to user.module, it goes into core/modules/user/Drupal/user/LoginBlock.php. If this were a contrib module, replace core with sites/all or similar.

Additionally, a module can add stuff to its .info file like:

namespaces[Drupal\user] = /lib
prefixes[SOME_PEAR_PREFIX] = /lib
namespaces[Symfony\Component\HttpFoundation] = /vendor

So, namespaced, or non-namespaced classes can go into any module-determined location within itself, but within that location, it needs to follow PSR-0 structure. "lib" and "vendor" are recommended guidelines, with #1400748: Proposal for unified namespace organization discussing when something is a full-fledged vendor vs. merely a lib. As shown in above example, you can even add the module's primary namespace to "lib" (or wherever you want, but our guideline is "lib"), which registers this *in addition* to the module root folder as per 1st code snippet. This allows (but does not require) a module author to separate their application-level classes (e.g., the LoginBlock) from library-level classes (e.g., a cache plugin or password hashing plugin or whatever).

In response to #54, while the default I'm suggesting in the first snippet still forces PSR-0, and therefore, adds more folder depth, I suggest it's worth it, because:

1. It's good practice to put classes in a separate folder from the .info and .module file anyway, and I don't think "Drupal" is any worse a name for said folder than any other option discussed in #1400748: Proposal for unified namespace organization.
2. Given the previous point, we're only adding 1 superfluous folder: i.e., we have $moduledir/Drupal/$modulename/$classname.php instead of $moduledir/SOMETHINGELSE/$classname.php. While I share a slight frustration about going 1 level deeper than necessary, I think it's an acceptable trade-off for complying with the only autoloader standard that is currently accepted by the PHP community.

Any feedback?

#112 sounds good, with one exception: Using $moduledir as the namespace root doesn't address the various problems pointed out in #1400748-160: Proposal for unified namespace organization.

To wit: It results in namespace roots (tests, vendor-based things, etc.) being within another namespace root. I do not know if that would cause things to asplode, but it strikes me as just asking for trouble, trouble that would be easy to avoid by just having another lib or src directory inside $moduledir that eliminates that potential entirely.

What's an example of something that would cause an explosion? Do you mean something like if I have an non-namespaced class like "PasswordHash" that I found on some random website and decided to use in my module as-is, and in my .info file have:

prefixes[PasswordHash] = /lib

Then if I had some code that called $obj = new Drupal\user\lib\PasswordHash, I would get an error due to the autoloader finding the lib/PasswordHash.php file (containing the non-namespaced class) (see Edit, even this wouldn't happen), but after loading it, still not having access to the namespaced class I asked for? Similarly, if in the #112 example, if I asked for $obj = new Drupal\user\vendor\Symfony\Component\HttpFoundation\Request.

In #1400748-160: Proposal for unified namespace organization, you say:

The odds of that blowing up in our faces in unpredictable ways is high.

But I'm not seeing the above examples as being particularly likely. Do you have something in mind that's more likely?

[EDIT: actually both of those examples wouldn't cause errors (beyond the normal error of asking for a class that doesn't exist at all). So now I'm really stumped on coming up with an example that would.]

I don't have a specific example at the moment, unfortunately. It's just something I look at and go "I can so totally see that coming back to bite us." I really don't want to find out in Drupal 8.1 "oh crap, yeah, THAT use case is now going to suck" when we had an alternative that we know cannot break.

Re: #116: We are over a year from release. I think we need not introduce complexity because we are worried about something coming up later. If something comes up, we take the complexity hit then.

@Moshe: I think the key question there is that I do not see a "lib" directory, which parallels core, as "complexity".

I can live with this approach, I just fear it is going to cause problems later and I hate the feeling in my gut that I am going to have to bite my tongue in 9 months and not say "I told you so". :-)

I dont think lib is a risk nor does it add complexity. It's a common practice to use "lib" as a top level directory to autoload from. I've seen this in ruby, perl, nodejs, java, etc. Just because drupal developers aren't familiar with this doesn't mean it's complex.

We don't need to take the name of the directory so literally, it can mean slightly different things depending on how it's used. Especially if we have a vendor directory next to it. The answer to this post explains it pretty well:

http://programmers.stackexchange.com/questions/123305/what-is-the-differ...

Also, I think it's important to have a vendor directory for modules (think a module with a composer.json), so it would be very awkward to have autoload points at different places in the module's directory hierarchy.

msonnabaum++

Like pounard, msonnabaum++. A lib directory is very common practice "out there".

msonnabaum++

Let us please put this issue to rest.

Trying this one more time...

I'd still like to comment on this before we consider it RTBC. Will do so later today. Also, for now, this is about deciding on a convention, there's no patch to review here at this time. #109 was moved to a separate issue as per #111.

Ok, based on the comments since #112, and a conversation I just had with @msonnabaum, here's my updated proposal (the numbered bullet points in each section correspond with each other).

The questions this issue needs to settle:

1. Which (if any) directories should we register with the autoloader in the absence of anything specified in the module's .info file?
2. What should we allow the module to customize via its .info file?
3. Which director(y|ies) should we use for classes within core modules?
4. What do we expect from contrib?

My current proposal (cocreated with msonnabaum):

1. For each enabled module, core's loader initializer autoregisters the module's namespace to the module's lib folder as follows:
   

   $loader->registerNamespaceFallbacks('Drupal\\' . $modulename, array($moduledir . '/lib'));
   

2. Allow the .info file to contain stuff like (example is for user.info):
   

   namespaces[Drupal\user] = ""                      ; or "/src" or "/app", etc.
   namespaces[Drupal\user\SomeSubNamespace] = /app   ; or whatever
   prefixes[SomePrefix] = /lib                       ; or whatever
   namespaces[Symfony\Component\HttpFoundation] = /vendor
   

   Basically, for each module, core's loader initializer does:

   foreach ($moduleinfo['namespaces'] as $namespace => $dir) {
     $loader->registerNamespace($namespace, $moduledir . $dir);
   }
   foreach ($moduleinfo['prefixes'] as $prefix => $dir) {
     $loader->registerPrefix($prefix, $moduledir . $dir);
   }
   

3. For core modules, use lib for library-level classes (e.g., cache plugins) and app for application-level classes (e.g., block plugins). To reduce file_exists() misses, when adding a .info entry for the app directory, do it for a subnamespace rather than the module's root namespace when possible (i.e., when there are no application-level classes at the root namespace) and not too cumbersome (i.e., when there aren't a bazillion subnamespaces). For example, if the only application-level classes in the User module end up being Drupal\user\Block\Login, Drupal\user\Block\New, and Drupal\user\Block\Online, then a .info entry of namespaces[Drupal\user\Block] = /app is preferred to namespaces[Drupal\user] = /app.
4. Contrib might be all over the place. Some contrib maintainers might put all their classes in /lib. Some might want to use "app" for some classes. Some might want to use the module's root directory. Some might come up with other ideas.

Reasons for above proposal:

1. Autoregistering lib is consistent with what we do for the non-module part of core. This seems useful as many modules will have library-level classes, and /lib appears to be a well-established convention for placing library-level code.
2. Allowing the .info file to specify additional registrations is consistent with prior discussions about this topic. Note that allowing empty string as a value (resulting in the module root directory being used) is frowned upon by @Crell and some others, but I have yet to hear a convincing argument for disallowing it (#115 is a good explanation of why I'm not convinced by #116).
3. While as per above bullet point, I think we should allow contrib to use the module root (if the contrib author feels strongly about that being desirable), I don't think we should do so in core modules. Having read http://drupal.org/node/1400748, and looked into some Ruby and nodejs projects, the one consistent thing among modern projects seems to be to not put the PSR-0 root at the top of the project (either for an entire system, or a module/plugin/extension to that system). It makes sense to me that core should follow what appears to be a now accepted pattern. Similarly, I have yet to see a project that places an application-level class into a lib directory. ZF2 and Symfony use a src directory for application-level classes, but that was rejected for Drupal in #1400748-30: Proposal for unified namespace organization on grounds that we still have lots of procedural source code too. Ruby and nodejs projects use an app directory for application-level classes at the top-level of a project, but Mark and I couldn't find many examples of modules/plugins/extensions containing application-level code in these systems. We did find one, a Radiant CMS extension, and it does use app for its application-level code. So, of everything I considered so far, app is my favorite convention to adopt for application-level code within core modules.
4. I think it's good for our system to allow contrib maintainers some flexibility in choosing their organization. Best practices evolve. Forcing something on people inhibits evolution.

I'm leaving this at "needs review" to give people a chance to respond. Hopefully, it's close enough to what's already been discussed that we can move this to RTBC fairly quickly, and start moving some module code to classes.

Re #125:

1) Agreed on point 1, /lib as the automatic default. I suspect extremely few contrib maintainers will bother doing otherwise, which is fine. Also agreed on allowing contrib maintainers to register other stuff wherever via an info file, as previously discussed.

2) I have absolutely no idea how you're differentiating between $moduledir/lib and $moduledir/app. That split makes no sense to me at all, as all such code is coming in the same package, a module. There's very little code that we're looking at using namespace discovery for at this point; the DB layer is the only example I can think of, and I'm open to changing that, too, if we can come up with a clean, non-Drupal-based way to do so.

(I know in earlier discussions of the plugin system we talked about using namespaces for organization by plugin type. In hindsight I believe that was a bad idea, and we should not do so afterall.)

But given that, the vast majority of code my module provides will be in Drupal\mymodule. I have no clue how, or why, I'd want to try and differentiate it between "lib" and "app". moreover, as acknowledged in "reasons" point 3 in just about every other project "app" means "site-specific stuff"; that's more analogous to the "Site specific custom.module" that most Drupal sites have in practice.

Unless there's an order of magnitude difference in performance (and I don't really see how), I would not encourage an /app vs. /lib split in modules. Naturally we cannot stop someone from doing so, but I don't see any compelling reason to even mention it as something anyone would want to do.

Crell++ Most module will have a really few code so separating "lib" and "app" would make it worse. Plus I don't think there is really a logical separation between the two now, maybe we could later separate the views and controllers from lib, but that's pretty much it. I don't think they worth it right now (and maybe never).

Frankly I like the info file support, and I'd like to see us only register directories that exist. Can we not just agree to the info spec in 125 and leave it at that?

FWIW I too am not sure I _GET_ the split between lib and app, but I'm sort of proposing we just adopt code that supports both (NOW) and we can continue to bikeshed the "proper" use of it as long as needed. This should make moving code around later pretty easy if necessary and won't prevent me from continuing forward with the blocks & layouts initiative. It looks like we're very close to an agreement here. Is what I'm proposing amicable to everyone? I'd even write the code this weekend if we can get an agreement on it.

Eclipse

Lol, "Eclipse aKa Care Bear^2000"++

My understanding of one of the reasons this issue is pressing, is that EclipseGc is working on making blocks into class-based plugins. Core currently has 15 modules with a hook_block_info() implementation. Does everyone here really feel that these new block classes belong in lib? If so, I'll go along with it. But I'd feel better if some of the current maintainers of these modules give their +1.

#130 can be extended to "candidate subsystems for moving to Plugins / Plugin implementations as classes" - that is, according to the self-stated scope of the Plugins API effort that is now a part of the "D8 blocks" initiative, pretty much anything that is currently exposed using an _info() hook pattern :
- blocks (new D8 initiative)
- Field API field types, widgets, formatters... (work in progress)
- potentially and hopefully: image effects, input filters, views handlers, ...

So just to be clear, the question in #130 is : "should all the classes for 'a block, an image effect, a field type, an input filter...' live in $module/lib, or be targeted to a more specific $module/app folder".

That being said, I don't think I have a strong bias right now, I can live with /lib

Re #130, I feel that all Drupal-module-namespaced code provided by a module belongs in a single directory root in the module, which at the moment would be /lib. So yes, image styles, blocks, fields, widgets, text filters, and all the rest of that jazz should be in /lib.

That's the only disagreement I have with #125, I think, unless I misread something. :-)

Yeah, I would really favor a single dir per module for classes (at least in core). I don't want to shove that down contrib's throat, and I think the info file approach makes all the discussed options viable, which is why I'm pushing for "let's code something and argue the semantics later". I think there's enough consensus at this point to pull that off. If I'm mistaken I'll happily back off, but I would like to move forward if I am not mistaken.

Eclipse

Can we not just agree to the info spec in 125 and leave it at that?

I just chatted with EclipseGc and suggested that I agree with him implementing that at the bootstrap phase where it's possible to do so (i.e., where system_list() is available). For block plugins, that's plenty early enough. For other plugins, we might need to be able to move that to an earlier bootstrap phase, but can deal with that when the use-case arises.

FWIW I too am not sure I _GET_ the split between lib and app, but I'm sort of proposing we just adopt code that supports both (NOW) and we can continue to bikeshed the "proper" use of it as long as needed.

Ok, if you're willing to code up some block plugins in the /lib directory now, but are ok with moving them later should the bikeshed discussions go that way, then great! Go, go, go! :)

As far as getting the split between lib and app, I think it's a fairly common goal among software developers to differentiate between library-level (more generic/reusable) and application-level (less generic/reusable) code, though where to draw the exact line varies from project to project (see for example, http://stackoverflow.com/questions/1270729/difference-between-library-an...). If we're willing to call all core module classes (including block controllers) sufficiently library-like, then I'm fine with /lib. However, if we believe there is a meaningful difference between the two levels, but want a common root directory for them anyway, then I'd rather we rename the folder to /classes or /autoload, or something else that doesn't imply "only library-level code goes here".

However, if we believe there is a meaningful difference between the two levels, but want a common root directory for them anyway, then I'd rather we rename the folder to /classes or /autoload, or something else that doesn't imply "only library-level code goes here".

Ok, one counter to my own argument comes from this Wikipedia entry on Library:

Most Unix-like systems have a "search path" specifying file system directories in which to look for dynamic libraries...Developers of libraries are encouraged to place their dynamic libraries in places in the default search path.

This, of course, includes the well-known /lib, /usr/lib, /usr/local/lib directories. So if the proper way to understand "lib" is the folder where code can be placed and automatically found, then placing block controllers there works for me, and I'll stop insisting on distinguishing library-level module classes from application-level module classes.

Ok, sounds like we have agreement. Paraphrasing, to the best of my understanding:

1) Core uses a core/lib for any "framework" classes (required by core/includes and such).
2) Each module can also have a /lib directory for their classes (blocks, plugins, etc.)
3) If that doesn't work for you as a module developer, we'll also allow additional autoload directories to be specified in .info files.

SHIP IT!

Ok, so let's just keep this moving... something like this?

A few observations and notes:

1.) I'm using drupal_parse_info_file() here and that's really bad because drupal currently doesn't call very often cause it's kind of expensive. In short we should probably have a cache of some sort here, but I just want to get this ball rolling.
2.) I've added this into system_list() because from previous conversations with catch I _THINK_ this is where he had said something like this should probably reside.
3.) It seems totally reasonable to me to implement both a namespaces and a fallbacks array into the info file and let modules define both of these things as I would imagine most classes provided by core (at least) would probably want to allow contrib to override them so they should probably be a fallback and not a namespace, still that seems an exercise that's sort of irrelevant at this stage, but I wanted to mention it.

Eclipse

Made a new issue for the patch #1467126: PSR-0 namespace auto registration for modules

The patch from #137 has been moved to another issue, so back to RTBC.

1. The point of PSR-0 is to enable autoloading without a complex registry.
2. Another point of PSR-0 is to enable autoloading without dependencies on expensive infrastructure (like the database system and module system).
3. The PSR-0 standard was designed for developer frameworks and does not account for modular, extensible, configuration-based, profile-based, multi-site systems. (#54)
4. Therefore, the minimum information required to enable autoloading for system extensions is
   1. a list of enabled system extensions
   2. the filesystem path to each extension
   3. a default file naming pattern for autoloadable code within each extension
5. This additional, manual registry allows the system to find specific code in a guessed file name of a specific extension out of 100+ extensions, without checking all 100+ extensions for the file.
6. All code outside of the namespace/scope of an extension cannot be autoloaded through the default naming pattern and has to be registered manually.
7. Despite being incompatible with the system's handling of extensions, there is a desire to retain the structure of the PSR-0 standard at the extension level. (This desire is not backed by technical reasons, only personal opinions.)
8. Due to that, the otherwise simple and logical mapping of a \Drupal\Module\Foo namespace to its actual location sites/[site]/modules/foo as default namespace mount point was dismissed early in the discussion (for non-technical, unknown reasons).
9. Instead, the idea is to duplicate an extension's PSR-0 namespace within the extension's actual filesystem namespace. This idea cannot be expressed simply in a sentence, as it requires additional pointers:
   

   [namespace]                 »  [filesystem namespace]  /lib/[PSR-0 namespace]
   \Drupal\foo                 »  sites/[site]/modules/foo/lib/Drupal/foo
   \Drupal\foo\Block\Settings  »  sites/[site]/modules/foo/lib/Drupal/foo/Block/Settings.php
   

10. To separate the already PSR-0-namespace-separated code of the extension from the extension's other files, the default naming and lookup pattern uses a lib/ directory between the filesystem namespace and PSR-0 namespace. The given technical reason for that is to prevent false-positive lookup results if an extension developer puts non-autoloadable code into the file structure for code to be autoloaded, and any extension in the system happens to attempt to load exactly that name in that extension through the autoloading mechanism.
11. With all of that structure and logic in place, the system will go ahead and
    1. automatically register [filesystem-namespace]/lib as PSR-0 namespace for each extension, so the extension's code is looked up in [filesystem-namespace]/lib/[PSR-0 namespace] by default. For example,
       • core/modules/node/lib is registered for the node extension's \Drupal\node namespace, so the actual autoloadable files live in core/modules/node/lib/Drupal/node
       • profiles/portfolio/modules/bar/lib is registered for the bar extension's \Drupal\bar namespace, so the actual autoloadable files live in profiles/portfolio/modules/bar/lib/Drupal/bar
       • sites/all/modules/foo/lib is registered for the foo extension's \Drupal\foo namespace, so the actual autoloadable files live in sites/all/modules/foo/lib/Drupal/foo
    2. check each extension's meta information for
       1. an overridden relative sub-directory name for the filesystem namespace, which adjusts the lib/ directory part in the automatic default naming and lookup pattern above (but not the PSR-0 namespace, nor the extension's namespace in the filesystem).
       2. any additional namespaces of code that lives within the extension's filesystem namespace but is not within the extension's namespace. These are relative sub-directory names within the extension's filesystem namespace, which each denote the custom mount point for the PSR-0 namespace of the foreign code. For example,
          • a value of lib/ for \Drupal\baz makes the autoloader search in sites/all/modules/foo/lib/Drupal/baz
          • a value of vendor/ for \OAuth\Client makes the autoloader search in sites/all/modules/foo/vendor/OAuth/Client
       3. any additional, custom names of code that lives within the extension but not in any namespace.

If this is RTBC, this really needs an issue summary PLZ!

Given #140, the current proposal feels too complex and overengineered.

1. Since any code outside of the extension's namespace has to be manually registered either way, duplicating the entire PSR-0 namespace into the extension's (filesystem) namespace is superfluous. It only seems to be proposed for the sake of making PSR-0 purists happy, but has zero technical merits otherwise. I disagree with that duplication.
2. The PSR-0 namespace structure only needs to be fully contained for code that is shipped with an extension but which is outside of the extension's namespace. These additional, custom namespaces need to be registered manually. They are entirely new, and neither known, nor part of the regular existing namespace structure. So it's only logical that the full PSR-0 namespace directory structure needs to be contained in the mount point that has been registered for a custom namespace. I agree with that for custom namespaces.
3. Any code outside of the extension's namespace cannot be and will not be autoloaded by default (without manually registering it). Thus, putting everything into a single container is misleading and inappropriate:
   

   sites/all/modules/foo/lib/Drupal/foo/Block/Settings.php
   sites/all/modules/foo/lib/Drupal/bar/Block/Settings.php
   

   No, \Drupal\bar\Block\Settings will not be autoloaded by default.

4. ./sites/all/modules/foo is the extension's namespace already. PSR-0 is merely incompatible with a system that's compound of self-contained extensions. We only have ./sites/all/modules/foo, because we don't want to go the exorbitant mile and have ./Drupal/Module/foo instead.
5. Because of that, it makes much more sense to use a default naming and lookup pattern of:
   

   \Drupal\foo\Block\Settings  »  sites/all/modules/foo/lib/Block/Settings.php
   

   If your extension ships with external code, register its namespace in the filesystem (which happens to be in your extension):

   \Flickr\Client              »  sites/all/modules/foo/vendor/Flickr/Client.php
   

See working implementation in attached patch.

This is not PSR-0?

Yes and no. This could rather be considered as PSR-0-NG, because PSR-0 does not allow to alias a specific sub-namespace to a different directory.

We need to write autoloader code for this?

Yes. We are PHP developers.

Will other PHP developers understand this?

Yes. It's a small tweak to PSR-0 only. PHP developers should be able to understand this enhancement.

re #144 PSR-1 is "a standard coding convention for PHP." I don't think that's what you mean.

The proposal in 144 is to NOT use PSR-0 for modules. The proposal I am reading is to use a custom invented in Drupal autoloader that extends and makes use of the Symfony UniversalAutoloader - but one that does not rely on the PSR-0 standard. While a valid option, it is not one that I agree with.

The issue is called 'Namespace strategy for module-provided classes'. What I am hearing is that modules developers get in the bad habit of creating classes that aren't namespaced in a Standard way.

By this I mean, while the namespace is defined in a PSR-0 compatible way - the directory structure is not. That means that if I really like \VenderFoo\Awesome\Class and want to use it in a different framework that is PSR-0 compatible, I can't just grab it and go without first re-creating the correct directory structure.

Also, it doesn't allow me to define or use a different autoloader class that *is* PSR-0 compatible - without first doing a similar hack.

Re: Vendor name: Reading back through the issue at some point the idea of enforcing the 'Vendor' portion of the namespace was pooh poohed. Unfortunately its not optional for PSR-0. I saw some talk of forcing all module maintainers to adopt a DrupalContrib (or similar) Vendor name: this is kind of arrogant on the part of the project. Why shouldn't a Vendor (a single developer, a company, a consortium, any group agreeing to fly their code under a single flag) just deliver code under their own Vendor name? If I've misunderstood what's been discussed ignore this part of my comment.

#136 Offers hope. I think this is my understanding as well.
i.e. those namespaces are registered at path_to_module/lib/

I strongly disagree with the pseudo PSR-1 way sun is proposing. Either we're doing PSR-0 either we're not: saying we do PSR-0, doing it partially for core, but having exceptions for modules *is* confusing and actually breaks the standard we are trying to catch up with. I'm against Drupal WTF and Drupal-ism if it implies bending a standard and lying saying we meet its criteria at the same time. The extra folder are not that bad, please make peace with it: using a good IDE anyway will shortcut it for you, and not using a good IDE is suicidal if you develop PHP 5.3 namespaced code anyway.

Note: Anyone calling the proposed non-PSR-0 class loading strategy "PSR-1", this is one of the rare times that I can say outright: "No, you're flat out wrong, period." PSR-X means very specifically standards issued by the PHP Framework Interoperability Group. Their second standard issue (PSR-1) looks like it will be either about caching interfaces or code formatting standards. NOT an extension to class loading. Calling any sort of alternate or extended class loading "PSR-1" is, quite simply, wrong and misleading. Please stop now. You may as well start calling it IETF RFC 7652, which would be equally flat-out-wrong.

As for extending the class loader spec, honestly, my position is simple. It's a directory. It's a directory that people coming from any other modern project (who we want to bring into Drupal) are going to expect, and get confused if it's not there. Get over yourselves. I swallowed my pride and recommended we move to PSR-0 in the first place, despite being an opponent of PSR-0 originally. You can to.

Is this PSR-1 an actual proposal for the Framework Interop Group? The only thing I've seen called PSR-1 outside of this discussion is a proposal for PHP coding standards across frameworks. Lets not call it PSR-1 - its confusing.

@donquixote please don't pull quotes out of context and pick on them. I was simply re-stating my understanding of the patch and concluded with "While a valid option, it is not one that I agree with." i.e. you could implement it this way, but I don't think its a good idea.

Most of the rest of what I was writing was making a case for interoperability. You can't be kind-of interoperable.

+++ b/core/lib/Drupal/Component/ClassLoader/ExtendedUniversalClassLoader.phpundefined
@@ -0,0 +1,152 @@
+ *     $loader = new ExtendedUniversalClassLoader();
+ *
+ *     // Register vendor classes with namespaces.
+ *     $loader->registerNamespaces(array(
+ *       'Symfony' => __DIR__ . '/vendor',
+ *     ));
+ *     // Register our own framework classes with namespaces fallback.
+ *     $loader->registerNamespaceFallbacks(array(
+ *       'Mine' => __DIR__ . '/lib',
+ *     ));
+ *     // Register extension classes with namespaces.
+ *     $loader->registerNamespaceExtensions(array(
+ *       'Mine\MyPlugin' => __DIR__ . '/extensions/myplugin/lib',
+ *       'Mine\ThirdAddon' => __DIR__ . '/extensions/thirdaddon/lib',
+ *     ));
+ *     // Activate the autoloader.
+ *     $loader->register();
+ *
+ * In this example, if you do this:
+ *
+ *     new Mine\MyPlugin\Block\Client();
+ *
+ * the autoloader will look for files in this order:
+ *
+ *   1. extensions/myplugin/lib/Block/Client.php
+ *   2. lib/Mine/MyPlugin/Block/Client.php

What is trying to be accomplished here is possible with the Composer ClassLoader.php.

I've put together an example at the Symfony Update module. The idea is to replace the provided Symfony version provided by the Symfony module. The key is to add all the new namespaces, and then when registering them, make sure to pass TRUE to tell the ClassLoader to prepend the new namespaces rather than adding them at the end of the discovery trail.

Here's another example of a Config Update module that would update some of the components in Drupal\Core\Config:

function config_update_init() {
  // Retrieve our instance of the Composer ClassLoader.
  $loader = drupal_classloader(); // Needs update to switch to Composer's ClassLoader.
  $path = drupal_get_path('module', 'config_update');
  $loader->add("Drupal\\Core\\Config", $path);
  // Register the new namespace, making sure to prepend the namespaces.
  $loader->register(TRUE);
}

Now loading Drupal\Core\Config\DrupalConfigVerifiedStorage will check modules/config_update/lib/Drupal/Core/Config/DrupalConfigVerifiedStorage.php before hitting core/lib. Once #1424924: Use Composer for updating Symfony components (without removing Symfony code from repo) is in, we'll be able to do some of this. It would be nice to be able to declare some of these in the .info files via namespaces[] and namespacePrepends[] or something like that, but that's for #1467126: PSR-0 namespace auto registration for modules.

I understand that it's nice to write and maintain all our own code, but why do we need to if this already works for us, with working, proven code.

24 days to next Drupal core point release.

Any modifications to PSR-0 are off the table IMO. If we do that, we're breaking a standard and creating another Drupalism before we've even attempted to use it the way it was designed.

The idea that PSR-0 isn't designed for extensions therefore it's inappropriate for our module system is just wrong. Look at how this is handled by composer. It's a solved problem.

Doing what's proposed in #144 adds a significant amount of complexity to our implementation because we're breaking the standard. What do we gain there? Fewer directories? Are we really going to keep holding this issue up because of that?

Let's please adhere to the standard and move on to more important work.

Agreed. Any comment in this thread after #139 is just a waste of everyone's time. Really, this issue is done. It's dead. Implement it in #1467126: PSR-0 namespace auto registration for modules and be done with it, so we can move on to something that actually matters. There's enough layers of paint o this bikeshed as is.

Since the strategy has been concluded and implementation is proceeding elsewhere, this issue is now fixed.

The point of #144 is:

You cannot re-use classes of Drupal modules/extensions somewhere else either way, as they depend on Drupal APIs and subsystems.

If you really want to write re-usable code, then that's most certainly not going to live in the module/extension itself, but in a separate namespace instead.

Separate namespace means manual registration, which points to a directory that contains the full PSR-0 structure.

Actionable items to think about?

• Start using Drupal\Components and decouple systems via the Framework Initiative
• Keep up with the Unix Philosophy
• Better use of interfaces and live by the Open Close principle
• Dependency Injection (or something similar) for extension discovery

WP and Joomla are not using namespaces yet. All of the PHP 5.3-based frameworks I looked at are using PSR-0.

#155: Your premis is based on the assumption that a module will never contain Drupal-agnostic code, and therefore compatibility doesn't matter. That's nonsense. 1) Drupal-agnostic code is something Drupal needs more of. Really, that's just basic good architecture and programming practice 101. Fewer dependencies == good. Second, Drupal\Component is exactly for Drupal-originated but separable code. It doesn't have to have a separate namespace. Third, even if no code is reused outside of Drupal using the same class loading strategy as every other PHP 5.3 project improves DX and helps new developers get aclimated.

Rob Loach: Yes, Yes, Yes, and Yes. Not in this issue, but yes. :-)

There is no further debate to be had here. Drupal is not a special snowflake. It's just a PHP project, and if we insist on wasting our time figuring out how to be different, we will be dead. It's that simple. Let's get back to doing something useful, which this thread no longer is.

Drupal\Component is exactly for Drupal-originated but separable code. It doesn't have to have a separate namespace.

If you use the Drupal\Component namespace in a module/extension, that code will not be autoloaded. You need to register that component's namespace manually first.

The envisioned approach of #144 does not prevent you from using the full PSR-0 structure for all of your module's namespaced code at all. So if you really believe that your module's classes are so abstract and generic that they ought to be a stand-alone library, then you can still register and use the full namespace for your module.

For most other Drupal modules/extensions, that won't be the case. So why force it on everyone?

I agree with closing this issue as fixed, and moving on (for now) as per #136. Specific implementation follow-ups have been created and linked to in #138 and #152. However, I don't feel that enough respect has been given to the DX question, but I think the only way for that to change is for the people who this will most impact to join the conversation, and they won't do so on an issue that's already 160 comments long. Therefore: #1469102: Gather feedback about DX of Drupal\user\SomeClass existing in deep directory core/modules/user/lib/Drupal/user/SomeClass.php.

This issue is entitled "policy, no patch" but I have no idea what the final policy is. And that's even after I read through all 160 comments, which 99% of core developers aren't going to do...

For that reason alone I don't think this issue can be considered fixed yet. We could really use some simple documentation or an issue summary that core developers can quickly understand. I'm not sure what the proper status is, so I'm going with "needs work".

***

On top of that:

1. To the extent that #136 is the policy that was agreed upon, that seems pretty straightforward, but I question its usefulness because as far as I can see, @sun's patch in #144 is completely compatible with #136, but everyone is at each other's throats over his patch anyway :) So if we stop there we're really just deferring the same arguments to another issue.
2. To the extent that people instead want to call this "fixed" based on a PSR-0-only approach, it's pretty clear that there is no consensus whatsoever on that point. If this issue were blocking lots of other issues, I could see the rationale for rushing it in anyway, but I can't think of a single initiative or major Drupal 8 issue that would actually be blocked on this decision. If there is one, can someone share what it is (and why it technically requires adding namespaced classes to a module)?
3. The one issue that I can think of that actually would be blocked on this is one which (as far as I know) hasn't even been filed yet, but I think is an undertone in a lot of the discussion here anyway, and that is something like "Remove the registry and replace it with a filesystem-structure-based autoloader such as PSR-0". It's important to note that we actually discussed doing something like that even in Drupal 7; see #497118: Remove the registry (for functions) for that discussion. The decision was eventually made not to, for a number of reasons. I'm not bringing that up to say we shouldn't do it (I was mildly in favor of it before and still am now), but just pointing out that this issue may be putting the cart before the horse in some ways; we have other things that need be discussed before we could do that, which go beyond a discussion of how many levels deep the proposed file hierarchy should go.
4. What I think would be a mistake would be to go ahead and mark this issue "fixed" and then have a bunch of patches that add namespaced classes to various places in core, then wind up with a Drupal 8 release that has two or three different class autoload strategies in use by modules (some using files[] and the registry, others using namespaces[] or whatever it winds up being called, and others autoloaded automatically without explicitly registering themselves)... I don't think we need #1469102: Gather feedback about DX of Drupal\user\SomeClass existing in deep directory core/modules/user/lib/Drupal/user/SomeClass.php to know that combination would be bad DX :)

" I can't think of a single initiative or major Drupal 8 issue that would actually be blocked on this decision"

The Layouts initiative, at least, is currently blocked on this decision, because part of it is moving module-exposed blocks to classes, which requires some kind of agreement on where those classes should live. Anything else that attempts to move to plugins (e.g. Fields, etc.) will also be blocked on this decision, too.

I don't think anyone is advocating for #4. I think we're saying "Can we please just say 'good enough for now' and move on, so people can hurry up and get crackalating on D8?" Now that we're on Git, we can move files/directories around very easily if we decide later to revisit this decision, but continuing to not make a decision here is completely gumming up the works (not to mention completely destroying morale).

Fair enough point on the lack of a clear issue summary/conclusion, however. Tagging.

The Layouts initiative, at least, is currently blocked on this decision, because part of it is moving module-exposed blocks to classes, which requires some kind of agreement on where those classes should live. Anything else that attempts to move to plugins (e.g. Fields, etc.) will also be blocked on this decision, too.

We already have dozens of classes in core modules now (not to mention hundreds or thousands in D7 contrib), and they all already live somewhere in the filesystem which hasn't hurt anyone up to this point :)

What is special about the proposed new classes that makes them different?

Because we don't have any psr-0 classes in modules yet?

which hasn't hurt anyone up to this point

You probably are the luckiest person on earth if you never experienced problems with the core autoloader. Because the legacy core autoloader is not a viable solution, and when it breaks it becomes a real nightmare to fix. files[] must die. And PSR-0 code can have a 100% db/cache free autoloader which is easier to maintain, quicker to bootstrap.

From my understanding, we already have our action item over at #1467126: PSR-0 namespace auto registration for modules.

"We already have dozens of classes in core modules now (not to mention hundreds or thousands in D7 contrib), and they all already live somewhere in the filesystem which hasn't hurt anyone up to this point :)"

Ok, fair point. :) However, we made the decision some time ago to move to PSR-0 for autoloading. So what I'm hearing from various folks here is that adding new classes to core that do not conform to this just creates a bunch of clean-up work for afterwards which we might not actually have time for. Versus "wholesale rename/move directory foo to directory baz" is a much easier thing to script, especially if everything's in a consistent pattern (though this will only happen if we actually make a decision here and move on).

However, we made the decision some time ago to move to PSR-0 for autoloading.

Where? I follow Drupal core development closely, but this is actually the first I'm hearing of that... We definitely did decide to move classes in core/includes to PSR-0 (in #1240138: Adopt PSR-0 namespace convention for core framework classes), and there are a number of good technical and social reasons for that. Modules, however, are a whole different ballgame, and I thought discussing that was the entire point of the current issue.

I think we need to keep in mind that there is a big difference between:

1. Forcing all modules to use namespaces if they want to have their classes autoloaded (and therefore converting all core modules to use namespaces).
2. Supporting modules which want to use namespaces.

This issue is only "blocking" other initiatives (and even then only pseudo-blocking) if we've decided to go with #1.

But if we've decided to go with #1, that is much more than a bikeshedding discussion; there ought to be a good technical reason. As mentioned, one possible reason would be to allow us to ditch the registry. However, the tradeoffs there are the same as they've always been: Any file-structure-based autoloader (namespace-based or not, PSR-0-based or not) only works if we force everyone to conform to it. In Drupal 7, we decided not to force people to do that, which is why we have the registry. In the discussion here, it sounds like people don't want to force it either, which is why ideas such as namespaces[] in module .info files have been proposed, and why the implementation issue in #1467126: PSR-0 namespace auto registration for modules is starting to go in that direction. What I don't understand is how that is going to be any less brittle than the registry. You will still have the problem where a module's code can change in a way that Drupal crashes before it's able to figure out where the new code is. So it seems like if we did that, we'd be imposing namespaces and all their various complexities on all contrib developers with no real benefit to show from it.

So really I think we need to either:

1. Bite the bullet and say we're going to ditch the registry and force all modules to use the same file structure for their autoloaded classes, period. If so, @sun's proposal seems very competitive and needs to be seriously considered.
2. Support namespaced classes in modules as an optional thing. In that case, we don't need to provide infinite flexibility either (since it's optional in the first place), so there's still no need for namespaces[] in .info files, and it probably does make sense to just use PSR-0 because "library-like code that lives in a Drupal module" is the exact kind of code that we'd most expect to use this. But again, if we do that, I don't see how this issue would be blocking anything. I don't think that any classes that currently live in core modules are particularly "library-like" (maybe some in system.module but if so those would make sense to move to core/lib anyway), nor does it sound like any of the ones that are being proposed are either... block and field API plugins, for example, certainly sound very Drupal-specific to me.

My understanding always was and is that we want to remove the registry entirely. All OO code will be namespaced and thus autoloaded.

The class loader supports different methods for registering namespaces, and it's those methods that we are going to make available in .info files. The class loader also supports to register non-namespaced code to a limited extent. If these methods are not sufficient for some particular code for any reason, then a module has to add its own class loader (that is executed on top of core's then).

The manual registration of additional namespaces in .info files is optional, however. As outlined in #140, a naming and lookup pattern based on PSR-0 will be the default. This means that all namespaced classes in every module (which follow the default pattern) will be autoloadable without any extra .info file properties.

The remaining disagreement is on the concrete default pattern for modules/extensions. As outlined in #144, there's a relatively large negative impact on DX for Drupal module/extension authors, as the current proposal foresees to enforce the full PSR-0 directory structure on them, starting below the lib/ folder in a module/extension.

To remedy the negative DX impact, I've proposed an adjustment to the class loader, so that the default naming pattern understands the lib/ folder within an extension as an alias to the extension's primary namespace, since the extension's directory in the filesystem sufficiently declares the extension's namespace already (i.e., \Drupal\[modulename] is aliased to sites/[site]/modules/[modulename]/lib).

The proposal in #144 still allows developers to use the full PSR-0 structure for their namespaced files by registering the namespace in the .info file. (This is not yet part of the patch though.)

In general, the expectation for D8 is that all modules will contain namespaced classes. At minimum, the Layout initiative intends to convert all page callbacks into so called block plugins (please refer to its roadmap/documentation for more information). In turn, I personally expect namespaced code in pretty much all Drupal 8 extensions (core + contrib + custom). In turn, almost all developers will have to deal with namespaced code in extensions.

The proposal in #144 still allows developers to use the full PSR-0 structure for their namespaced files by registering the namespace in the .info file. (This is not yet part of the patch though.)

If we only wanted to support PSR-0 and "PSR-0-NG" (or really any small, finite number of possibilities) couldn't we just register them both automatically? Then we wouldn't have to rely on the .info file for critical information. We could skip that entirely, or still have it but only rely on it as a suggestion for where to look first (if that even matters for performance).

But like I said, if we instead allow people to register any random namespace structure they want to in the .info file, it seems like we'd be right back where we are now with files[] and the registry. Perhaps it wouldn't be quite as bad (since developers might be less likely to reorganize entire namespaces than they are to move individual classes) but I don't think the frequency is the main issue now either. Most of the time, the registry works fine; the issues come when one particular module just happens to make the "wrong" kind of change, and then thousands of sites whitescreen when updating their code.

@David, how the classes actually get registered is in #1467126: PSR-0 namespace auto registration for modules, I think I agree with you on namespaces[] at least for getting it going in the first place, but it's not really about the discussion here which is where the default namespace on behalf of modules would live. That needs to be resolved regardless of namespaces[] or not. I've posted (twice, while writing this) on that issue but that particular discussion feels like it's going to further derail this one to me.

I would be very happy if Drupal 8 did not ship with the registry, see #1320650: Stop scanning /includes with the registry which is part 1 of removing it. That should not block any other issues though really (unless someone wanted to refactor the existing registry or something).

@pounard: the autoloading of classes for modules is not going to be 100% cache/registry/database free, because the module list itself relies on all of those things. Disentangling the dependencies of the module system itself is something that really needs to happen, but switching away from the registry does not solve it, it's a tiny, tiny step towards that maybe and will hopefully remove some of the symptoms if we can avoid making some of the same mistakes, but it's not a panacea.

I don't like the 'double namespacing' of having the module folder, then a subfolder, then another folder with the module name again, but I also don't care that much at this point.

I believe quicksketch was at one point talking about adding back root modules and themes directories (i.e. /modules and /themes and possibly even settings.php at the same level as /core, and then keep the /sites directory for multisite only). This would mean /modules/mymodule/lib/MyModule/class.php which is still a bit redundant, but actually the same number of directories as sites/all/modules/mymodule/class.php when looking through an install. Even when developing contrib modules, I sometimes do so from the root directory of a Drupal install, so for me at least this would ameliorate the 'more typing' argument, and I can't get excited about the ugly directory structure itself either way.

This issue feels like it's at stalemate, and I can't find the energy to have a strong opinion on it other than wanting it to be over, so I'm assigning it to Dries so he can chime in if he wants.

1) We don't want to rely on cached directory and file scans for autoloading, and we don't want crazy big info files.

Everyone who is saying this (it is not just donquixote) please stop, because it's not true. If we're registering stuff for modules, we need to know where the module are, that list depends on 'cached directory and file scans', and it's unlikely to stop doing so (it might be possible to make it more sane, but it's not going away unless we drop multisite features that have been in core for many years and tens or hundreds of thousands of sites rely on). It will remove some file directory scans/caching, but not all of it, so there is still a dependency there.

I'm fine with sending this to Dries to be the final arbiter, but we need to get an updated issue summary here. It's already tagged, so I dunno what else to do, but if Dries needs to find time in his schedule to read 177 comments, I can guarantee this issue will go nowhere until August. :P

@David, how the classes actually get registered is in #1467126: Provider based PSR-0 Classes, I think I agree with you on namespaces[] at least for getting it going in the first place, but it's not really about the discussion here which is where the default namespace on behalf of modules would live.

Thanks, I was trying to figure out the difference between this issue and that one.... still not totally sure I get it, but it makes more sense :)

But the reason it's somewhat relevant here is that there was a lot of discussion above along the lines of "let's just pick a default without worrying much about it, because if people don't like it they can override it in the .info file". What we're saying is that unfortunately that may not be the case. But per your comment in the other issue, maybe modules that want to do something different can actually just register their own namespace directly at an early enough point in the bootstrap instead. That would be more straightforward; I like it a lot.

I believe quicksketch was at one point talking about adding back root modules and themes directories (i.e. /modules and /themes and possibly even settings.php at the same level as /core, and then keep the /sites directory for multisite only). This would mean /modules/mymodule/lib/MyModule/class.php which is still a bit redundant,

For the record, I think it would actually be modules/mymodule/lib/Drupal/MyModule/class.php, which is one level deeper...

For the record, I think it would actually be modules/mymodule/lib/Drupal/MyModule/class.php, which is one level deeper...

Whoops that is true, off by one errors etc. :p

Also, I think the agreed upon namespace is lower casing the module in the namespace so for user module the directory would be:

core/modules/user/lib/Drupal/user/MyClass.php

Entirely revamped summary.

Thanks. I've entirely revamped the summary, keeping the content, but shortened, and using clearer structure.

Updated issue summary.

Updated issue to reflect the PSR-0 disagreement since really that's the only thing we've ever argued. The folder structure if we do PSR-0 just needs a decision and I think everyone one will deal with it.

I really think think this issue is wrongly named though. We need a PSR-0 for modules policy and I'm tempted to say if you want something else, open a new issue for a separate autoloader. The core autoloader is PSR-0 and if we're going to support something else it should be a different autoloader not something duct-taped to the side. This is a separation of concerns plain and simple.

I would like to propose that we use the full PSR-0 directory structure in modules for now (just so that we can make some progress). Then, if it proves to be too painful, then we can revisit it.

I corrected the summary, because it is a DX problem to not follow PSR-0 as well, as it would be confusing for new developers expecting it.

From http://drupal.org/principles:

Standards-based. Drupal supports established and emerging standards. Specific target standards include XHTML and CSS.

PSR-0 is an "established and emerging standard". We've already decided to follow it. Let's do. If we want to also provide an alternate, non-standard, proprietary class loading strategy as well, the onus is on the supporters of such to justify not following an "established and emerging standard".

Updated issue summary.

To reflect @donquixote's comments, I updated the summary as follows:

• For "Goal", I changed "how" to "if and how".
• For "Details", I added a bullet point clarifying what PSR-0 requires.
• For "Agreements", I changed the first item from saying that we agree to PSR-0 to saying that we agree to using namespaces within modules, and a directory structure that supports autoloading without a per-class registry.

The summary currently states that we disagree on whether to make the default pattern for modules PSR-0 or not PSR-0 (that's what it already said, I did not change that). If that's true, then that's a blocker for #1467126: PSR-0 namespace auto registration for modules, which itself is a blocker for converting blocks and other things to plugins. I haven't counted, but I think the majority of people on this thread prefer to proceed with PSR-0 than not. Therefore, does anyone object to moving that to the "temporary agreements" section, or otherwise suggest how we can proceed with #1467126: PSR-0 namespace auto registration for modules and the plugins work?

Therefore, does anyone object to moving that to the "temporary agreements" section, or otherwise suggest how we can proceed with #1467126: Provider based PSR-0 Classes and the plugins work?

I still don't understand why this issue is a blocker for plugins. I asked above and never got a good answer. The best I heard was that people don't want to write code/classes now that might have to get renamed+moved+namespaced later, but (a) if everyone thought that way, no new features would get added to core ever, (b) it's really not that hard to rename a class, and (c) if the plugin stuff gets committed first, it won't even be the job of people working on plugins to rename them.

I would also assume that any work on converting, e.g., all blocks and page callbacks to plugins would start with a small example first (e.g., one core module) and get that functionally working before proceeding to a mass conversion. So we're a long way away from the point where the number of classes that would need to be renamed+moved is even a large number.

So in short, isn't the way to proceed with plugins to... just proceed with them?

The best I heard was that people don't want to write code/classes now that might have to get renamed+moved+namespaced later

I don't think there's been much objection to moving files around later if needed, but I can understand why the people who are most interested in writing the plugins (at least the first few) want to do so using namespaced classes, rather than starting down a road of non-namespaced classes when it appears that ultimately, there's agreement to have them be namespaced eventually.

Until a PSR-0 autoloader is in place, just list those files in the info file.

According to @Crell in IRC, our current registry doesn't work for namespaced classes.

Given above though, I'm not really sensing any disagreement with proceeding with finishing #1467126: PSR-0 namespace auto registration for modules and using that (PSR-0) for the initial plugin work. Am I reading the temperature right on that?

I think our summary is mostly good now, so removing the tag.

Updated issue summary.

I added a "Follow ups" section to the summary reflecting #194. I'm assuming someone will comment here if they object to that.

Updated issue summary.

EclipseGc might have a different mileage, but as far as "plugins-oriented Field API" is concerned, this is not really a blocker. I simply use the existing autoloader for the Drupal\Core\* namespaces for now, and put the files under the corresponding folder in core/lib. I'll just move them under their regular module when something gets in. While my API is still in flux, keeping classes next to each other is easier anyway.

Aside from that, of course, the sooner we get "some" resolution the better :-)

Yeah I actually wrote up a response to this a couple days ago and decided not to post it in the hopes that we had reached a resolution for the time being, however all my plugin & blocks work is PSR-0 in nature, and I have a lot of stuff already done and working within that paradigm. I'd prefer to not have to move it out of psr-0 and back again, so resolution here is a definite must in the very near term future in my opinion. The number of classes in my initiative is already close to 20 and that number is only going to grow. I definitely consider this to be a "blocker" and would like to see it resolved at or before Drupalcon if possible. We really don't have a lot of time left between now and Dec 1st, and this affects core, test, and at least my initiative if not others.

We have been arguing this since late September of last year. We can afford to argue it NO LONGER. PSR-0 is in, it's working, and it's good enough for other php systems. WHY then do we continue to suggest some drupalism? I have not yet seen how PSR-0 is insufficient in any way for our needs. It simplifies much of our existing registry needs (should we continue to maintain one of those) and it is already an agreed upon standard in the greater PHP world. We claim to care about standards, can we please just embrace this one and move on with life?

Regarding #200, and the general idea of supporting both PSR-0 and PSR-0-NG-like hybrids, I'd just like to point out that one of the loudest criticisms of Drupal's DX is its inconsistency and unpredictability. So having a \Drupal\Foo\Bar namespace whose class sometimes is in directory A, and other times is in directory B, and still other times might be in directory C, smells like an expletive-filled rant from the first people who try and do anything useful in D8, or worse, try and debug it.

So I think we just need to pick one way and move forward, personally. Allow it to be overridable, sure, but core should set one clear pattern and 99% of contrib should conform to it. Failing to be decisive here in order to appease module developers is just going to make "normal" developers' lives more miserable.

Although I'm only core issue queue lurker, not core developer, and I will not be able to write sufficient modules until D8 ship out - I was thinking same as webchick @ #202: be consistent and move on forward, please.

My two cents for this discussion and community consensus.

I agree. Let's just use PSR-0 and get on with it. Consistency is a good thing, and so are standards.

Changing title to reflect what the issue is proposing.

It is in this kind of issue (over 200 comments on 7 months) that I'd like Dries to make a decision.

All sides have their pros and cons, and they're all on this thread.

Now, let's get over it and go make a decision.

We need to move on.

@Ralt: It's been assigned to him but he's very busy and travels a lot so probably just hasn't gotten to it, yet.

Use. standards. PSR-0++.

PSR-0 is a standard. I expect more projects to adopt it, so if we take a long term view, I think we should adopt it too. So I approve the patch that has been committed. The only thing that is unconventional is that sites/all is in all path, not the fact that there are many subdirectories under modules/$module. We may be able to address that later, but what we have is good for now. Marking this 'fixed'.

Mention that "PSR-0-NG", while home-made, is not totally arbitrary.
Link to xautoload as a demo implementation.