Move token.inc to a module

.

Hrm, the token API benefits contrib *greatly* by being available by default in core and not being an optional dependency, so I have to disagree with this for now.

I think token.inc provides great benefit, but I think we should seriously consider making it a module. It's just that much less code that has to be executed on every page load. If I have a contrib module that needs it, it will get turned on automatically for me.

@davereid, is there any major problem with making contrib modules declare 'dependencies[] = token'?

Because now http://drupal.org/project/token is screwed out of being the testbed for moving code into core.

@Dave Reid: Nothing really to worry about, you can keep the project namespace, just simply rename the contained module to token_ui.

It's not just a token UI - it's a testing ground for tokens that should have been in core. And hopefully we'll have moved the UI part into core for D8.

Okay, so can we get the rest of token moved into core, add the tokens that should have been available, and then take care of this issue? Would that be acceptable, Dave?

I think token.inc provides great benefit, but I think we should seriously consider making it a module. It's just that much less code that has to be executed on every page load. If I have a contrib module that needs it, it will get turned on automatically for me.

Just to put things in perspective, token.inc includes 86 lines of PHP code: everything else is PHPDoc.

The only bulky code lives in the token implementation files scattered across all of the core modules that provide tokens, but that code exists in separate .inc files that are only loaded when required. We'd basically be trading a more complex dependency chain for 86 lines of PHP in an area that has no UI or interaction complexities. Just maintaining a pre-populated list of country names is in iso.inc is, literally, several times larger.

Okay, so can we get the rest of token moved into core, add the tokens that should have been available, and then take care of this issue? Would that be acceptable, Dave?

The problem isn't that a couple tokens got left out. It's that when complex features get added to core, figuring out how to best implement token support for them takes time and architectural (not just development) resources. Unless "full, flexible, extensible token support has been added for all entities, data types, and so on" can be made into a release blocker for new versions of Drupal, there will always need to be some sort of 'token module in contrib.' It shouldn't be a show stopper if moving the .inc to a module is the right decision, but it's not just a matter of getting a couple of 'missed tokens' in.

I had a look to see how much token is represented in system module, since that's usually a good sign that something has gone wrong:

Yeah, that strange stuff in hook_hook_info is a direct legacy of the registry related stuff. Originally, token had a several line helper function that looked for .token.inc files in each module's directory, very similar to what Views does. That was made unnecessary by the registry, but then the registry got pulled out and a special token-only mechanism for locating .token.inc files was determined to be too weird.

As has been mentioned in the main framework metathread, the inability to effectively make things 'optionally loaded' without putting them in classes or putting them in modules is the underlying problem. Token is represented in system.module because 'optionally loading things' got pushed off onto yet another hook, and hooks require modules.

Then there is also system.tokens.inc, but many of the tokens that are in there are implemented on behalf of other stuff that is misplaced, and none of them are needed to have a token replacement API.

Yeah; there are a few things like date handling and site information that -- due to the nature of the hook system -- have to live in some module but are not clearly linked to any single module's functionality.

Discussed with @catch in IRC that we first want/need to do a full run-down on all /includes files to come up with a clear idea and rules for what can be or should be an include and what shouldn't. #1224666: [meta] Unofficial Drupal 8 Framework initiative only covers the high-level ideas around that, but for issues like this, we need a low-level consensus.

Thus, marking this particular issue as postponed until that consensus has been reached.

Then there is also system.tokens.inc, but many of the tokens that are in there are implemented on behalf of other stuff that is misplaced, and none of them are needed to have a token replacement API.

This quotes makes me wonder if makes sense to use the namespace tokens (note plural) in core and leave token for development in contrib for further architectural design and development. Then everything needed for tokens can be moved out of the system module and hopefully can reduce some of the frustrating dependencies.

The only stuff in system.tokens.inc that is implemented on behalf of other things is the file tokens, which we can move to file.module now.

Closing in favor of #1969540: Convert token.inc to a service.

Agreed!