Find a consistent way to document plugin types

#2086411: find a standard way to document plugin types in api.php files pick one

If you look at https://api.drupal.org/api/drupal/core!lib!Drupal!Core!Entity!Annotation..., compared to https://api.drupal.org/api/drupal/modules!system!system.api.php/function... --

- we've lost the general intro to the concept
- we've lost readability. The list of class properties is much harder to read down that the properties docblock we had for the hook
- we've lost sample code
- that means we've also lost the sample values for each property that are given in the sample code

Furthermore, there are new concepts in the plugin system that developers need to know about but which aren't represented here:

- what's the subdirectory to use for the plugin?
- what's the name of the service to get the plugin manager?

Another point to consider is that there has been talk (on IRC) lately of how useful Module Builder will be for getting developers accustomed to working with D8, and with coding some of the complexity of classes that D8 requires.

I made a rough start on a module builder generator for a plugin: #2086181: plugin generator.

However, in order for that to be useful, there's a bunch of information that needs to be available in a form that can be parsed:

- the service name
- the plugin class
- the plugin subdirectory
- the plugin namespace
- the annotation class
- sample code for the annotation

We'd need all of the above to be able to provide a similar experience when using MB to create plugins as we did for using MB to create hooks.

Compare: asking MB for 'hook_entity_info' got you:

- function already named with the right parameters
- sample function body

We'd want to be able to ask MB for 'entity_type' or 'EntityType' or whatever, and get:

- the folder structure
- the plugin file
- the annotation, with sample values
- the plugin class with the parent class
- sample class methods

To achieve that, I think we need to define sample plugins in the same way api.php files define sample hooks.

Furthermore, we need:

- a standardized name for a plugin type. Eg, for entity types is it EntityType or the string you pass to get the service?
- standardized ways of documenting plugin subfolder, annotation class, and so on, so that that data can be retrieved programmatically

Tagging.

> will make the file invalid PHP

api.php files are never executed, so I don't think that's a problem.

How about we have a sample annotation on the sample class?

so:

/**
 * A plugin for making sandwiches.
 * 
 * [Further explanations...]
 *
 * [Machine-parsable properties...]
 *
 * [something to announce that what follows is a sample annotation]
 *
 * @Sandwich(
 *   id = "sample_id",
 *   subject = @Translation("Sample title"),
 * )
 */
class PluginSandwich {

  [ sample methods here ]

}

I don't imagine there'd be a problem with the discovery system finding that annotation, but it there is, we should be able to tell it to just ignore files ending in api.php.

If it makes it easier, I'm fine with plugin sample classes to each have their own file. Would that fix the problems to do with PHP syntax?

I do note though that declaring multiple namespaces in one file is allowable: http://www.php.net/manual/en/language.namespaces.definitionmultiple.php

If we did have one file per plugin sample, that means they're not in the same place as the api.php file :/
Though api.php files aren't really mean to be opened and read -- they're meant to be read on api.drupal.org.

I don't think I really get what the problem is.

Are you talking about there being a problem with discovering plugin types? They all have @Annotation in them, and those are collected at
https://api.drupal.org/api/drupal/core!modules!system!system.api.php/gro...

Or are you talking about a problem with, once you have discovered an annotation class, how to figure out how to use it? I think most have base classes, and hopefully all of the Annotation classes have an @see to that base class... if they don't, that is a docs bug that we can address.

Please update the issue summary because I don't really get what the problem is after reading it.

I'll try and explain the problem here before updating the summary, to check it makes sense:

On Drupal 7:

A human wanting to implement a hook goes to read the docs for hook_entity_info() at api.d.org. The docs specify:

- the name of the hook
- the function signature
- an explanation of what the hook is for
- a bullet list of parameter, with in-depth explanation. Nested arrays in parameters or return are expanded into their sub-elements as far as they go.
- [missing] there is no standard way of telling the developer that a hook should be placed in a .inc file

A programmatic system like Module builder goes to read the same function in the api.php file.

- the function declaration can be parsed
- the function body can be used as sample code
- [missing] the system can't tell from the docs in which file a hook should be placed, though that can actually be figured out by invoking hook_hook_info() on a Drupal site that has that module. Which is what MB does ;)

On Drupal 8:

Human developer:

- needs to figure out that the Annotations topic is about making plugins. Because the summary line at 'Topics' says 'Annotations for class discovery and metadata description.' (minor point I know)
- the list at https://api.drupal.org/api/drupal/core!modules!system!system.api.php/gro... has loads of Symfony classes which (AFAIK!) are totally irrelevant
- the descriptions are mostly pointless. Eg 'Action': 'Defines an Action annotation object.' Well I could have deduced that from the class name! ;)

Those are minor so far and all easily fixable. But now...

- The class docs for https://api.drupal.org/api/drupal/core!lib!Drupal!Core!Entity!Annotation... as a list of variables that is (AFAICT) in alphabetical rather logical order
- the table is much harder to read than the D7 docs: there is loads of extra cruft such as the classname repeated on each row, the modifier, and the type. The important description is squished up.
- only the first line of the property description is shown.
- for a property that is an array you have to follow the link to that specific variable
- some properties appear twice (is this a bug in API or something else?)
- there is no sample code, so you can't copy and paste that to get started with. This means that there's also no example values for properties.
- there is no indication of how this plugin type's discovery method works. Where does a dev go to find out what subdirectory or namespace EntityType plugins should use?
- there's no indication of what base class should be used for creating a new EntityType plugin.
- there's no indication of how you then *get* your plugin: we need both a @see to docs about using the service container, and each plugin type needs to tell you what its service machine name is.

Automated system such as Module Builder:

- can't get the subdirectory or namespace to use for a new plugin
- can't get the base class to use for a new plugin
- the annotation class variables can be got with get_class_vars(), but those have no sample values. So while I can build an annotation block, all its values are just ''. There's no sample values or sane defaults to get started with.

I think that's about it :)

One more thing though -- MB really needs a notion of a name for a plugin type. Is that the annotation class name, or the service machine name?

RE #10 - I feel your pain. There is quite a bit of topic documentation that hasn't been written, and unfortunately quite a bit of the D8 class documentation is severely lacking. IMO there was not sufficient enforcement of the Documentation Gate but I've hashed through this before and am not going there now... Let's instead of griping figure out what we can do to fix the situation.

So...

a) There is already an issue on the API module about the Annotations topic not including vendor annotations. Also for the "things appearing twice" issue.

b) IMO, a lot of this would be solved if there were @see links (and/or descriptive text) linking each Annotation class with the interface such a plugin needs to implement, and the base class that exists for making this implementation easy. If there are cases where this doesn't exist, those are docs bugs and we should file/fix them. Once you get to the base class or the interface, api.drupal.org will easily tell you which classes implement/extend these, which will give you sample code.

c) Regarding the difficulty of reading the api.d.o page for the Annotation classes, I agree but there is not a lot we can do about it since they're just classes. I can suggest that you open the "view source" link down at the bottom and use that, because that will resolve most of the issues you have with reading the page -- all the information is there without any clicking-through needed.

Sooo... It seems like the action list here might be:
- Fix up the Annotations topic a bit so it is clearer it's about plugins, and explains how to make a plugin... Or maybe we should actually have a separate topic called Plugins that explains this (and link both of them to each other), because Annotations are about more than just plugins? Yes, I think that would be better. The Plugins topic should also explain about plugin managers and how to get a list of plugins of a certain type.
- Make sure each Annotation class is linked to its base class and interface (and vice versa). File individual bugs if they aren't.
- Fix the bugs in the API module that are listing vendor annotations on the Annotations topic and making duplicate entries in class member lists.

Thoughts?

Well, the situation for plugins, as you describe it in #12, is not all that different from the situation for hooks.

In D7, you could discover hooks in the following ways:
- The Hooks topic on api.d.o (all hooks are supposed to have @ingroup hooks in them)
- Looking at (module).api.php files

In D8, you would currently discover plugin types in the following ways:
- The Annotations topic on api.d.o (and yes you are correct that not all of them are plugins, but most of them are -- aside from the Vendor ones that are there due to a bug in how that list is constructed)
- Looking at (module)/lib/.../Annotations for these classes if you are not on api.d.o -- I think you are wrong about there not being a rule for where to put annotation classes -- I think they're all in a standard place, arent' they?

Neither one is centralized.

I think we can improve things somewhat by having a @ingroup plugins (similar to what we have for hooks now) and making a Plugins topic, as I said in #11, and hopefully making all of the specifically plugin annotations be part of this group. Then you could use grep to find them in D8 too.

Thoughts?

> I think we can improve things somewhat by having a @ingroup plugins (similar to what we have for hooks now) and making a Plugins topic, as I said in #11, and hopefully making all of the specifically plugin annotations be part of this group. Then you could use grep to find them in D8 too.

I think that's definitely the right place to start.

We can then think about things like documenting the plugin type subdir, service name and so on.

One thing that confused me -- yesterday on IRC EclipseGC said that you woudn't want to start from the plugin annotation when reading docs, but from the plugin manager class!

Are we going at this the wrong way round? Also, I have the impression (from grepping code, so could be wrong) that not every plugin type has its own plugin manager class.

> > c) Regarding the difficulty of reading the api.d.o page for the Annotation classes, I agree but there is not a lot we can do about it since they're just classes.

Could we make a specially formatted output for Annotation classes?

(Also, sorry, didn't mean to come across as griping. I was just trying to be thorough in explaining where the pain points are.)

I disagree that starting at the Manager class is smart. For instance, if I have a module that wants to provide a Block, why should I have to learn about the Block Manager? All I want to do is make a block plugin. To do that, what I need to know about is the base class and/or interface that my plugin class needs to implement/extend, and the annotation I need to put at the top of my class docs. Right?

So either the interface, the base class, or the annotation class should be the entry point, not the manager, which I really don't need or want to learn about.

Well, if that is the case, then I think @ingroup plugins is the way to go. That can go on the annotation class or the interface, whichever is most appropriate.

And definitely the generic @Plugin is going away. There's another issue for that.

I realized I forgot to address #14 (c) - about having a better display on api.d.o for annotations classes... I can't see that we'd have a better way that what View Source is already giving us, so I don't think I'd want to spend the extra effort doing it.

> I can't see that we'd have a better way that what View Source is already giving us

What I have in mind is producing something like the nested list you get for hook_entity_info():

- take each class variable
- make a new bullet point for it
- use its main description as the text for the bullet point
- use that variable's own bullet points (for the case it's an array) as nested bullet points

Yeah, but I don't see how that would really be all that much more readable than View Source really, and it's kind of a pain to do. It's a question of cost/benefit, and of complexity/maintainability. If you feel strongly about it, file an issue in the API project.

related: #2062573: Add method defaultConfiguration() in ConfigurablePluginInterface

I do feel strongly about it -- #2092757: specialize output for Annotation classes, with pictures :)

> when all they have is the code itself.

I would strongly recommend that anyone coding for Drupal without an internet connection take the time to install API module and run their own localhost version of api.d.org. There are instructions on d.org on how to do this and it's pretty quick -- last time I did it took me under half an hour.

And if you don't have that, you can grep the codebase for topic tags like '@ingroup foo'.

> This will only solve the api.drupal.org issues, and only for annotated plugins. Plugin types using a non-PHP-based discovery method will not be able to be discovered by the API module like this

This is a very good point.

bump - we have the getDefaults() method on the interface we can use now as a way to document.

Regarding a plugins.txt file, the main problem we have with listing things in separate files outside the main code base is maintainability. We already have/had this problem with the api.php files for hook docs... introducing another file that has to be remembered to maintain will not help the situation.

correction: it's the defaultConfiguration() method added to \Drupal\Component\Plugin\ConfigurablePluginInterface, which we need to extend on other sub-ierfaces.

I might even consider this a feature request.

I personally don't agree with the demotion from critical, but this is *at least* major.

"Using Drupal" nowadays, does not only mean downloading and installing it, it also means using it as a platform, i.e. working with the code. That doesn't mean every documentation problem is as problematic as a UX issue, but plugin information is a very entry-level piece of our documentation. Not only module authors will be providing plugins (note: not plugin *types*, but plugins), but writing e.g. a plugin YAML file will be the first dive into code for many Drupal 8 site builders.

Note that while the documentation of annotated classes is not optimal it at least exists. YAML-based plugin types, however, do not have *any* way to document the keys currently. There's no way to find out which keys to use but for looking at examples or actually reading the implementing code. That's not an excepatable situation.

OK, let's step back and get the big picture here, rather than quibbling over the issue priority. What are the goals?

a) Developers should be able to discover what plugin types Drupal Core or contrib modules have available for them to implement.

b) Once they find a plugin type, they should be able to find the information they would need in order to implement a plugin of that type. Which might include:
- The interface they need to implement in their plugin class.
- The base class(es) provided as a helper in implementing the interface.
- The annotation structure that is needed.
- Some examples of plugins of this type.

Right?

So it seems to me that one way to handle this, without defining any new ways to do documentation, and without needing any new code on api.drupal.org, would be:

a) Define a topic/group called "Plugins" (@defgroup plugins). This would go into an api.php file in system.module.

b) Each individual plugin type defines its own @defgroup plugin-block (for example). This would go into the .api.php file for the defining module, or if it's outside of a module, a .api.php file in system.module.

c) Within the specific plugin type defgroup, put an "@ingroup plugins" so that the topic itself shows up as a subgroup (I think this is possible; if not, then we would need to go to the plugins defgroup and do an @see there, but either way we would want the individual plugin topics to be listed on the main Plugins topic page).

d) Then each individual plugin type defgroup can have a description of their plugin (what it is for and what the main classes, interfaces, etc. are), and their main plugin interface, base class, and annotation can all have an @ingroup so they will be listed on the topic page. If they want to point to an example plugin class, they can use an @see in their defgroup.

I think this would take care of both of the goals. Plus, it's simple and complies with our current standards. People know to look for .api.php files to see our API, and plus it would be displayed in a discoverable way on api.drupal.org.

Thoughts?

Sounds like a very good start.

Ideally I'd like something that Module Builder can parse, and so offer developers automated scaffold code for different plugins, but let's start off with what humans need from the docs :)

RE #31 and Module Builder - that seems nearly impossible to achieve, given that to create a plugin module you need to make an annotated class that has a particular namespace and a particular annotation, and the information about all of that is all over the place.

Anyway, if we adopt the proposal in #30, we could have a patch for (a) now and maybe one representative plugin type (such as Block plugins). Then fill in the others as we go? This could become a meta issue?

Well presumably if we document plugin types, we can bring all the information together?

Several people have requested that MB be extended to generate code for plugins, because it would significantly ease the learning curve for D8.

> Then fill in the others as we go?

Given my experience on some long-running docs issues, this is definitely the way to go :)

> given that to create a plugin module you need to make an annotated class that has a particular namespace and a particular annotation, and the information about all of that is all over the place.

But presumably if we document plugin types, then we should be documenting this, and in one place.

So what we could do is present each plugin type like this:

/**
 * @defgroup plugin-block
 *
 * Blocks are plugins which yada yada.
 *
 * - Annotation class: Block
 * - Namespace: Drupal/Block/Foo/Bar
 * - Otherthing: FooBarWhatsit
 *
 * Paragraph of more detailed info begins here...
 ....

That would be enough for automated systems like Module Builder to parse. It would also provide a nice overview for humans.

I like it! A few nitpicks:
- Blocks are plugins *that* (not which)
- Our list-making standards require the line before the list end in :
- Forgot the @ingroup plugins
- Probably the base class is present often enough to make it part of the standard, and I think the interface as well?
- If you are implementing a plugin, you need to put it in your module's namespace, but within that at a specific place, and that isn't clear from your Namespace line.
- All classes mentioned need to be given namespaces.
- Doesn't follow defgroup standards

So maybe:

/**
 * @defgroup plugin-foobar FooBar Plugin Type
 * @{
 * FooBar plugins (description of what they do).
 *
 * Plugin summary:
 * - Annotation class: \Drupal\foobar\Annotation\FooBar
 * - Namespace suffix: Plugin\FooBar
 * - Interface: \Drupal\foobar\Plugin\FooBarPluginInterface
 * - Base class: \Drupal\foobar\Plugin\FooBarPluginBase
 *
 * More detailed info here, if appropriate.
 *
 * @ingroup plugins
 * @}
 */

Is the fact that annotation classes are barely documented related to this?

I'm trying to find out what the different properties are for ConfigEntityType, and the obvious place to look, https://api.drupal.org/api/drupal/core!lib!Drupal!Core!Entity!Annotation... has nothing.

The Entity annotation class is a special case being covered in a separate issue.
#2189497: Object-based Annotation classes need links to interfaces
Guess I should go commit that patch. :)

Thank you for creating this issue to improve Drupal.

We are working to decide if this task is still relevant to a currently supported version of Drupal. There hasn't been any discussion here for over 8 years which suggests that this has either been implemented or is no longer relevant. Your thoughts on this will allow a decision to be made.

Since we need more information to move forward with this issue, the status is now Postponed (maintainer needs more info). If we don't receive additional information to help with the issue, it may be closed after three months.

Thanks!

This is absolutely still relevant. There is no proper documentation on plugin types the way that there is for hooks.

Nobody has worked on it because it's a big undertaking.