Problem/Motivation

The Examples for Developers project helps drupal developers to quickly see and understands how drupal core APIs work; maintaining several submodules that show how to interact with each API subset.

This (meta)issue is about moving every submodule to drupal core.

Why adding it to core?

  • To ship an functional example of how to use every API in core for developers(also improve visibility of those helpful examples).
  • To help maintaining a the examples, which are useful, but with not enough maintainers.

Other considerations

  • Examples project submodules should be treated as documentation inside core.
  • There was some concern about miantaining example submodules tests working while making major changes in core, but that was solved by this propossal: "Just remove examples or disable tests at the beginning of the cycle, and then make sure they get done at the end. The reality is they could be added in one at a time just like we're talking about doing now for D8.".
  • Tests(and test helper modules) and examples modules are different, they have different targets. There is no code duplication — or much rather, there should not be code duplication, since all test modules should contain code that you should NOT consume as examples in any way.

Proposed resolution

In order to get every examples project submodule into core, we can follow an iterative process of 'staging' submodules to the Examples project, and then eventually creating a ticket for inclusion in core.

The process might work like this:

  • Improve the example module(based on some reviews in this issue, it seems like it should be needed). Examples should demonstrate best practices in formatting and documentation.
    • Port to D8
    • Review with coder
    • Make sure the documentation is clear, well-written, and follows our documentation standards on http://drupal.org/node/1354
  • Add it to 8.x-1.x branch on examples project
  • Open an issue for inclusion in core("Add example xyz to core")

Remaining tasks

Some work has begun here: #1880976: [meta] Port examples to D8

  • Propose a list of the new API subsets which in D8, so we can have an example for each of them.
  • Port main examples and add them to core(list based on webchick analysis): block_example, dbtng_example, entity_example, menu_example, nodeapi_example, page_example, theming_example.
  • Port the rest of examples and add them to core.
  • Decide where example projects will live inside core: see alternatives at comment 81

User interface changes

None.

API changes

Examples project uses @defgroup/@ingroup to define an 'examples' group and a child group per module. We will need to discover whether this causes a problem for api.drupal.org's group organization.

#1880976: [meta] Port examples to D8

Original report by rfay

Over in #1516840: Get more active maintainers and allow a greater diversity of examples I proposed breaking Examples up into separate modules so that more maintainers might take full ownership and maintain the various modules. But so far, that hasn't gone very well. I'd say the majority is against.

In #1516840-18: Get more active maintainers and allow a greater diversity of examples, though, @moshe suggested that it's worth considering moving Examples into core. This would have a number of advantages both for the examples and for core itself:

  1. Core changes that broke an example would be shown in the automated test process. This could result in a much earlier understanding of a core shift.
  2. Likewise, API changes would have to be reflected immediately in Examples changes.
  3. Example modules could be used by core tests as "Mock modules", so fewer of those mock modules would be required.
  4. A culture of creating and maintaining examples along with new code would be great for the community, resulting in major learning gains in new releases. In reality, every new API *should* have maintained example code, by policy, IMO.

How could we do this?

  • Create a checkbox on the modules page that would show or hide example modules (with them hidden by default).
  • Put the example modules in their own "examples" directory, and perhaps not in the regular core/modules. (Perhaps we wouldn't have to do this.)

Comments

On the one hand I think this is a fabulous idea. Examples module is completely amazing, particularly for new developers, but it's all but impossible to find unless you're already "in" with the community. Including it in core would greatly improve its find-ability/accessibility, and I really like the idea of making "provide an /examples sub-folder in your module" a best practice we employ in core and encourage community-wide (though it's unclear to me how this would work for hooks exposed by non-modules such as the AJAX or mail system; I guess we'd just shove 'em in modules/system/examples).

On the other hand, I'm concerned about the extra maintenance burden this places on core developers. In addition to fixing test failures in core modules that end users use, they have to fix them in modules that only a tiny fraction of our community (relatively speaking) uses. This same sort of maintenance burden is what has been behind the thrust to jettison modules "no one uses" (we don't have any actual data to back this up, but it's what core developers feel) from Drupal 8, such as Blog and Trigger. This would add another 30,000 lines of code or so to maintain, that would need to be updated for every API change. That itself wouldn't be so bad, but we have another 50,000 lines of code in automated tests that re-tread a lot of the same ground, in some cases.

On the OTHER other hand, that maintenance burden still exists either way; the difference is it's currently shared among a much smaller team with much fewer eyeballs on their issues and patches. However, that team would in theory only have to port the code once, towards the end of the release cycle, not on an ongoing basis as 8.x develops.

I think overall I land in favour of this, but would like to thinker on it some more.

Won't fix for me, since Drupal core moves too slowly.

The Examples project in contrib worked out, works out, and is generally awesome, because it is able to move fast, without barriers.

Remove that advantage and you kill its essence.

Couldn't we just say that the /examples directory in the core distribution is much more flexible with what we can do with it? I mean, most people won't be running examples on production, so if we make API changes in the examples, then it's not that big of a deal. If examples were included in core, I'd look at it more as documentation.

I'd prefer to see Examples stay in contrib, with perhaps a link on d.o/project/drupal

@Everett, any particular reason?

@cweagans it's extra code to update and maintain, Core maintainers should have a smaller, not a larger code base. At the same time, it is very useful documentation of API implementation, and serves as a good resource for those looking to work with Core APIs, so it should be easily discoverable.

The number of (kilo|mega)bytes that is taken up by Drupal core has never been the basis for features that are included or excluded. The goal here is to make examples easy to discover and use. If it's included with core, then that problem is solved. It's a great resource, only covers core material, and is useful to lots of people.

FWIW, I would *love* to see major Drupal core releases ship with a full set of examples. It's almost like curated, versioned documentation for developers.

I think I'm in favor of having them in the core distro... Would they be classified as "documentation" for purposes of who can commit changes and for purposes of the Documentation Gate?

IMO, if examples were included in the core distro, treating them as anything other than documentation would be a huge lapse of judgement, so my vote would be yes.

IMO they should be considered documentation (but of course be fully functional and have complete tests, and be executable if enabled).

I'd be great with having them be committable by the docs committer.

I'm on board with the concept of having them in core if commitable by the docs maintainer.

I think it's a wonderful idea as it is actually already in the api docs.
The downside would be that once a core change is pushed, not only tests & docs need to be updated but examples module would need the "check" too.

@pcambra, actually, the Examples tests would often catch problems with a core change, and force the fix of the example before a patch could be committed. It basically would increase the code coverage of our tests quite a lot, and also increase the "real-life" use cases that get tested.

Maybe we could merge the Examples project with our current tests.
As #13 states, in many cases the Examples project has more in-depth tests of core features.
On the other hand, putting the advanced and heavily-documented hook implementations etc. in our test modules wouldn't hurt anyone. The increased maintenance effort only exists when actually implementing new features, not when updating existing tests. And in that case, advanced and heavily documented example are probably even helpful in the review process.
We could then just document somewhere on Drupal.org: "Want to learn how X works? Look into X_test.module!" (We could also, of course, rename our test modules to X_example.module)

RE #14 - I don't think this idea is quite right:
a) I don't think that the Examples project should be imported as tests -- they'd be buried deep in the file structure that way. If we put them into core, I think they should be in their own top-level directory.
b) The test modules that we have now are often set up to mock APIs or features, not to serve as examples, so I don't think we want to point people towards looking at test modules, as a general rule, as a good way to learn Drupal.

Well, what if we put the heavily documented and heavily tested examples in core/examples and then simply drop the test modules, which often are a subset of the example modules. We would need the ability to run the tests of the example modules in the core development process, but other than that, sounds like win-win, no?

I think, like any core Drupal module, the tests should be left in there with the example modules, in subdirectories. We're talking 8.x anyway (where the tests are being moved to core/modules/(whatever)/lib/Drupal/(whatever)/Tests directories within the core/modules/(whatever) directories). Then we'd just need to have Simpletest notice/run the tests from core/examples as well as core/modules... which would mean making sure those are loaded via PSR-0 I guess... Anyway, we don't have a central place for tests now.

No, the test module code you're referring to is mostly poor test module code. Proper test module code contains totally weird stuff, say, like every possible incarnation, variation, and permutation of #default_value, possibly in combination with other properties, and test code also mocks plenty of things to resemble different scenarios. Definitely not something you want to show to anyone as an example, because factually, you should not write a module that contains such code.

It's also impossible to document proper test module code, because all the code is doing is to assert the proper behavior of API code in case of X, Y, Z, reversed, and all of them combined. There is no use-case behind that and it wouldn't make sense to imply any, as that would make authoring of tests only harder.

Also, #17: Only test classes are moved into the PSR-0 structure. Test support modules will remain in $module/tests/$testmodule/.

To expand a bit on #18, a good example shows how to do the normal stuff, but a good test exercises every possible path through the code including not only the normal uses but all the edge cases and potential problems. So the goal of examples is practically orthogonal to the goal of tests.

Re #18/#19: I guess that's true, at least in theory. Maybe we could have the best of both worlds, if we have example modules with the most basic use-cases explained and documented in detail and then we have test modules which contain only the edge-case usages. Then from the test files we enable both the example modules and the test modules, and then assert on them as if there were no difference.

As an example, I'm thinking of node.modules's hook_node_info(). There would be a node_example.module and a node_test.module (where these would be located in the file system can be decided separately). node_example.module contains only a basic implementation e.g. a 'blog' node type. node_test.module contains further node types in order to test all of the different options and possibilities. If there were no node_example.module, node_test.module would probably still contain at least one basic node type and assert some stuff for that one. So the added maintenance effort only applies to the in-code documentation of the 'blog' node type. If we can avoid duplicated code, I think we should.

The benefit is that the added maintenance effort because of the examples is minimal and the maintaining of the examples code (though not the in-code documentation) becomes part of the core development process. (Note that this is not a given, even if the Examples are moved into the core tree.)

OK, let's step back a moment... Maybe I can summarize the discussion (as I see it):

a) We have this great code base of Example modules, currently in the Examples for Developers project, which include tests (which are examples of how to write tests, and which also, when run, ensure that the example modules actually work). They are readily available, and a great resource, but not necessarily known to new developers.

b) This issue proposes adding the Example modules (with their tests) to a new "examples" directory, and to have them treated more or less as "documentation" (meaning that they can be updated after the first full release of the branch without it being considered an API change, and that changes to them can be committed by the documentation maintainer). The idea is that distributing them with Drupal Core will raise their visibility, and that maintaining them with Drupal Core will ensure that they are up-to-date.

c) A concern has been raised that as core devs change APIs in the in-development branch of Drupal Core, this would create an added burden on them to update the Examples as they go. (Presumably also the Examples tests would be run along with the core tests, to ensure that the examples are updated with each patch that would affect them?)

d) There has also been some discussion of whether there could be some merging of the Examples with the existing tests or the modules that exist to support the tests, but sun, jhodgdon, and others do not think this will work well (the purposes of tests and their supporting modules is not the same as the purpose of examples, and their different needs will not really support them being merged).

So... I think the main thing we need to decide is whether the added burden of (c) or the added benefit of (b) is stronger. And if (c) is stronger, perhaps we can think of a way to increase the visibility of the Examples project (such as adding a link to a README file, or even adding an examples directory containing just a README file that links to the Examples project?).

Thanks for the summary, @jhodgdon. I think one more thing was a consensus item;

e) The examples would be considered documentation and thus committable by documentation committers.

Note that the added burden of c) (maintaining the examples) either has to fall somewhere, or Examples will fall by the wayside. I bore much of the burden of maintenance in D7 (discovering API changes or bugs often by accident when the Examples tests failed), and I'm not willing to do that going forward. So this issue was a result of trying to solve the problem of maintenance.

Note that if we view Examples as documentation, and assume that they only break with API changes, then it's perfectly reasonable to do as we always have done and fix the documentation as core changes right along with the core API change.

(e) in #22 is part of (b) in #21. :)

And yes, that is the basic question: Are the Examples worth enough to shift the burden from them being a contributed module (which needs a new maintainer for Drupal 8, or risks becoming obsolete) to being maintained as part of the core documentation?

I think they are... but we need to get buy-in from the core devs/maintainers that they are willing to maintain them as the Drupal Core API evolves.

Yes, I think the core dev team is willing to maintain these modules and tests.

FYI, I'm a core dev and maintainer of 5 subsystems in MAINTAINERS.txt.

There is one problem that we'll face to get to a patch. Examples already has a backlog of 8.x problems; Contextual Example doesn't work; theming and render examples don't work (probably due to misunderstandings of how they were supposed to work). Configuration needs to change...

I think it's fairly likely that we could get an online sprint together and make it happen pretty fast.

Probably the best technique would be to move in the modules one patch at a time; this would then become a meta-issue.

Thanks moshe for chiming in on your willingness. As the maintainer of documentation, I will also chime in and say I'm willing to spend time doing final review and commit for these patches.

I think we should just wait for catch/Dries to OK the idea of this going into Drupal 8.x, and then proceed as suggested in #25 (one examples module per issue, with this becoming a meta-issue). I'm also not clear on what we would need to do in order for the Examples tests to run along with the rest of the core tests?

What about also moving them to Core for Drupal 7.x? They are already working there (right?) and there should not be any/many API changes that would require them to change...

This is really a question more for Larry, Greg, sun, chx, and others actively making tremendously huge changes to the D8 code base (as opposed to subsystem maintainers), because it requires ongoing burden to getting those patches in.

Another proposal we might want to look at is adding this code post-code freeze. Then it would be a one-time major upgrade, but not an ongoing one.

I'm concerned whether this will raise the barrier of contributing to Drupal core even more.

While I'm fairly sure that I could handle the extra workload myself (although not really ideal for epic/large changes during code thaw), I'm mainly thinking of other developers (especially coming from contrib), who keep on telling us that the barrier for getting even small changes into core is ridiculously high already today, and so they rather try to avoid to get involved in core altogether, despite having good and reasonable change proposals.

To me it looks like the actual, underlying goal behind this proposal is to 1) raise visibility for the Examples project, 2) get more contributors (and also maintainers) involved, 3) keep the examples up to date and working (which may or may not serve as additional regression testing).

All of that sounds fine to me. However, I wonder whether the problem doesn't actually boil down to packaging instead. Which immediately reminds of #1260214: Choose your Drupal (i.e., having a Drupal Starter/Standard Edition ship with Examples module, which everyone downloads instead of the plain Drupal core) — but since there's close to zero movement and yet still no common agreement on that, the d.o packaging process for Drupal core would have to be special-cased instead.

Well, I don't think Choose Your Drupal solves this either. Randy's whole point seems to be that because this critical code is in a separate repository with separate maintainers and without core tesbot coverage, core developers change things willy-nilly that break things in it all the time, and then we have 1-2 overly swamped maintainers keeping up with the momentum of 1000+ contributors. We have no solution to that problem, either technically or socially, currently.

@sun, I would say that the goal is completely *not* visibility for Examples. It's got plenty of visibility, but no sustainable development model. And it's a fundamental part of the documentation of Drupal core. So IMO point 1) is not correct at all. The following points are true (more maintainers, better maintenance) are valid.

But point 4) should be: Examples are a way to validate and ensure the quality of core. Maintain them with core.

While I'm fairly sure that I could handle the extra workload myself (although not really ideal for epic/large changes during code thaw), I'm mainly thinking of other developers (especially coming from contrib), who keep on telling us that the barrier for getting even small changes into core is ridiculously high already today, and so they rather try to avoid to get involved in core altogether, despite having good and reasonable change proposals.

I'm pretty much in agreement with this. I'm one who doesn't often get a lot of time to work on Drupal core issues, but when I do, it's already intimidating enough to (a) wrap my head around the issue, (b) investigate the problem and the proposed solution, (c) write up a patch, (d) make sure tests pass, and if not, fix code and/or tests, and (e) work on a change notice, or at least summarize what's changed so someone else can do a change notice.

Adding in another step (f), making sure the examples module corresponding to the patch's area of concern is up to date, pushes the limit of what I would put up with.

The examples module is awesome, and I have been helped by it tremendously (enough to submit a patch or two to improve it when I see fit), but I don't see the value of having it all in core in relation to the detrimental effects it would have to long-tail contributors like me.

Version:8.x-dev» 7.15
Assigned:Unassigned» orland
Status:Active» Needs work

Version:7.15» 8.x-dev
Assigned:orland» Unassigned
Status:Needs work» Needs review

This is an 8.x initiative. Not sure why you edited it @orland

This needs to be taken into account for moving examples into core #1796606: D8 Port: field_example

I am starting to contribute to core, so I am one of the people @sun referred to in #28.

I agree with pretty much everything that @tstoeckler said in #14 and later comments. My vote is +1 for moving Examples into core. I agree with the idea that Examples are part of documentation.

I came to these conclusions before reading this issue, while working on #1847198: Update the structure returned by hook_toolbar(). As I tweak the toolbar hooks, I know that I will have to update the documentation in toolbar.api.php and update the tests. It makes sense to me to develop the API, the examples, the tests, and the documentation in parallel.

I already have an example module in my sandbox. My current plan is to submit it to the Examples project, then strip out most of the comments and use it as the basis of the test module. After reading #15, #18, etc. above, I guess the test module should include many more variations.

The problem with my current plan is the duplication between the module I plan to contribute to Examples and the test module I plan to contribute to core. I hate code duplication. If Examples are moved into core, then I will not have this problem.

The question of where to put the examples is clearly secondary, but here are two suggestions, using the Toolbar module as an example:

toolbar/tests/modules/toolbar_example/
toolbar/tests/modules/toolbar_test/

or
toolbar/example/modules/toolbar_example/
toolbar/tests/modules/toolbar_test/

In either case, the idea is that the tests can enable the example and/or the test modules. The example module should contain realistic use cases with copious comments, and the test module should contain more variations for "stress tests."

Version:8.x-dev» 9.x-dev

I definitely encourage you to port Examples module to D8, but unfortunately the activity and work done in this issue doesn't qualify it for feature freeze extension. So this is going to be D9 at this point. :\

I personally see examples module while useful not something that belongs in core. It would increase the size without adding any real functionality.

Examples project bugs counted as part of thresholds in some fashion.

Examples bug roulette then becomes a training/live fire usage of the changed code that would be a useful vector for participation with very well-defined (and scoped) workload.

Category:feature» task

@webchick - I don't think of Examples as features. They are developer docs, IMO. I think they are eligible for D8 but I won't flip the Version unless others agree.

Version:9.x-dev» 8.x-dev

I totally agree. In principal I see no difference in committing all examples to core and fixing a typo in the hook_init() docs. Obviously the scope is different, and it is questionable whether we will actually achieve this for D8, but I don't this should actually be postponed.
Also, I don't think the 9.x version is actually intended, see #1856142-7: Typo in a comment, so moving back to 8.x and tagging as Drupal 9, even though I personally think that tag should be omitted.

Edit: Instead of referencing the issue, reference my comment over there.

Version:8.x-dev» 9.x-dev

I am still scared by this idea, still based on the same considerations as in #28.

Meanwhile, I'd guess that roughly 50% of my core development time is spent on tests, coding style/standards, phpDoc/documentation, and change notice tasks for all patches. That's way too much from my perspective (especially when considering that half of the code is usually completely refactored and rewritten by someone else in a separate issue later on, so all that remains from the hard work are the tests in the end, if at all).

Adding yet another barrier for contributing to core therefore does not seem to be a wise idea to me, because all of these add-on tasks are not fun. People want to work on and help to improve Drupal if it is fun. If it is not, then they won't, and as an open-source community/product we have a critical systematic/structural problem. From my perspective, we are very close to cross that line. (I could go on to explain this in detail, but alas, wrong issue.)

Randy's whole point seems to be that because this critical code is in a separate repository with separate maintainers and without core tesbot coverage, core developers change things willy-nilly that break things in it all the time, and then we have 1-2 overly swamped maintainers keeping up with the momentum of 1000+ contributors.

I think it's mainly the not explicitly stated but underlying goal of adding test coverage for example modules and requiring core contributors to update all possibly existing and related example code along with every single improvement they are contributing that scares me — i.e., not only adding examples to core, but also making them a hard requirement and formal gate/barrier for all changes.

If we were able to remove that from the equation and we'd say that there are examples in core, but they are detached and we ensure they are detached from the rest of core (development), then I wouldn't see an issue with this. (You'd still get more visibility, you'd still get 1000+ contributors to potentially look after them [potentially nice novice tasks], but we don't disrupt core development and we don't make contributing to Drupal even harder as it is already.)

What do you think?

(Also note that I'd agree with @moshe on D8 material if we'd remove the gate/test/barrier aspect from this proposal, since that would essentially mean that examples are just a library that is not bound to core development and thus also not bound to any release cycle phases/dates.)

To some extent, the Examples modules duplicate code in the test modules. If we remove the duplicated code from the test modules and rewrite the tests to use the Example modules, then it adds to the burden of completing this issue, but it reduces the long-term burden that worries @sun in #41. When the API changes, the modules that use it have to be updated, whether they are considered test or example modules.

If we adopt a file structure something like the ones I suggested in #35, and keep

packge = Example

in the .info files (or is it Examples?) then it should not be hard to find some way to make them easily discoverable.

If we remove the code duplication, then we do not just shift the burdern from the Examples maintainers to others: we reduce the overall burden.

As mentioned in #18 and before/after, test helper modules are not example modules. To summarize once more:

There is no code duplication — or much rather, there should not be code duplication, since all test modules should contain code that you should NOT consume as examples in any way.

If that doesn't help, then let me put it around: If the code in a test module looks like code that could seemingly work as example code, then it is almost guaranteed that the corresponding test lacks test coverage, as it most likely does not cover all possible cases. Proper test helper code looks horrible and should perform very nasty things, which no one should put into any real module. ;)

Consequently, that means the exact opposite: Test helper modules are the anti-examples. If they look too pretty and don't do unholy things, then that's a relatively good indicator for missing test coverage, and it might be worth to create an issue to verify the test coverage and ensure to make that particular test helper module ugly.

@sun:

Thanks for the clarification. I thought that test modules should include realistic use cases as well as unrealistic ones. I gather from your response that they should include only the latter.

I think they can include both, but the real value in test modules is doing things like running through every possible permutation of setting and checking to make sure the results are as expected. Versus the real value in example modules is something with a concise and focused use case, with extremely detailed documentation describing how it works. So from that respect they're often at odds with each other in terms of their ultimate goals.

Status:Needs review» Active

OK... let's step back here a moment -- we seem to be going around in circles. Here are a few thoughts:

a) As a practical matter, getting the Examples project ready for inclusion in Core would be a large project. It needs a lot of cleanup to come close to conforming to our code/documentation standards, unfortunately, and although it has a huge amount of very useful documentation, it could really use some copy editing. Plus it hasn't yet been ported to 8.x. So... the first step to making this happen would be to get Examples ported to 8.x and cleaned up. I haven't seen any work on that. How about if the people who are interested in making this happen start an issue in the Examples project to port to 8.x (or find the existing issue?) and get to work, and open a second one to make it adhere to our coding/docs standards and get to work on that too?

b) Once that happens, we can return here and see about actually getting it added to Core. There are a lot of good reasons to want to do that, and a lot of good reasons that we wouldn't want to do that. If I can summarize:

1. On the plus side: the Examples project suffers from lack of maintainership, and having it in core would ensure that this set of very useful examples gets maintained.

2. On the plus side: not everyone who could benefit from Examples knows it exists, and its visibility would increase by it being in core. And it is very beneficial for new Drupal developers.

3. On the minus side: during the early stages of new-branch development, having to update Examples while developing new APIs and features would be **very** painful. The people who would suffer are obviously not the target audience for Examples, so it's hard to convince them that they'd want to incur the pain of maintaining the Examples project -- it's one more hurdle to people wanting to contribute to core.

4. IMO, the argument being made above that having Examples in Core would be good because it would add more tests etc. is not really valid -- I don't think the tests in Examples are adding much if anything to our test coverage (if there are cases where our test coverage is lacking, we should add to our core test coverage). The real value is not the test coverage, it is the developer-oriented examples.

c) I would like to propose a couple of ideas:

1. We could make sure Examples is mentioned in the README file, for higher visibility, if it isn't already (if this is a good idea, let's file a separate issue).

2. We could consider making a separate DEVELOPERS-README.txt file that would:
- Point people to api.drupal.org.
- Point people to the Examples project.
- Point people to http://drupal.org/developing/api/
(with some notes about what these are good for) (if this is a good idea, let's file a separate issue).

3. Maybe a viable workflow, going forward, could be that we keep Examples as a separate project, but when it is deemed "ready" it is added to Core (after or around the time of the first major release on a branch). So for instance if the 7.x branch gets cleaned up and meets our documentation/code standards, it could be added to 7.x at that point, and when the 8.x branch of Examples is ready, we can add it (maybe it would come in for 8.3 or something). I am not sure if this is a good idea or not, but I thought I would put it out as an idea of how this issue here could possibly be resolved.

Thoughts?

Version:9.x-dev» 8.x-dev
Assigned:Unassigned» Dries
Issue tags:+Needs committer feedback

I think it is problematic to ask folks to port Examples to D8 and then "we'll see". This could happen so much quicker if a core maintainer could say 'Yeah, I want this, or No Thanks.'. I'm tagging this and assigning to Dries. He can decline at this time to decide if he wants.

Also, lets remove the gate aspect of this. These are just docs (for now).

I'm open to adding this to core. I think it is actually healthy to make examples when we make API changes or do API design. It should help reviewers tremendously.

Let's update it to Drupal 8, and once it meets our quality standards we could commit it. I certainly don't want to commit it before.

Assigned:Dries» Unassigned

Given this pretty significant downside...

3. On the minus side: during the early stages of new-branch development, having to update Examples while developing new APIs and features would be **very** painful. The people who would suffer are obviously not the target audience for Examples, so it's hard to convince them that they'd want to incur the pain of maintaining the Examples project -- it's one more hurdle to people wanting to contribute to core.

...we may want to consider evaluating the examples in Examples module and pulling only the top N deemed most critical to developer understanding (and developing some kind of criteria for this going forward), rather than all of them (examples vs. "examples extras" in contrib). OTOH, having the set of examples that ship with core be as comprehensive as possible is nice because of all of the things Randy mentioned in the OP.

But agreed that porting them to D8 and improving the quality of them in the existing contributed Examples project is step 1.

Status:Active» Postponed

So...

Btw, in case it's not clear from my previous comments, I'm actually happy to see Dries having come down on the side of including Examples in core, because of all the things Randy said in the OP (before I was a core developer, I was a documentation team member working on developer docs :D). I postponed this issue because "fix it in contrib, then propose a core patch" is congruent with the approach that Email, Link, Profile2, Entity Reference, etc. have taken in D8 core. But if that's not the right thing to do here for some reason, then feel free to revert that status change.

I do think it's important though to at least consider, and hopefully try and balance, developer onboarding experience with core contributor experience, so that we don't make Drupal 9 even harder and more onerous to contribute major API changes to. That's why I suggested taking a look at if there's a way to move only some of these examples rather than all of them, but I'm also fine with being told "no, that's a stupid idea." ;)

@webchick, choosing the most important examples and porting them is a great idea.

Actually, I think it would be best to allow the various example modules to come in one at a time. Otherwise it's an insurmountable hurdle. I would assume that would fit with the spirit of this.

Unfortunately, while everybody thinks it's a good idea to port Examples... nobody has yet stepped up to it. It's not really time yet, but I haven't seen any passion to actually do the work at this point.

rfay just tipped me off to this issue...

Here's a list of the ways Examples helps people learn Drupal:

1) Core devs can have a framework implementation to think through their API. This theoretically improves DX.

2) Anyone learning Drupal APIs can look at the examples and figure them out.

3) Examples (hopefully) demonstrate best practices in formatting and documentation.

4) ..and testing. :-)

5) Someone managing a contrib set of examples can manage them without having to take time or energy away from core.

6) Novices can find bugs in examples, learn to find an issue for it or file a new one, learn to make a patch, learn to submit a patch, learn to see how their automated test is proceeding, learn how to have their patch reviewed, and then eventually feel good about contributing to Drupal, all without filing core issues. It is the Drupal ladder without working on core.

Obviously I think #6 is important.

YIPPPEEEEE!!!! I think this is an excellent solution.

So, yes -- let's get examples ported and cleaned up one-by-one. There is no reason at all that we need to have a monster patch that does them all, as far as I can tell. If someone wants to champion porting and cleaning up their favorite example model, that would be great!

So maybe we can discuss here a priority list for which Examples modules we should port first? And then turn this into a META issue, with separate sub-issues for each example we want to add?

When we add the first one, I also suggest we create a README file that explains what the examples are for, how to use them to best effect, and maybe where to find additional examples.

let us make some timelines and borderlines and port examples to D8.

@rfay #53:

I marked all the D8 issues as postponed, because we haven't yet hit feature freeze. Obviously that can change.

The most substantive project thus far has been #1630762: Configuration example for D7, so that the upgrade path to D8 may be demonstrated. and #1615520: Example module for the new Drupal 8 configuration system: new module needed, both by alberto56, which combine to form a testable upgrade path from variables to the new config system. Something everyone will need, so a good place to start.

Whether Examples is in core or not, anyone can offer a patch. http://drupal.org/project/issues/examples

I also think one at a time would make it easier.

I started porting cron example: #1867662: Port cron_example to D8

Not sure if I should add issues to examples project or core directly.

How about this as a process:

1. Get the patch into the Examples D8 queue
2. Get it committed with tests
3. Take that if it seems ready and propose it as a patch to Drupal Core (with a new issue)

That way the work gets integrated immediately where it can be useful. And then it also has a path into core.

I think we should discuss how specifically the example modules will get added to core. I personally would find it weird to have ajax_example, form_example, menu_example etc. just lumped into core/modules with everything else, so I would vote for a core/modules/example folder where all example modules go.

#59 sounds good. As a note, all of the examples that I have looked at in the D7 repo (which is only about 5 of them) have serious violations of our documentation standards in them, and need a lot of writing/grammar cleanup in the documentation that is there too. So I would add "Review with coder" and "Make sure the documentation is clear, well-written, and follows our documentation standards on http://drupal.org/node/1354" to the process list.

RE #60, yes, we definitely need the directory to be separate from core/modules. I personally think it should not be inside core/modules at all if possible, because it will not be easily visible/discoverable. I also think this has been discussed in prior comments.

I think we need an issue summary here to summarize the plan. And I also think that we either need to make this into a META issue, or start a new issue to be the META issue (maybe that would be preferable, since this issue is already at 60+ comments). So we could leave this as the issue that discussed when/how/whether to add Examples to Core, and have a new Meta issue that is mainly just a place for collecting the separate issues about "Add example xyz to core"?

Anyway, we do need to resolve the idea of where to put them, which is probably appropriate to discuss here. Thoughts?

Fixing tag

I propose core/examples/modules as the location for modules. I could see themes and other examples appearing there.

I like the idea in #63. But what would someone need to do in order to enable one of the example modules/etc. in that case? Would we need to patch core to look there for modules/themes/profiles/libraries/classes/etc., or would the developer need to copy the example module somewhere else?

Yeah, I think we mention in README.txt that folks should copy to /modules in order to enable an example module.

OK. We need to say "the module directory and all of its sub-directories" then, because increasingly there will also be classes embedded (and the tests and other classes will be buried in lib/x/y/z/...).

#59 is super-great. There would need to be clear documentation that this is the process, explaining to users why the examples module they're looking at might not be there, and in core instead, after the next push.

#65 gets the thumbs-up as well. It should require a file move before you can enable the examples.

Also: #61 @jhodgdon... "As a note, all of the examples that I have looked at in the D7 repo (which is only about 5 of them) have serious violations of our documentation standards in them, and need a lot of writing/grammar cleanup in the documentation that is there too."

Bring on the patches. :-) Or at least the issues as you find them. Tag them as novice and/or documentation and people can learn how to follow the documentation guidelines. We always suggest using Coder, and you might be interested in talking to amitgoyal: #1795594: Patch to fix 80+ coding standard issues in XMLRPC Example module

Also I should mention this: #1861598: Examples for Drupal 8 needs relatively stable feature set

The proposal is that Examples' 8.x branch won't see much updating until the feature freeze. Differing viewpoints very welcome, on that issue thread.

RE #61/67 - I'm sorry that I didn't file issues for the problems I found -- they were pervasive, and I unfortunately don't have time right now to go back and find them... I assumed the Examples project wasn't really trying to follow the docs standards, due to the pervasiveness of the lack of following standards, which is why I didn't file issues before. But this will definitely need to be addressed before the examples can be added to Core.

RE #68 - Regarding waiting until the 8.x core APIs are relatively stable, that is not a bad idea.

We still need an issue summary with a proposed plan incorporating all of this!

I thought about the location of example modules some more, and have come up with a new proposal: /modules/example/

/modules is for contributed modules and themes that people download, so this is a folder we *want* them to look in anyway, in great contrast to /core/modules. We could put a little README.txt at /modules/example/README.txt explaining what this is, and that you have to remove the "hidden = TRUE" from the foo.info file to be able to enable the example modules.

I would like that more than a /example folder, because
1. /example is a non-standard location, so doesn't have the learning-effect of something in /modules.
2. I think removing or commenting one line is a more trivial task than copying a folder for novice users.

Both of those are debatable and not really killer arguments, so I'd be fine with them being dismissed, if you guys don't like this proposal :-).

No, then we'll have to deal with Examples modules passing SimpleTest, and the extra "hidden = TRUE" seems like more work than copying the directory.

Oops.

My proposal of core/examples/modules doesn't hold water because

1. Example modules *must* pass their tests. We *want* that. I know there are alternate opinions, but core is the only place that tests will be maintained, so the tests for the examples *must* be run by the automated testing system, and broken things must be fixed. That's the whole idea here.
2. IMO Example modules must be hidden from the modules page - we don't want people accidentally encountering these. We *could* put them in a closed fieldset, or have a checkbox that says "show nonessential modules" or something. This is the same problem with mock modules for simpletest, and is easily solved with "hidden = TRUE" in the info file... but of course people have to be able to find theem as well.

So I'm back to agreeing with core/modules/examples.

@rfay: I totally agree on the tests. There's little point having them in the core repo if they're not run regularly. Any thoughts on /modules/example(s), though?

No thoughts on the 's' (core/modules/example or core/modules/examples). I think the first patch gets to choose :-)

1. Example modules *must* pass their tests. We *want* that.

Well... a strong -1 on that. Plenty of reasons in #41 and #28.

As someone who's regularly inventing and re-inventing core APIs, I feel that I already have more than sufficient work to do to get major API changes into core today. Updating API documentation and correcting modules that are actually used is one thing. Having to update code that only exists for documentation purposes, as well as very potentially tons of textual documentation within (which in turn not only requires updating, but also novice-user testing, grammar and spell-checking, etc.pp.), goes way beyond of what can reasonably be asked from a core contributor.

And as mentioned before, I probably need to exclude myself here, since I belong to the group of ~0.001% contributors who might actually be able to deal with this additional barrier. My concerns mainly revolve around new/other core contributors, who are staying away from core due to its barriers even without this additional one already. Our intentions and goals really need to be to get those people involved [again], instead of raising barriers even more.

But then again, TBH, before I'd do that, I'd send offending files+tests to /dev/null, and create a Novice follow-up issue to re-create them from scratch for the new API.

In summary:

1) If anything lacks tests, then we need better test coverage, not examples.

2) Examples are pure documentation and should not conflict with active development.

As others mentioned before, one possible way to combat 2) would be to only introduce examples after ("final") feature freeze of a major core version, AND: To delete them entirely when opening up the new major version branch for development. That would ensure that examples do not hinder ourselves in re-inventing Drupal for the next major version, and essentially treats them as documentation, which can be changed and improved at any time and is not bound to feature/code freezes in any way, by design.

I do not question the usefulness and benefit of examples in core. Not at all. However, it is easy to predict the negative impact of example module code, documentation, and tests on the re-invent/feature/development phase of a release cycle and how it would affect everyone who wants to improve and contribute to Drupal core. I'm 100% confident that we do not want that to happen.

Sorry, @sun, just like with every other feature of core, Examples can't be maintained without tests. Consider the tests to be for the examples.

However, my experience in D7 says that the Examples tests are the canary in the mine. They are early warning of an unintentional API change or API failure.

But it would be completely impossible to maintain examples without automated testing of them. Not a -1 but a -infinity. This would take us back to the days of D5/D6 when the few examples we had were unmaintained and unreliable. Can't happen.

Putting Examples in core innately means that we have to run their tests in the automated testing system.

Can you provide reasons for that?

I fail to see how and why any example code would be able to break. The actual API needs sufficient test coverage. If that is not the case, then the actual problem is a lack of test coverage.

It cannot be in our interest to have additional test coverage for API functionality in completely unrelated example modules. Doing so would be architecturally wrong. Tests are tests, examples are examples.

If you want to form the DCTAIAS Club™ (Drupal Core Tests Are Incomplete And Suck Club™), call me in and count on me, yesterday. :-) However, that task force needs to work on improving the tests and test coverage for code in core. Adding tests to documentation only works around the actual problem. If something does not work as expected and it can be tested, then there has to be test coverage in the originating code itself. Adding it elsewhere instead of the origin's tests would be illogical. And, if it cannot be tested automatically, then there cannot be tests in the first place, neither in examples, nor in the code that examples are trying to use.

Can you clarify what I'm missing?

@sun, the only thing you're missing is experience maintaining examples.

1. Example code is like anything else. A simple patch can make it stop working, even if multiple people have reviewed it. Examples is not just "example code", it's *working examples* that are known-good. And one way they're known good is that their tests run and keep them good. (Despite the valid complaints about code format standards, the actual functionality and tutorial quality of the examples has been quite good in recent years. )

2. Core can and does change when it doesn't mean to. This breaks examples. They then need to be fixed (or on rare occasions core needs to be fixed). Without a way to detect this, all is lost.

IMO the thing you're missing is that Examples are live software that has to be maintained throughout a software lifecycle. Automated testing is a standard part of that expectation in all modern software development. We have to have them tested or they can't be maintained at all.

Hm. I need to think some more about #78.1).

However, #78.2) is the exact case and incarnation of what I was referring to:

Core can and does change when it doesn't mean to. This breaks examples.

In that case, core does not only break examples. It also breaks a dozen of hundreds of Drupal sites out there. API contracts cannot and must not be broken. That's what test coverage is for: Asserting expectations.

Examples are not here to assert expectations. They demonstrate potential usage as a documentation instrument in a complex, modular, and intertwined system. Due to that, 2) is an invalid argument for test coverage from my perspective. I still need to think about 1) though... It's true that any kind of code can contain bugs, regardless of who wrote it and how extensively it is manually tested. OTOH, those hypothetical tests would duplicate existing test coverage across the board. Duplicate code is bad already. But duplicate tests are every developer's nightmare. Again, 1) is food for thought.

Let's step back here a second and not have a flame war! As I see it, here are the questions:

a) Are Examples in Core considered "Documentation"?

b) If so, are they a part of the Documentation that needs to be updated at the same time as a patch that changes an API that they use -- i.e. part of the "gate" for patches? (the alternative would be that they could be updated in a follow-up patch)

I believe that rfay is advocating that the answer to (b) is YES, and the way we ensure that YES is followed is by having the Example tests be part of the Core tests that get run for each submitted patch. (Which is actually better than the situation for other documentation, which relies on human patch reviewers to catch places where we should have updated existing or written new documentation when an API changed.) rfay: if I have mischaracterized your position, please correct me.

Any other thoughts? Have I missed anything fundamental in summarizing the discussion from the past few comments?

Also, back to the question of where to put these example modules... here are some ideas:

Proposal A: In /core/modules/examples (or maybe core/modules/example without the s?)

Proposal B: In /modules/example(s) -- i.e. under the directory where we are currently recommending people put their custom/contrib modules

Proposal C: in /core/example(s)

Proposal D: in /examples (outside of core completely)

If we do A or B, I think the proposal is to have the "hidden" flag set in the modules' .info files, so that unless that is edited, the module cannot be enabled from (or seen on) the Modules page. (Does that still work in D8?) This would make the README instructions basically say "To enable an example module, edit that line in the .info file and then go to the modules page or use Drush" or something to that effect. And we wouldn't have to do anything for the tests and other PSR-0 classes in examples to be detected (which could be a drawback if we do not want the example tests to be run with the rest of core tests, which is still being debated).

If we do C or D, then the instructions would be "copy the directory and all its subdirectories to /modules and then enable", or something like that. And if we wanted the tests to run automatically (which is still being debated), we'd presumably have to patch core so it would look there for PSR-0 classes.

Were there any other ideas? Any pluses and minuses or votes on any of the proposals?

#80 @jhodgdon: "having the Example tests be part of the Core tests that get run for each submitted patch."

+tenmillion.

Of course that's also a commitment to updating the examples when the tests fail in order to make a core patch, as part of the core dev process.

#79 @sun: "OTOH, those hypothetical tests would duplicate existing test coverage across the board. Duplicate code is bad already. But duplicate tests are every developer's nightmare. Again, 1) is food for thought."

One thing that never fails to annoy me while I'm looking at tests for Examples is that there is no core test for API compliance. Like, if you implement a field, there should be a semi-automatic test that checks, for instance, that you've implemented hook_field_info() and other tell-tales.

So the answer to this is to make usefully sub-classable tests in core that developers can and will use when they implement the APIs. This will lead to better contrib, because there's no excuse not to write tests.

I did a little bit of this in the field_permission_example module. Viz.

But that is, as they say, another ticket.

At this point in D8, having to fix Examples test failures really doesn't matter anymore.
But if this had been done before feature freeze, I personally would have likely quit core development.

The amount of time it takes to get a major change into core is already astounding. This would have put it over the edge.

If this does happen, I'll probably be the first to reopen an issue when D9 development starts to revert it.

@tim.plunkett, your fear of maintaining Examples through a core cycle is perfectly reasonable (and is why we haven't tried to do Examples through the D8 cycle). I did it in D7 and it was horrific.

There's an easy solution though - Just remove examples or disable tests at the beginning of the cycle, and then make sure they get done at the end. The reality is they could be added in one at a time just like we're talking about doing now for D8.

Okay, I'm on board with that. Thanks @rfay.

Hm. If we aren't going to maintain them through a release cycle, is our plan then to remove them as soon as we start development of each new branch, and then port them at the end of the release cycle as we are proposing doing now for d8?

If so, I don't think this proposal solves the "no one is maintaining them" problem -- if developers are not maintaining examples as they go, we'll still need a dedicated team to port them in the end. It would only be possibly solving the "no one knows about them" problem.

We still need an issue summary...

I did some statistical analysis on the hook implementations across contrib: https://gist.github.com/4409685

Looking at that list, it looks like for core we might want to focus on:

- block_example
- dbtng_example
- entity_example
- menu_example
- nodeapi_example
- page_example
- theming_example

I have just added an issue summary, hopefully it's ok.

@webchick: thanks for the analysis!

I think it's safe to convert this into a meta issue.

New item to do: Propose a list of the new API subsets which in D8, so we can have an example for each of them.

I've just opened a more specific meta-issue for the initial port to D8 which seems to be previous to add it to core: #1880976: [meta] Port examples to D8

Thanks, marvil07.

++ for inclusion of Examples in Core, especially ++ @webchick's "shortlist" (#87).
++ @rfay (#84): "Just remove examples or disable tests at the beginning of the cycle, and then make sure they get done at the end."
++ @jhodgdon's Proposal A: In /core/modules/examples (or maybe core/modules/example without the s?)

++ for inclusion of Examples in Core, especially ++ @webchick's "shortlist" (#87).

To the shortlist I think it would be really helpful to add rest_example and serialization_example as those are new systems and critical to positioning Drupal as a viable mobile/web-app backend.

Status:Postponed» Active
Issue tags:+d8dx

This should be a key part of the D8DX initiative.

Issue summary:View changes

Start the issue summary.

Issue summary:View changes

Some updates and a lot of clarification.

Just head about this initiative to put the examples module in the core in D8 and think this is the best idea I've heard in a long time so came here to voice my support.

And while we are here please take the PHP filter module out at the same time

@dahousecat: PHP module has already been removed per #1203886: Remove the PHP module from Drupal core