[META] Getting started with Rules 8.x

thanks, I've replied here: http://drupal.org/node/764558#comment-4728092

Since #764558: Remove Trigger module from core is picking up, want to turn this issue into a "[Meta] Prepare Rules for Drupal core platform"? Something like:
// Checklist & TODO
- organize documentation
- clean up issue queue
- port D7 to D8
- etc

Let's use this issue to track topics that relate to this contrib project's prospective merge/partial-merge with core. A good example is tracking efforts by maintainers and contributors on both ends that match each other's expectations. Another topic to discuss is how core changes would necessarily affect Rules' API, and possibly vica versa.

One commonly talked about core improvement that I don't know where any development is happening is for getting core actions to match feature parity with Rules' actions api. See also #1008166-10: Actions should be a module.

I asked sun for his perspective of the overlap with Rules' API and D8's platform direction and planning requirements, what he thought was that Rules' engine needed a major reworking in order to make it core-ready. Almost direct quote, what goes into core "will need to look very different from what it is right now." See: #1498908: Internal review of Rules API design.

His reasons boiled down to Learnability, especially with respect to DX but also UX. He also said providing a simplified UI would put the proposed inclusion at par with features that other core features provide. We don't have an issue for this yet.

Another issue that came up in Dries's core conversation, relates to the implications for fago and klausi's continued control over the direction and release cycles in the future. Dries focused on how packaging distributions on d.o might enable a core install profile to depend on contrib projects. This is one option for allowing "core," or at least common core products, to use Rules without Rules being directed by the core development cycle. I highly recommend this video.

He also described Phoenix as a core-like distribution that would depend on contrib projects. I didn't finish watching it yet, so I don't understand much about it, but I also heard that Phoenix had also been proposed as a solution for rapid release cycles, (like Mozilla and Google are doing for their browsers).

And then the another key topic in this issue is Snowman Project. This project shows how core developers need robust solutions and that, in the mid-term, could alternatively mean redeveloping simplified APIs, see #1210366: Per-bundle node listing pages, blocks, feeds., versus working out merge/migration paths for more robust solutions.

Open questions in my mind:
How would symfony's context object / session object / path router thing be supported by Rules D8, as an event? What other changes do people see for Rules in D8?

Has there been filed any issue against core for inclusion of a basic/API version of Rules (Rules Core + Rules Engine I guess)?

The most important one is:
#1696640: Implement API to unify entity properties and fields

The TypedDataAPI there will be the foundation for all over Rules, replacing the entity property info system and its Rules extensions of D7. This means, Drupal's data will natively be described in a way it works with Rules.

Then, scotch is about to work on conditions in core - see #1743686: Condition Plugin System. As mentioned there, actions should just work the same. I.e. having a revamped and unified actions API that Rules can leverage is *important*, but for that we need to get the TypedDataAPI done first.

I'm happy to have found this issue. Just an update...

#1696640: Implement API to unify entity properties and fields is in! And #1743686: Condition Plugin System needs reviewers!

It is obvious that Rules will not be added to Drupal 8 core. Disclaimer: this is not a bad thing. My main question is: are there concrete plans regarding the development of the public API for Rules for Drupal 8? The earlier that API has been conceptually finalized, the earlier other developers can start offering Rules integration. I'm trying to see if it's possible to coordinate development efforts across different projects (so this is by no means a "Rules 8.x-3.0 should be released the day Drupal 8.0 comes out" post).

Right now my focus is on bringing conditions and then actions apis in core into shape, such that they will be re-usable for Rules-8.x - see #1743686: Condition Plugin System and #1846172: Replace the actions API. Afterwards, we should start on flushing out the Rules-API based on plugins asap and make sure the system works for us. As second step, I'd care about the UI and lastly about the upgrade path. Of course, any help would be greatly appreciated here!!

Just following up to see how this is progressing now the the 2 issues above have been resolved (for the most part).

Any updated information would be nice.

Thanks in advance.

Suppose we have a nice phpunit examples in core views. At the same time views uses own conditions (filters) and contexts (contextual filters)

I'd like to point that page manager and panels require contexts and arguments too so better fix this in core! Also this will help a lot of contrib modules like context, spaces, purl and probably others

@klausi Is there any core issue already about conditions and contexts already to polish (improve)? I think this could be done under 'API addition' tag ;)

Condition plugins are already ContextAware, but Action plugins are not.

I guess conditions don't need to provide variables, but actions need a mechanism to optionally specify what their outcome is and the type of newly produced variables.

And I need to check how certain conditions in D7 make the type of parameters more specific, e.g. how the "content is of type" condition populates the bundle with article so that article fields show up in the data selector for actions.

thanks klausi for the great start. I'd suggest we start creating individual issues for hard questions and assign this issue as parent - so we can keep discussions of a single topic in a single place.

I guess conditions don't need to provide variables, but actions need a mechanism to optionally specify what their outcome is and the type of newly produced variables.

Yep, core provides the context system for paramters. We'll have to ensure this meets our requirements, i.e. have optional contexts, allowed-empty contexts vs. non-allowed empty contexts (default). Then the harder part is that we need the same metadata for configuration paramters and a way to know where we can find them in the form, such that rules can override them.

I created #2145343: Rules events for Drupal 8 as a follow-up.

@fago, as you go, keep Drupal 8.1 in mind. We have an opportunity to extend the platform so that it can provide the extra pieces you need to support rules.

Hi,

on our way to #drupaldevdays I put together some notes while discussing the Rules to D8 port with fago & klausi:
https://drupal.org/files/issues/Rules%20D8.png

Sources in Dropbox.

Watch out for more details during the conference.

Let's discuss plans and get started tomorrow at the DrupalDevDays sprint. If you are interested in helping out, please join and sign-up at https://docs.google.com/spreadsheet/ccc?key=0AmlCU4x5nvoBdF80dW9hU1ppZWR... (at the bottom)

Here is a summary of our discussion at Szeged. It ended up being rather long, but well we discussed quite a bit :)

Summary

During Drupal Developer Days in Szeged me, dasjo, klausi and Xano discussed the port of Rules quite a bit and made a plan for moving on. Here is a summary of our conclusions and discussion points:

General

Given all changes in Core, porting Rules to 8.x is not a trivial tasks and requires updating of the existing concepts to leverage the new core APIs instead of their D7 counterparts. As lots of those APIs changed fundamentally (New action & condition APIs, Entity property info -> Typed Data, Exportables entities -> Config entities with CMI,..), huge parts of Rules have to be re-factored and updated accordingly.

Forking action and condition systems

Given the release of Drupal 8 is nearing, it is critical to get Rules 8.x out of the door as soon as possible. In order to be able to move on as fast as possible, we decided to basically fork the Drupal core action and condition systems for now, i.e. we’ll override the services with Rules equivalences. That way we can easily work in one repository and quickly do necessary improvements to the action & condition systems. Once we’ve got Rules 8.x working with Drupal 8, we’ll know more on what exactly is needed for the core implementations be useful for Rules, whether there are BC breaks and see whether we can get the improvements into Drupal core or not.

In case this does not work and the Rules and core action & condition systems cannot operate on the same set of provided actions / conditions, Rules will have to provide separate, i.e. duplicated action and condition systems again. Unfortunately, this is not unlikely to happen if we fail to include the improved versions in Drupal 8. E.g. the differences on how you provide actions are significant when you’ve got support for token replacements provided by the engine (Rules or core actions) or not - as in the latter case the action will have to take care of token replacements.

Action / condition context and configuration

We decided to merge configuration and context metadata of conditions and actions in a single “context” array, similar to the “parameters” array in Rules’ Drupal 7 actions. This solves ordering questions and works along with what Rules has been doing in Drupal 7. Action and configuration plugins can be provided with configuration forms in core now, what makes sense to us in particular for stuff like boolean options - however it is not yet exactly clear how the configuration form will relate to the Rules generated form.
One idea that I came up was letting the plugin have full control over its configuration form (if it wants to), while it can embed regular Rules UI elements via respective API calls. Generally, the required context of a plugin will be defined similarly as one defines field definitions for an entity, so defining custom related widgets/formatters will probably be possible in a similar way. Lastly, the so usual approach of having Rules auto-generate the whole form and doing only alterations will still remain possible.
The API for contextual conditions in core has no real usage in practice yet. It’s a very slim layer around typed data definitions, which will have to prove its usefulness and/or might require some refactoring for being able to solve real-world use cases.

Typed data support for tokens

For Rules everything that is selectable via its data selector (=all available typed data), will have to be available as token replacements as well. Instead of integrating all typed data tokens directly with the core token API, we decided to use/provide a separate token replacement API service which will use the regular data selection API to navigate through the data, while it invokes the token system for rendering the last part to a textual token. That way we do not conflict potentially differently provided tokens in the system (as it is the case with entity tokens), while we are able to leverage the token system for textually formatting data.
Xano has been up for doing the necessary improvements in core, but it has yet to be determined whether this will reach community consensus. (See #2164635: Automatically expose typed data to token API and related issues). Otherwise this will be provided as separate API in contrib.

Global context

For the selection of global context and global (site-related) tokens, we need a way for modules to define/add globally available data. Instead of providing a data type with a “global marker”, we’ve been discussing doing a separate, small plugin type which allows one to easily register globally available data (e.g. like og context).

Allow the re-use of underlying API additions

While a lot of the underlying APIs of Rules 7.x were easily re-usable otherwise as part of the Drupal 7 entity module, there are more API components of Rules that are useful outside of Rules. This comprises stuff like the Rules data selection widget, Widget and Formatter plugins on the typed data level, but potentially also stuff like general Context mapping components (API with or without a respective UI) as it would be needed for Blocks&Layouts or Panels in D8 as well. As for actions and conditions, all those components will be developed driven by the Rules use cases only so we can save time for now, while making sure the components are decoupled to allow for potential re-use. Whereever these components end up being distributed/packaged will be determined later on as well, for now they will be developed in a single repository.

Event plugins

One idea that came up during the discussion, is relying on Symfony events for providing Rules events. That could be done in a way modules can implement Symfony events without having to depend on Rules, while they provide the additional metadata needed by Rules as well. Next, this would allow us to easily expose existing Symfony events - even of 3rd party libraries - as Rules events.
This could work something along the following:
- Symfony events are based on an event base class, which provides access to related event variables. This class, can serve as plugin class to Rules if modules put it into the right folder and add the right annotation(s) providing all necessary metadata.
- Additional Rules specific event related code (settings forms, event dispatcher) would have to be provided by implementing optional interfaces, or maybe by related classes/plugins also.
- For adding symfony events, we would have to add in necessary metadata only, e.g. by support reading related yaml files.

Configuration schema

A tough issue is how to provide configuration schema for a Rules configuration as it depends on which action / condition plugins are used to configure it. That’s a problem we share with Views though - so there should be a solution for that in core.

Process

Given the amount of work that is required for the port, it seems unlikely that the port can be completed anytime soon without any funding. Therefore, we’ve discussed funding possibilities and started estimating the amount of work necessary. fago, and klausi would be up for working on it, while dasjo would help out mostly on the coordination & management side of things. Inspired by the success of the VDC initiative, we’d also love to get new contributors on board, potentially sponsored by their companies to help making Rules in Drupal 8 a success.

We’ll do some more planning and provide additional information once we have figured the details, but for now the plan is to first talk to potential corporate sponsors and try crowd based funding in second step.
During the port we’d plan to have at least bi-weekly meetings on IRC/hangout and public status updates.

Misc

The RulesLogger class will be converted to a service.
Configuration entities will serve as storage only, while the main API does not rely on them in any way. The Rules API & UI needs to be re-usable independently on how the resulting config is stored.
We decided to tag the ported code as 8.x-3.x unless some other semantic versioning guidelines are released in meanwhile.

i have added a meta issue for the roadmap + created individual issues for the sub-tasks of the first milestone:
#2245015: [META] Rules 8.x Roadmap

We've got #2245015: [META] Rules 8.x Roadmap now and I've created #2245611: [META] Rules 8.x architecture for an issue that can hold serve us as anchor for technical planning + provides an overview of the latest plan. For a better overview, I think we should close this one. I've linked it from the new issue though.