Problem/Motivation

This issue is a continuation of the discussion started at #218755: Support revisions in different states. It may be beneficial to look there for history on this topic.

Currently Core effectively understands two workflow states: published and not published. This is insufficient for all but the most basic workflow implementations. While there have long been contrib modules which attempt to rectify this shortcoming, it would be far more powerful and efficient to build a foundation for workflow support into Drupal Core. A first-class CMS deserves first-class workflow support.

Proposed resolution

Implement support for defining an arbitrary number of workflows, each containing an arbitrary number of states. Allow each entity to be managed by a different workflow and each revision of an entity to be assigned a different state.

Out-of-the-box, Drupal would have one default workflow. This could be today's two-state workflow—"unpublished" and "published"—but there has been support for increasing this to a three-state workflow: "draft", "published" and "archived".

Remaining tasks

  • Identify common needs of comprehensive workflow systems.
  • Design and implement a workflow API in Core.

User interface changes

Much of the UI for managing states could easily be left to contrib to solve. If core moves to a three-state default workflow, then there will need to be some changes to the UI, differentiating between "draft" and "archived," both of which are unpublished.

API changes

This will mean the creation of a new API to manage workflows and their associated states. Existing code will also need to be changed to recognize states beyond published/not published. The precise changes required are still TBD.

Related issues

Related modules

Comments

videographics’s picture

As long as we're in the conceptual stage, I think it's important that we think about these 'workflows' as having the potential to be non-linear. To some, the word 'flow' suggests something linear and directional, but 'flow' can also be a back and forth or something that naturally 'flows' wherever it needs to go.

klonos’s picture

I'm not an expert but I believe that implementing non-linear workflow would be a lot easier if we had a basic/minimal Rules API in core. Too late for this?

PS: ...a little bit of usage stats to further support my case (everyone loves stats):

- About 1 out of 7 Drupal installations (currently ~725k, counting only D6 & D7) has Rules installed (~116k). This makes it a ~15%.
- Out of the total of the ~260k D6 installations ~40k have Rules installed. So also a ~15%.
- Out of the total of the ~465k D7 installations ~76k have Rules installed. ~15% again.

agentrickard’s picture

No Rules in core. It's code bloat and a recipe for fail. The current actions system is bad enough.

jstoller’s picture

Here's what I was thinking...

We allow the creation of any number of workflows. Each of these has a unique machine name, a human readable name and a description. Workflows are basically holding containers for groups of states and transitions.

Each workflow can have any number of states. Each of these states also has a unique machine name (perhaps name-spaced by the workflow), a human readable name and a description. Permissions for interacting with entities can be set independently for each state.

Workflows can define transitions between any pair of states. Only defined transitions will be allowed and permissions for each transition can be controlled separately. All transitions are directional so, for instance, you can have different permissions for draft-->published and published-->draft. The ability to set up transitions between any arbitrary pair of states means that both linear and non-linear workflows are supported. Every transition can be used as a trigger for actions and rules.

Workflows would be associated with entities. The core API should include a mechanism to assign workflows to entities based on their bundle, but it should permit config to use other criteria to assign workflows to entities (i.e. taxonomy, author's user ID, time of day, etc.). There should also be a method to override workflow assignments on a per-entity basis.

States are associated with revisions, so you can have multiple versions of the same entity in different states.

I think this is all pretty basic and extremely flexible. The goal is largely to provide a set of tools which contrib can use to build great things. By building the underlying framework into core though, we can reduce some redundancy and encourage greater compatibility across implementations.

klonos’s picture

Yes, as I said I'm no expert. It's just that the projects in the issue summary + any workbench* project put together total less than ~30k installations (only ~12k if you take under account the fact that their installations most likely overlap) and we're still considering adding their basic features in core.

While I completely understand that this issue here is specifically geared towards implementing/improving the workflow of node publishing and I honestly don't want to derail discussion, it still seems to me that having a basic Rules *API* (not the entire module including UI that could possibly mean code bloat as you say) would greatly help us achieve what we're after here as well. All the workflow-related contrib modules out there would greatly benefit from this in the long run, but this will only be a tiny gain compared to the one of having a generic and standardized way to have core "react" to various "things".

jstoller’s picture

@klonos, While I'm not opposed to improving triggers and actions in core to provide a better foundation for modules like Rules, I do not think this is the place for that conversation. I'd suggest opening a new issue, if one doesn't already exist. I'd also suggest avoiding the phrase "Rules in Core", since that seems to strike a nerve with many people. Perhaps "improved event reaction API" would work? In any case, I'd like to try to keep this issue on topic, if we can. It's a big enough problem to solve on its own.

I also want to emphasize that this should not be node specific in any way. All entities should support workflow states.

videographics’s picture

Issue tags: +access permissions

jstoller, I think #4 seems very well thought out. Can I try to simplify this conceptually to make sure I understand?...

  • A workflow is a set of one or more states and transitions between pairs of states.
  • Each state has a set of access permissions.
  • Each transition has a set of access permissions.
  • One or more workflows get associated with entities (entities, content types, individual nodes, etc.) at various degrees of granularity.

And, here's an example of how the existing two-state workflow might look: The workflow with two states (unpublished/published) would be assigned to all nodes by default and the access permissions associated with each state are the way they are currently. Anyone with access to edit a node has permission to publish it.

If you wanted to add draft and archive states, you simply create and configure a new workflow and assign it to just those content types that require the extra control. Then, instead of just seeing publish and unpublish, some users see a different set of options on certain content types. Other than that, nothing else will visibly change for the user...

A few more thoughts...

I also think that in some situations, you could maybe get away with having only one state (with no visible interface elements). (I actually work with lots of situations where the existing published/unpublished model gets in the way.) Also, if a user didn't have appropriate permission to publish, the publish/unpublish control could be hidden instead of telling them 'sorry, you can't publish that' after the fact. This could be done without a workflow API, but having one would make it a lot easier.

I think layering a contrib workflow on top of the existing publish/unpublish workflow using a parallel interface is just wrong. This API could help to simplify the UI by putting any additional workflow-related options in the same place as the current publish/unpublish controls.

I think in addition to having access permissions for each transition, each state will require its own access permissions. If each state has its own access permissions, and then this is then all slapped on top of the existing access permissions architecture, it might get kind of messy. BUT, I think here actually lies a great opportunity for integration between this proposed workflow API and the existing core access permissions architecture. The result could be a much simpler and easy to manage system for managing content permissions:

users -> roles -> states -> entities

This way, you could say... "Administrators can view, edit, and delete all content in an unpublished state" instead of having to manage huge arrays of checkboxes for this. The default interface for managing content access permissions could also be vastly simplified if it were to have access to the power of workflow states. You'd essentially be 'grouping' the various entity/content types according to their workflows and then assigning permissions to each workflow state. So far, this is the best strategy I've found for abstracting those endless grids of checkboxes we're used to seeing on our permissions pages. (The same level of granularity could still be achieved by creating additional workflows if they were truly needed.) Currently, a site with 10 content types and 5 roles currently presents administrators with a 5 x 50 grid of checkboxes for managing access when it could be done with 25 or possibly fewer.

You could also say, "Technical editors have permission to publish these 5 types of content on their own but these other types of content need to be submitted to a manager for review." Managing this kind of thing at a granular level is a currently a nightmare. Realistically, this is beyond practical. Assuming the different groups of content types had different workflows, this could be managed very easily with the proposed workflow API.

I think in addition to facilitating workflows, a workflow API like this could also end up making it much easer for site administrators to manage access permissions throughout a site. By assigning content access to states and then assigning states to content, we then have the ability able to abstract the complexity of managing access permissions in so many places as we currently do. When thinking about states' access permissions in this way, there develops a rough analogy between states and entities similar to the relationship that currently exists between roles and users. (Just imagine not having roles available next time you go to manage access permissions.) I think the workflow API has the potential to become at least as useful and vital as roles is today.

I'm curious to hear your feedback on this...

videographics’s picture

Issue summary: View changes

Changing the link to Workbench to a link to Workbench Moderation.

jstoller’s picture

@videographics, sorry it's taken me so long to respond. I've been a bit wrapped up with the space shuttle Endeavour and I'm afraid Drupal has to wait.

I think you've got the basic idea. A few clarifications though…

Each individual entity would have a workflow associated with it. Practically, this means we add a 'workflow' column to the node/user/comment/taxonomy/ tables, referencing the machine name of the workflow associated with each individual entity. Similarly, we would add a 'state' column to the node_revision/user_revision/comment_revision/taxonomy_revision/ tables, referencing the machine name of the state associated with each individual revision.

At this point I think we need to restrict ourselves to one workflow per entity. I just don't see a relevant use case for multiple workflows controlling a single entity. This of course would not prevent users from creating extremely complex workflows, with many states and non-linear transitions, as long as it's a single workflow.

I don't see any practical issues with a single-state workflow.

I think I understand what you're saying about access permissions and, yes, I agree this opens up great possibilities. It makes perfect sense that permissions would be state based, rather than based on an entity's bundle. And since the same workflow can work across many bundles, this could easily simplify permissions management. There are two caveats thought. First, because we're allowing unlimited states with unlimited transitions, that simplification can easily vanish. But I don't see that as a problem. Second, if we base permissions on workflow states, which we probably should, then we have to provide at least a minimal interface for creating new workflows. Previously I thought we could just throw the UI to contrib, but I don't think we can realistically get away with that now.

More thoughts…

The new 'state' column would replace the existing 'status' column. So being "published" means the default revision of a node is in a state that unauthenticated users are able to view. However, there may be many states which meet this criteria for a given workflow. In addition to the standard "published" state I could have an "archive pending" state, which the public could still view until an admin approved the transition to my "archived" state. Or a user could be bumped to a "questionable" state, keeping the account active, but noting suspicion of questionable activities which may warrant deactivation.

Permission to view revisions other than the default revision should be tied to the workflow managing an entity. So maybe I can view revisions for entities governed by the "marketing" workflow, but not for entities governed by the "accounting" workflow. Because entities of the same type can be governed by different workflows, this could again simplify access permissions and provide a ton of flexibility.

mitchell’s picture

Title: Support arbitrary workflow states » Support pluggable workflow states
Issue tags: +Usability, +Plugin system

If #1801570: DX: Replace hook_route_info() with YAML files and RouteBuildEvent provides pluggable request methods for interacting with entity revision states, then this could work hand-in-hand with Proposal for RESTful entity API.

mitchell’s picture

Issue summary: View changes

Added revisioning

jstoller’s picture

@mitchell, I'm afraid those issues are a little beyond my comprehension right now, but whatever works is fine by me.

Anyone want to pick this up and work on implementation? Please? :-)

lpalgarvio’s picture

lpalgarvio’s picture

Issue summary: View changes

uuid

jstoller’s picture

Re: #11, I've added #220825: Does an unpublished status mean Draft or Disabled? to the list of related issues in the issue summary, but marked that issue closed in lieu of the discussion here.

jstoller’s picture

It looks like feature freeze for D8 is fast approaching and I assume this qualifies as a feature. Anyone interested in putting together a patch, or are we kicking this to D9?

yoroy’s picture

Would be great to get some basic plumbing for this in place. Can't help with the code, but in so far as the new 3-state workflow isn't implemented yet I think that's where the work should start? A bullet list action plan might help.

jstoller’s picture

@yoroy, has a new 3-state workflow been decided on? If so, where is that discussion happening? It seems to me that if we're making the jump to a 3-state workflow, we'd be far better off implementing it as the default case of an n-state workflow.

I'm afraid I can't be much help with the coding myself. And in light of #9 above, I'm a little unclear how this feature should be approached. @mitchell, could you expound on your thinking a bit?

I'm not sure how relevant this is anymore, but for historical reference, if nothing else, here is the data structure I had been envisioning. I figured we need three new tables:

{workflow} table:
wfid <-- The machine-readable name of this workflow
name <-- The human-readable name of this workflow
description <-- A brief description of this workflow
module <-- The module defining this workflow
weight <-- The weight of this workflow in UI listings
{workflow_state} table:
wfid <-- The machine-readable name of the workflow this state belongs to
state <-- The machine-readable name of this workflow state
name <-- The human-readable name of this workflow state
description <-- A brief description of this workflow state
weight <-- The weight of this workflow state in UI listings
live <-- Boolean indicating if this represents a published state
access_callback <-- The callback which determines access to content in this workflow state
access_arguments <-- A serialized array of arguments for the access callback
{workflow_transition} table:
wfid <-- The machine-readable name of the workflow this state transition belongs to
state <-- The machine-readable name of the starting workflow state
new_state <-- The machine-readable name of the ending workflow state
transition <-- The machine-readable name of this transition
name <-- The human-readable name of this transition
weight <-- The weight of this transition in UI listings
access_callback <-- The callback which determines access to this transition
access_arguments <-- A serialized array of arguments for the access callback

Other schema changes would include:

  • Add a 'wfid' column to the {node_type} table (and any other table defining entity bundles), indicating which workflow should govern entities of that type by default.
  • Add a 'wfid' column to the {node} table (and any other table defining entity instances), indicating which workflow governs that node. This overrides {node_type}.wfid on a per node basis.
  • Add a 'state' column to the {node} table (and any other table defining entity instances), indicating the state of the default revision.
  • Change {node}.status and {node_revision}.status to {node}.live and {node_revision}.live. This boolean would still indicate whether the content is published or not. The name change is to prevent confusion between "state" and "status" when writing code.
  • Add a 'state' column to the {node_revision} table (and any other table defining entity revisions), indicating the state of each revision.

I'm mostly talking about nodes here, as an example, but this would need to be expanded to all other revisionalble entities. You'll also note that I am maintaining the old binary concept of a "published status", in addition to the new n-state workflow. I think a simple boolean indicating whether content is publicly visible will likely still be useful, even in more advanced workflows. However, there may be several states in a single workflow which are all "live".

Once this foundation is set, every core module defining an entity would need to define a simple 2-3 state default workflow for its entity bundles.

Most of the UI for workflow management could be left to contrib for now. I think the only thing we really need is a way to handle state transitions on the node edit pages, which it seems is being handled by #1751606: Move published status checkbox next to "Save".

Any thoughts on this? Anyone on this issue have the skills/desire to code it?

Sylvain Lecoy’s picture

As you said, a first-class CMS need a first-class workflow system.

I am for adding arbitrary workflow states natively in core, plus the entity API now support revisions natively.

yoroy’s picture

I think everybody would agree that something of the kind belongs in core. With another almost 3 months to get this feature complete, it would be wise to come up with the skeleton or prototype code to kick off the architectural discussion in a more hands-on way.

Paging mitchell…

Wim Leers’s picture

Issue tags: +in-place editing, +Spark

Note that all of this is also relevant to Edit, which is most likely going to get into core: #1824500: In-place editing for Fields. Edit uses Create.js, which has a Workflow widget that we should be able to integrate with cleanly. Then we end up in a really cool place: things like correcting minor spelling mistakes, but more importantly: the entire review process, can happen on the front-end…! The moderator will be able to moderate/evaluate the content while it looks exactly like it will eventually look.

Tentatively adding tags.

mitchell’s picture

Title: Support pluggable workflow states » Add a default workflow to standard install profile
Issue tags: -API, -state, -core, -Plugin system +Snowman, +☃

In #9, I started to say that this feature overlaps with the routes layed out for WSCCI in Crell's table. Both features depend on the same work, and the architecture choices there also describes this.

What I find interesting about that RFC is that both these UX/DX environments (forms & http) for working with content are tightly connected to diffs. See also, the Diff library in core.

The suggestion about using plugins wasn't relevant. I was thinking about how the PATCH method would be implemented in rest.module, see #1826688: REST module: PATCH/update. There's also a little more about the potential for using pluggable content in #79582: Add example content to the experimental out-of-the-box demo install profile, but there are better ways.
---

Some thoughts on implementing this:

This functionality would also be very useful to Snowman, so I'm grouping it with that initiative.
This would exemplify how workflows can be built, in general, so tagging Platform Initiative.

* access permissions:
There's also some more WSCCI overlap with: #1793520: Add access control mechanism for new router system. A plugin's access controls can be configured in code, or in the future, there may be a plugin ui for the condition plugin which would allow site builders to make access controls. This aspect can use some input from those issues.

* 'published' access controller property:
Entities could achieve this node-type-like feature using #1803064: Horizontal extensibility of Fields: introduce the concept of behavior plugins (links there to D7 example). What's the difference/connection between a published behavior and a status field? I know this has already been discussed, but I'm just mentioning it for the use of Entity Field API toward providing revisioning workflows to all entities rather than only node entities.

* publish action/operation:
See also #1839516: Introduce entity operation providers and #1846172: Replace the actions API.

* diff in general:
I think a thorough review of the implications of diff inclusion in core (functionality regression :) are in order.

* related issues:
This and #1776796: Provide a better UX for creating, editing & managing draft revisions. appear deaply connected. That is the front end, this is the back end?

webchick’s picture

"With another almost 3 months to get this feature complete"

Correct me if I'm missing something, but I don't immediately see anything here significantly in progress to warrant a feature extension.

jstoller’s picture

@webchick: This issue was created to continue the work of #218755: Support revisions in different states, which has a committed core patch. At the time that patch was developed, it was understood that it was only the important first step toward full implementation of the originally requested feature. We moved the discussion to this new issue purely for convenience and I hope that will not count against us. From where I stand the workflow initiative has seen great progress in D8, both from the afore mentioned core patch and work being done in other issues like #1751606: Move published status checkbox next to "Save". If someone was able to take this the final mile over the next few months and fully implement what we've been working toward for many years now, I would hope it could be committed to core. Unless you're saying this is a non-starter, I'm still challenging us to make this happen and I encourage those with the skills to pull this off to take a shot.

webchick’s picture

But unlike #1751606: Move published status checkbox next to "Save", this issue has had absolutely no patches, reviews on patches, testing of patches, etc. in the past ~3 months, since that original patch was originally committed. I don't see how it therefore could possibly qualify for "feature completion" phase. Unless I'm missing efforts that have been ongoing leading up to feature freeze, additional (and customizable) workflow states are going to have to be added in contrib.

webchick’s picture

Issue tags: -Spark

Also, removing the Spark tag. While we did work on the ability to get draft revisions in core as part of #218755: Support revisions in different states—which is a critical enabler to various workflow modules in contrib, as well as enables further refinements to the UX for #1824500: In-place editing for Fields—this particular functionality isn't something we're planning to take on.

jstoller’s picture

@webchick, I'm sorry to hear that. I may be biased, but I see this as making a number of recent and in-progress D8 features more complete. But my opinion is not the one that counts, so if this has to be pushed to D9, so be it. :-(

In any case, I don't want this discussion to die.

@mitchell, I'm still not sure I'm completely following you, but let me see if I understand what you're proposing.

The different workflows governing content are bundles of a Workflow entity type. Each instance of a Workflow bundle would represent a state within that workflow. So my "Default workflow" bundle might contain three entities: draft, published and archived. But I could create a "My custom workflow" bundle that contained six entities: draft, review, pending approval, approved, published and archived.

Alternatively, we could have two different entity types: Workflows and States. State entities would have a field referencing the workflow entity they're grouped under and workflow entities could potentially have a reciprocal field linking back to the states they contain. This approach may provide more flexibility.

However these workflow entities are constructed, all entities would have fields referencing both the workflow managing them and the state they are in. Entity bundles would have a property indicating the default workflow and state when new entities of that type are created.

Am I getting this right?

The idea of states and workflows being entities is certainly intriguing. I suppose then they could be fieldable themselves. Is using the entity and field systems like this acceptable, or will it add too much overhead? I recall that in #218755: Support revisions in different states I proposed something like this and Crell said using fields for states was a "non-starter." He indicated "Fields are lousy for extrinsic data." Of course, that was two and a half years ago. Have things changed? (@Crell, are you here?)

If this were implemented, would you still keep the current status column for entities, or do you see this reference field as completely replacing that?

Crell’s picture

Still here. :-)

No, Fields are still crappy for extrinsic data at an architectural level. We keep using fields for extrinsic data (cf #1751210: Convert URL alias form element into a field and field widget) because we don't have a proper extrinsic data system, but that doesn't mean it's a good idea to use that hammer on those screws. An entity extrinsic data system is totally not happening in Drupal 8, however.

That said, with the way entities have evolved in Drupal 8 such that they've split into two levels (content and non-content), using non-content entities for state *records* a la Workbench Moderation 2.x makes more sense. It does not make sense for the configuration OF a workflow or state; those should be CMI data, whether they're passed through the ConfigEntity system for convenience or not.

jstoller’s picture

Title: Add a default workflow to standard install profile » Support arbitrary workflow states

Changing back to original title. It seems more accurate to me.

Pancho’s picture

Priority: Normal » Major

Per #17 and others, this is a major feature which doesn't mean though it will happen in D8, unless avoiding API changes and DX repercussions (which should be very hard to achieve). Not bumping it to D9 for now, though.

Berdir’s picture

Version: 8.x-dev » 9.x-dev

I don't think this is still possible for 8.x.

Berdir’s picture

Issue summary: View changes

Add issue #220825 to the list of related issues.

catch’s picture

Priority: Major » Normal
Status: Active » Postponed

This is doable in contrib, and hard to get right, so postponing on a properly working contrib solution that doesn't have data loss issues between revisions (which I have not seen in existence yet). Something which blocks a contrib solution should go into a separate issue.

webchick’s picture

Version: 9.x-dev » 8.1.x-dev

That said, if such a thing were to materialize, it could go into a minor release, no?

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.0-beta1 was released on March 2, 2016, which means new developments and disruptive changes should now be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Crell’s picture

Status: Postponed » Active

This sounds like the "Workbench in core" stuff that was discussed as part of the content workflow initiative at DrupalCon NOLA. Reactivating.

timmillwood’s picture

Issue tags: +Workflow Initiative
amateescu’s picture

Status: Active » Closed (outdated)

We are indeed bringing Workbench Moderation in core :)

The plan issue is at #2755073: WI: Content Moderation module roadmap and the implementation issue is #2725533: Add experimental content_moderation module.

Gábor Hojtsy’s picture

Status: Closed (outdated) » Closed (duplicate)

Given that we have other issues where the same thing is being implemented this is duplicate. Outdated are for issues that passed in time (eg. implement support for PHP 4.4 in XYZ).