Deal with documentation role and documentation input format

/me whistles innocently.

This needs to be reviewed and decided individually on a per-content basis.

IMG is just one of many more tags available in the Documentation text format. It also allows for tables, PRE, and many more markup.

And as others already stated, some content is using the Documentation format on purpose to limit edits to a "trusted group".

I just want to add that I am not concerened about breaking existing pages: it is widely known that people shouldn't link to external images.

I'm not too terribly concerned with the "locked" pages being unlocked. Some of those locks are ancient and should probably be undone anyway. Anything REALLY important usually has Full HTML format anyway so only site admins can even get at it.

Docs input format isn't the right solution to managing precious pages as far as I'm concerned. If it's that important, we can compile a list of really important pages to protect and re-protect them.

I would love to see the docs admin role repurposed for people who will manage a curated docs area, and for allowing people to manage comments and url aliases - things that docs maintainers regularly do.

(Can this be reopened? I'm not really sure why sun closed it already?)

I am not sure either why was it closed.

Note: closing duplicate issue in Docs queue: #1254298: Refactor "documentation maintainer" role

Ok, here's what I would really like to see here:

1) Docs input format should only be used to "lock" VERY important pages -eg. ones authored by security team. Not just ones that someone annoyingly edits too much. If there is some other functionality available for "locking" pages, that'd be even better - would love to see "page locking" and docs input format de-tangled.
2) Docs Admin role be stripped from all users then added back for a selected (small) group of admins who will also likely become the curated docs editorial team.
3) Docs Admin role (after stripping it from most of the users that currently have it) should have the following permissions added:

- to create/manage URL aliases
- to manage book outlines on the http://drupal.org/admin/content/book/ pages
- to delete comments (to help with killing comments)

That will alleviate a large burden from "the docs co-lead in charge of online docs" aka. me - I cannot even begin to keep up with all of the requests being filed in http://drupal.org/project/issues/documentation right now. Having a team of people be able to address all issues in the queue and all docs maintenance tasks would be fantastic.

I'll post back if any other useful permissions come to mind.

+1 on #11.

So we need to figure out what those very important pages are.

Like for instance anything in Coding Standards, because they need to be addressed in issues before they are updated.

For locking pages - might as well use full HTML instead of documentation.

I am on the security team, but this is not official word from them. I don't think security documentation is that special; there are plenty of people capable of making it better, on and off security team. What we have to watch for is people making it worse, same as any other documentation page. A convenient way to watch pages would be nice to have, but that's another issue.

For URL paths, I think manually editing these is crazy. We should install pathauto, #1117436: Deploy pathauto & token for organization nodes?, and let the computers do this.

Ok, so an aside: this issue should be marked will not fix? #1285296: Should coding standards be locked

Re: URL aliases - i'm happy to let the robots do this, as long as we can still override it for eg. for top level module pages or top level book pages (there's no way for the robots to know which pages those are)...which means docs admins should still have access.

RE #13 pathauto - The vast majority of book pages on d.o have no aliases at all. Are you proposing giving all of them aliases? And what settings would you use for when the title changes (I think the choices are "keep the alias and add a new one" or "new alias and redirect from the old one" or something like that)?

I am not convinced that we want to take our often fairly long page titles (we encourage long titles, because that's the first thing you see in search results, and it's nice to be descriptive) and turn them all into really long URL aliases...

tagging

Even if we have pathauto, it seems sane to allow this "documentation team" role to override aliases and add them to pages outside the scope of pathauto. So, can we just agree to give that perm to this role and move the debates around pathauto for book pages to another issue? I'd prefer this issue didn't get sidetracked into a debate about the merits and appropriateness of pathauto for documentation.

Other random thought: does it make sense to rename the role to "documentation administrator" to make it clear it's a more admin-y role than just a "team"? (I.e. not to be given out lightly).

+1 to changing the role to "documentation administrator", and to adding URL alias perms to that role (at least once we restrict it to a few people).

+1 to not combining path auto with this issue.

FYI: cross-linking #1298020: fix nodes and comments with images that don't meet new requirements from greggles... It's related to the question of converting nodes from Documentation to Filtered HTML so I wanted to mention it here.

Bump. What do we need to do to get this to happen?

arianek and I discussed this in IRC today. Given comment #1 above and comment #11, here is a revised proposed plan (which arianek hasn't explicitly seen/approved by the way):

a) Leave existing pages as they are currently. As people request access to edit pages that have the Documentation input format on them, if deemed not to be "sensitive" pages, switch them to Filtered HTML (and fix any images that are problematic on those pages, via download from external / upload).

b) Going forward, only use the Full HTML format to "lock" pages. Change the help text for the Documentation format so that the first line says "DO NOT USE" or something similar. Or better yet, change its name to "Deprecated Documentation Format" or something like that (open to ideas on how to get across the point of "this shouldn't be used").

c) Add a few more tags to the Filtered HTML format that Doc currently has:
<br> <pre> <acronym> <summary> <div> <small>
Note: I don't know that any of these is really necessary, but at least it would mean there is no reason to use Doc format instead of Filtered HTML

d) Leave the Docs Admin role in existence with all the current users, but make it only confer the Docs input format usage priv.

e) Create a new role "Documentation Moderator" or "documentation administrator" or whatever it should be called. Give it to about 20 people. Give that role the privs currently on Docs Admin, plus (if not there already)
- unpublish pages
- use the Full HTML format and Documentation as well
- to create/manage URL aliases
- to manage book outlines on the http://drupal.org/admin/content/book/ pages
- to edit/delete comments (to help with killing comments)
- anything else needed???
These people would be watching the Documentation issue queue for requests to change input format on locked pages, delete comments, respond to inappropriate content, etc.

Thoughts?

arianek pointed out that unpublishing pages requires 'administer nodes' permission. Which is dangerous. Let's not include that after all.

21/22 sounds right to me +1

Jennifer and I were just trying to sort out what the best name for the role would be and wanted to suggest "Community Docs Moderator"

webchick: this was in response to your comment #1 above, that we didn't want to break access to existing pages for existing people?

Agreed with the form alter idea - much better than what I suggested.

Ok, if we're going to go the more complicated/lower technical debt route then.... I come back to wanting to remove the docs input format and switch all those pages back to filtered (if it's possible to make sure all the needed html tags are available in filtered - is that a security risk to implement?)

Then we're running into the problem of img tags referencing outside images -- see #1298020: fix nodes and comments with images that don't meet new requirements, which currently is only talking about pages that previously were Filtered HTML that was filtering out img tags...

ok so we're back to:

- add to filtered input: pre acronym summary div small
- do the external image handling patch
- move all docs input to filtered + delete docs input
- relock any needed pages with full html
- remove docs maintainer role from all users
- add it back to the smaller group
- add new perms

ya?

Bump. We still need to figure this out. My first priority would be to move all Documentation pages with the Documentation input format to have the Filtered HTML input format. Then we can go back and lock the few that might still need to be locked (by assigning Full HTML), and fix any broken images that result as time goes on.

Proposal for how to move forward on this:

a) All of the Book node pages that have Documentation format: change them to Filtered HTML. Any that have broken images can then, as people notice and see the magic Edit button, be fixed at our leisure (since everyone will have edit privs on them anyway).

b) Get rid of the docs admin role completely. If people complain that they can no longer edit their Project pages, or other nodes like issue summries, we can change the input format on those pages to filtered HTML and then they'll be able to edit them.

c) Rejoice and close this issue.

I'll put this into the issue summary, since there isn't one right now.

Well...... I'd like to change B and C instead to what was in #11/#21e, ie. to strip docs role from everyone, then add it back just to a small team who can help with those sort of meta docs management tasks. Though it seems we can leave out any perms geared towards curated docs maintenance for now.

Ah, good idea. OK, amended proposal is this:

a) Change book nodes with Documentation format to Filtered HTML.

b) Change name of Documentation input format to some other name, so it doesn't look like it's intended for Documentation. Maybe "Old deprecated don't use this"? "Nearly full HTML"? "We don't need this any more"? "Deprecated"? "Intermediate HTML"? Any better ideas?

c) Remove the current Docs Admin role.

d) Create a new role "documentation moderator" or "documentation administrator" or whatever it should be called. Give it to about 20 people who have recently been active docs maintainers (list TBD). Give that role these privileges:
- publish/unpublish pages -- I guess that means "administer content"?
- use the Full HTML format (and Documentation, so they can change pages from Doc to Filtered).
- administer URL aliases and redirects
- manage book outline
- administer comments

b) if all the nodes have had their format converted, we could maybe delete it? Not sure what will happen to revisions using this.

In (a) I am specifically only proposing this for Book node pages. As noted in comment #2 above, it has been used for issue nodes, project nodes, and others. If we change them to Filtered HTML and they have outside-the-site images, they will break...

OK, revised proposal:

a) Change book nodes and project_project with Documentation format to Filtered HTML. [Some images may break, but since the pages will be editable by anyone (books) or the project owner (projects), they can be fixed them as they are noticed.]

b) Change other nodes with Documentation format to Full HTML. This will leave the screen shots in them intact, at the cost of them being not very editable.

c) Remove the Documentation input format.

d) Remove the current Docs Admin role.

e) Create a new role "documentation moderator" or "documentation administrator" or whatever it should be called. Give it to about 20 people who have recently been active docs maintainers (list TBD). Give that role these privileges:
- publish/unpublish pages -- I guess that means "administer content"?
- use the Full HTML format (and Documentation, so they can change pages from Doc to Filtered).
- administer URL aliases and redirects
- manage book outline
- administer comments

I've reconsidered the deletion of the input format: we should rather make in unavailable to all roles instead.

Sounds fine (unavailable vs. deleting). I think we should also amend (a)/(b) in #36 to make sure this is done in all *revisions* of those nodes, not just the node's end revision, right?

I don't think we need to change the format on the old revisions if we keep the revision around but unusable.

If unusable means we can't look at diffs, then I think we need to change the format.

"unusable" means you can't use it on new revisions. Diffs should work fine.

Thumbs up from me!

OK, I think we have 3 thumbs up on this proposal:

a) Change book nodes and project_project with Documentation format to Filtered HTML. [Some images may break, but since the pages will be editable by anyone (books) or the project owner (projects), they can be fixed as they are noticed.]

b) Change other nodes with Documentation format to Full HTML. This will leave the screen shots in them intact, at the cost of them being not very editable.

c) Remove all permissions for use of the Documentation input format.

d) Remove the current Docs Admin role.

e) Create a new role "documentation moderator" or "documentation administrator" or whatever it should be called. Give it to about 20 people who have recently been active docs maintainers (list TBD). Give that role these privileges:
- publish/unpublish pages -- I guess that means "administer content"?
- use the Full HTML format (and Documentation, so they can change pages from Doc to Filtered).
- administer URL aliases and redirects
- manage book outline
- administer comments

(I'll put this into the issue summary.)

So I guess the next step would be to figure out the queries that need to be run in order to do (a) and (b), and the list of people to give the new role to in (e)? Assuming that killes, jhodgdon, and arianek is enough agreement?

Add a real issue summary

Not that you need more agreement, but thumbs up from me, too. ;)

For (a) and (b) it'd be nice if the queries were in a hook_update_N() in drupalorg so they could be tested on a dev site and automatically repeated on the live site without an opportunity for human error. Hopefully that doesn't slow you down, and will make this easier, not harder. If that becomes a pain, ignore this. ;)

Yeah, obviously we'll need the list of people to put in the new role, and you and arianek seem most qualified to produce that list...

Thanks,
-Derek

I can do (a)/(b) in a hook_update_N -- not a problem. I even have a dev server to try it out on. I'll take it on.

List of people... that we'll have to think about a bit. I'll confer with arianek next week after DrupalCon.

re: people - indeed - i've got some thoughts, and of course we'll have to give some brief instructions/best practices to those who get the perms.

Re: "we'll have to give some brief instructions/best practices to those who get the perms"

I highly recommend just writing a page under Drupal.org site maintainer's guide with whatever folks with this role need to know.

Also, we're going to need to do something about Documentation maintainers which is prominently linked from About Drupal.org.

Thanks,
-Derek

We should remove that link. Agreed with the idea of writing a manual.

Let me clarify. I don't want to give the impression that the people listed on the Maintainers page are maintaining the Community Documentation. That undermines the work we have been doing lately to get the community maintaining the docs. So I don't want to have a link that gives that impression on the About drupal.org page. Is it OK if I just remove it?

And as a note to me, the patch that does the queries should also remove the code that generates that docs maintainers page. I'll add that to the issue summary.

update to plan in comment #43

That all makes perfect sense to me. +1 to removing that link from the about page.

Thanks,
-Derek

I've made an edit to the "who are the maintainers" section of http://drupal.org/about-drupal.org -- take a look (will be happy to edit if it's not appropriate). (Removed the docs maintainers link and replaced with a sentence saying the community maintains the content.)

add note to remove code for maintainers page to taks

OK, let's see. On the docs-infra test site at the moment, running the query in #2 yields:

 SELECT n.type, COUNT(DISTINCT(nr.nid))
    FROM node_revisions nr
    INNER JOIN node n ON n.vid = nr.vid
    WHERE nr.format = 5
    GROUP BY n.type;

+-----------------+-------------------------+
| type            | COUNT(DISTINCT(nr.nid)) |
+-----------------+-------------------------+
| book            |                     918 | 
| forum           |                       3 | 
| image           |                       5 | 
| organization    |                       1 | 
| page            |                       7 | 
| project_issue   |                      17 | 
| project_project |                     246 | 
| project_release |                      50 | 
| story           |                       1 | 
+-----------------+-------------------------+

Which is similar to the results in #2, except fewer issues (the test sites don't have all the issue nodes).

So rather than trying to figure out which docs pages and issues have the Documentation input format, I edited a couple and gave them that format.

Then I ran the following queries:

UPDATE node_revisions nr INNER JOIN node n ON n.vid=nr.vid SET nr.format = 1 WHERE n.type IN ('book', 'project_project') AND nr.format = 5;
UPDATE node_revisions nr INNER JOIN node n ON n.vid=nr.vid SET nr.format = 3 WHERE n.type NOT IN ('book', 'project_project') AND nr.format = 5;

I got the right numbers of updates:
1. Query OK, 1165 rows affected (0.41 sec)
2. Query OK, 86 rows affected (0.75 sec)
and afterwards, the first query above yielded no results.

I also spot checked my two test nodes, and their input format was set correctly (the book went to Filtered HTML, and the issue went to Full HTML).

I then went in the admin interface (again, on my test site), and set the Documentation format so that no one had permission to use it.... Hmm... It looks like anyone with 'administer nodes' permission can still use it though. Nothing much we can do about that. :(

I then logged out, and logged in as a normal user, and verified I could still view revisions and revision diffs correctly on both test nodes, and that I could edit the book page and not the issue, which is what I'd expect.

So all that remains is to make these queries into a patch, which I'll do now.

Correction to #52: You need 'administer filters' permission, apparently, to use filters that otherwise have no role permissions.

Meanwhile, here is a patch, which after attaching, I'll apply on my dev site, test, and report back about shortly.

It should take care of items a, b, and c in the Proposed Resolution section of the issue summary:
- book/project nodes with Docs format -> filtered HTML
- all other nodes with Docs format -> full HTML
- remove code for generating the docs maintainers page (node/109372).

To test this, I did the following:
a) Applied the patch from #53 on http://docs-infra-drupal.redesign.devdrupal.org
b) Went to http://docs-infra-drupal.redesign.devdrupal.org/node/109372 - it is missing the list of docs maintainers, as expected.
c) Logged in as user/1.
d) Since I had already run the queries in #52, I set two nodes (one book, one issue) back to the Documentation input format.
e) Ran update.php. It offered to run drupalorg update #6604 (as expected), and the output was (as expected):

drupalorg module
Update #6604

    UPDATE {node_revisions} nr INNER JOIN {node} n ON n.vid=nr.vid SET nr.format = 1 WHERE n.type IN ('book', 'project_project') AND nr.format = 5
    UPDATE {node_revisions} nr INNER JOIN {node} n ON n.vid=nr.vid SET nr.format = 3 WHERE n.type NOT IN ('book', 'project_project') AND nr.format = 5

f) Went back and checked my two nodes. The issue was set to Full HTML, and the book node was set to Filtered HTML (as expected).

So I think this is working... tentatively setting to RTBC, although I would suggest testing on a full d.o staging site before deploying live I would think?

Committed and running on staging.devdrupal.org.

Deployed.

Excellent!

We still need to take care of one final step:

f) Create a new role "documentation moderator" or "documentation administrator" or whatever it should be called. Give it to about 20 people who have recently been active docs maintainers (list TBD). Give that role these privileges:
- publish/unpublish pages -- I guess that means "administer content"?
- use the Full HTML format
- administer URL aliases and redirects
- manage book outline
- administer comments

So we need to come up with a list of people, create the role, and assign it. I've posted to g.d.o to ask for volunteers... and we can also contact people we think should be appropriate and see if they're willing:
http://groups.drupal.org/node/221109

Back to Documentation project for this final step

For future reference, I think that users that had this role should have been notified of this change. It was a bit jarring to suddenly find that i could no longer include screenshots from skitch (etc). Obviously this isn't the biggest deal, but it would have been nice to know...

/me feels demoted

Sorry about that! It was discussed extensively on two issues in the Documentation, Infrastructure, and Webmasters issue queues (see references above).

I renamed the Documentation input format to "DEPRECATED Documentation" so I (and others, but mostly me) would stop using it. :)

fix some inconsistencies

Status update: Quite a few people have said they would like to be docs moderators, but most of them don't apparently have much experience with documentation. So, I posted http://groups.drupal.org/node/221109#comment-755743 to ask them to start doing the types of contributions that don't require elevated permissions. I also created http://drupal.org/node/1588928 as a more permanent page to describe the responsibilities.

Just filed
#1623014: Create docs moderator role and grant to batigolix
so we can put this issue to rest.

That other issue has been fixed, so this issue is done! As more people step up to the responsibilities of being docs moderators, we'll add this role to them.