Allow README.md to optionally render as the project page

I've wanted this for a very long time.

Not only would it be less painful than writing HTML, but it would most likely lead to better project descriptions with more information on the project page. I often have to dig into the repository to find the README for the information I need. Github has shown that you don't need a separate lengthy description (yes, they do have the tiny one at the top) and a README, you can just render the README.

Maybe project pages created on d.o. should also add a readme.md in the repo?
So in both directions...

This would be totally rad. Here is the library Github uses to render markdown (points to some other libs too)

https://github.com/tanoku/redcarpet

Good idea, but needs code.

Some remark:

- We wouldn't want to directly read files from the webhead. We'd need to keep them in sync on all of them.

- The contents of the files should be written into the database by some job triggered from the git commit.

Also, since we don't want to complicate the D7 transition this should be a feature for after that transition was completed.

Sounds like this should maybe be a small contrib module that extends D7 Project* to enable this functionality? That way we have something to test and review?

I'm in favor of Dave Reid's suggestion, I've been helping with the D7 Project* stuff and I would be glad to help test and contribute to a project markdown page module (or whatever we call it).

Don't see why this would be a big deal to implement. It's an awesome idea by the way. Got my buy in.

+1

So far we've got a lot of markdown fanboys in favor (I'm among them). So, let's figure out why people might be opposed and address that.

Are there some people who *don't* want this? I think I probably don't want it for most of my projects personally. I assume the web content is more for evaluators while my README.txt is more for people who have committed and need to know how to configure.
How will we let people disable this feature if they don't want it?
Who is willing to write the code to get this done? (and why haven't you started!)
Is there anything at the infrastructure-level they will need to get it done?

From drumm in #2229141: Update projects homepage text with the README.md content if that exists which is now a dupe:

This could go to versioncontrol_project module. I'm guessing it will be implemented via the plugin system in the versioncontrol_* modules.

Nobody seems to be opposed to this after 1.75 years of talking about it!

So...if anyone wants this enough to work on this please go ahead.

If you want to implement it in PHP, then it's just this:

composer.json:

{
  "require": {
    "php": ">=5.3.0",
    "michelf/php-markdown": "~1.3"
  }
}

ReadmeReader.php:

use Michelf\MarkdownExtra;

require __DIR__ . '/vendor/autoload.php';

$markdown = file_get_contents($filename);
$html = MarkdownExtra::defaultTransform($markdown);

As @sun pointed out (in IRC), this has been done by multiple project hosting systems and isn't really "github" based.

Also, this should be optional (perhaps enabled by default) in the project's settings. I would suspect that there will sometimes be a need for both.

Personally, I am not sure this is a good idea. For my point of view, project pages and the README-files serves two different purposes.

• Project pages are intended for users that are searching for a contributed module that will fulfil some requirement they have when site building. Good project pages are often mildly SEOed (to make them findable by search), and need to tell prospective users what to expect from the module, and how it is different from similar modules. There are guidelines for project pages that sums this up.
• README-files, on the other hand, are intended for users that has downloaded the project and now is looking for instructions for REQUIREMENTS, INSTALLATION, CONFIGURATION and TROUBLESHOOTING. That is also what the README template says.

By rolling the two into one, that distinction is lost, and I am not convinced that is a good idea.

PS: Just for the record, a link to the official guidelines for README.txt (they're, among other things, used as canon by reviewers of Drupal.org Project applications.

If you end up making README.md the source code for project pages, these guidelines probably need to be changed as well to reflect that change.

In #13, I suggested this should be completely optional. Until a project has such a need for distinguishing a difference between the project page (SEO/whatever) and the project's README (for whatever other reason), it should remain enabled by default. Yes, the guidelines would also have to be updated to reflect these changes once it is implemented, not a big deal.

Also, existing projects should not be forced to have this feature enabled by default and should remain as an "opt in" feature. The "enabled by default" should only apply to newly create projects.

Also, FWIW, "two different purposes" usually only really applies to larger projects. The majority of projects would benefit greatly if they didn't have to maintain two different "README"s (which are often the same).

Thank you for the feedback, Mark Carver.

For the record: I've changed my mind. I now think that this will be beneficial. As you say, project owners that think their project will not benefit from this can always opt out.

I've amended the guidelines to make README.md formally acceptable in the project application queue (AFAIK, they're already informally accepted).

+1 to get README.md parsed and shown (optionally) on the project's homepage

I'm not against this, but there are many markup languages, and that's what drupal filters try to serve, so we should consider using one of them as the "render code" portion.
Agreed about non-directly reading the git repository directly from webheads, so this probably would need the same idea than git code parser, and notice we have the filenames per branch already in versioncontrol tables, so the missing part would be the code to store the rendered output.

It sounds like a new "plugin", so it could be flexible(i.e. not only for markdown).

One extra note not mentioned before: A project has different branches, and naturally can have different versions of that README(even radically different) in different branches. How that should be picked up?

marvil07 wrote:

A project has different branches, and naturally can have different versions of that README (even radically different) in different branches. How that should be picked up?

The obvious rule is to use the README from the newest recommended release available. (i.e. D8 if a recommended release for D8 exists, otherwise D7, D6 and D5 in that order).

Since the use of README for the project page shall be optional, project owners can always create a custom project page "by hand" that describes all the releases available and how they differ, if they think that will be better.

As for there being "many markup languages": Please note that the documentation guidelines only permits markdown (in addition to plain text). Since these files need to be human readable in addition to parseable by the filter, I don't think we should go overboard in what type of markup we permit.

If a project has more than one branch I think the easy way would just be to render the one they have chosen as the default branch (that would also get more people to not have it set to the master branch).

Or the other way would be to do it Github style and have a branch/tag selector on the project page and it would change based on that.

https://github.com/fish-shell/fish-shell/tree/1.16.0
vs
https://github.com/fish-shell/fish-shell/

DrupalDelphia is coming up in two weeks, Sept.12th, anyone up for sprinting on this then?

http://drupaldelphia.com/

p.s. Are there instructions for getting development setup for D.O?

Here is a sample mockup of the option we could put on the project settings page just below the description. If you click that it could just grey out the description above so the maintainer would still have access to the data if they ever need it.

I'd like to chime in. Before coming to Drupal I did some WordPress work and put a plugin on WordPress.org: http://wordpress.org/plugins/facebook-albums/. Their interface is strictly based off of parsing the plugin's readme. Which is kind of cool, but sucks if you just want to make a simple change. (Also.. they're SVN, and the plugin page is built off of the release tag files and not the trunk... which if we're off of default branch HEAD, happy camper here.)

I dig #24. If I want to use the module's README off of the current branch's HEAD I check that. If I want to make a minor announcement without committing, I should be able to disable that option and edit imported HTML/markdown. Then, deciding "Ok! We roughed things out enough" I can enable this once more and it imports the README and project page is shiny and updated.

As for #15, I agree that there are two different roles (now.) However, maybe this can push people to use D.o documentation pages and link to them from the README? More projects are utilizing their own Wiki vs long and drull READMEs (as far as I can tell, don't shoot me for the generalized comment.)

Are there instructions for getting development setup for D.O?

Yes, https://www.drupal.org/node/1018084.

If we go with a checkbox, I'd put it above the description - so it is more likely people will read it before spending the time to edit the description.

Thanks Neil!

It's a neat idea but I wouldn't use it because I use project pages completely different than the readme. I see the intent is already to make it optional so I'm just chiming in as another pro-optional voice in case that ever changes.

I think this is a great idea so that time does not have to be spent maintaining multiple documentation. I think README. could benefit from being more like project pages and project pages could benefit from being more like README. It seems like a win-win to me.

An Minimum Viable Product suggestion though would be to just enable the markdown filter for the description field so that maintainers can simply copy and paste the README.md from their module and paste it in the description to have it display what they want. In this manner, the maintainer can decide which markdown they want to display simply by which they choose to copy and paste. It is an extra step, but it is a pretty small one and gets us 90% of the way there.

A couple of suggestions for the longer term plan:

* I like the checkbox idea for using the markdown file or not.
* Keep the description field active and displayed even if "show the README' is selected, so that if a maintainer wanted to display extra information, they could.

readme-drupal.redesign.devdrupal.org is being built out to work on this, requested by BR0kEN at #2512612: I want a drupal.org development site to create support of *.MD files for projects.. See https://www.drupal.org/node/1018084 for information on Drupal.org dev sites.

We'll want 2 fields added to the Features exported to the drupalorg project:

• The checkbox for using README.md
• A text field with the content of README.md, using a hidden form widget, so it isn't visible when editing.

This will let code disable that checkbox when README.md isn't present; since we always have a copy in the DB. To do that, let's start with a drush command to populate the field for a single project. We can script running that for existing projects that have README.md, and hook it up to code arrival and default branch change events later. We can make a temporary clone and pull it from there, if there isn't a more straightforward way.

I'm not totally sure if this should live in drupalorg or this project. This project is certainly technically ideal, but will need to come with things like an update hook to make the fields, in addition to drupalorg's Feature.

Think I've done with this. Was added a possibility to use README.md from branches and tags.

See demonstration here: https://readme-drupal.redesign.devdrupal.org/project/mimeinfo

Completely forgot to remove my test repo from code. Here is correct patch.

+++ b/sites/all/modules/drupalorg_markdownizer/drupalorg_markdownizer.module
@@ -0,0 +1,97 @@
+            $block['markup'] = preg_replace('/&lt;\?php<br(\s+\/)>|(\?&gt;)?/i', '', $block['markup']);

Need to mark (\s+\/) as optional.

+++ b/sites/all/modules/drupalorg_markdownizer/drupalorg_markdownizer.install
@@ -0,0 +1,139 @@
+    'type' => 'list_text',

Better to use "select" instead of "radio" buttons, due to economy space on the page when project has a lot of branches and releases.

Need to remove this line, because default widget for "list_text" - is a "options_select".

Previous patches should be ignored.

Manually tested and look through code, all seem to be fine.

Setting to CNW for the following reasons:

1. There are @todo's in the above patch
2. #2191525: [PP-1][policy, no patch] Extend Markdown coding standards to support API module (DOXYGEN) (which this should probably be postponed on anyway, but leaving as CNW since it can be worked technically and independently of that).
3. I've created https://www.drupal.org/project/commonmark, which is a little more comprehensive than existing Markdown modules. I also created it under the assumption that the above issue will be adopted (which is probably the direction this issue should be moving towards too).

I "very like" the community. Instead of using working feature it postponed it for a "ten years". Cool.

We are like perfectionists: want to do ideally or nothing.

I am setting this back to RTBC. I think that the push-back in #41 was unnecessary and based upon not reviewing the correct patch and jumping the gun on a policy change that is still under review (and not likely to be resolved soon).

Here's addressing each single point raised in #41.

1. There are no @todo's is the #38 patch (the @todo's are in the #34 patch, which is no longer relevant).
2. The current status of #2191525: [PP-1][policy, no patch] Extend Markdown coding standards to support API module (DOXYGEN) is "Needs review". As long as there is no official decision on this, I think it is fine to make this work with the current de-facto standard markdown module.
3. You shouldn't tie the completion of this task to your own CommonMark project. While I understand that you want to see this more widely adopted (there is currently 2 installs resulting from 848 downloads), your project is irrelevant here until CommonMark is given official approval.

When and if CommonMark is officially approved, this may need to be tweaked. But it is IMHO quite wrong to block this very important project from progressing because of a possible (and IMHO unnecessary) future policy change.

It is wrong to proceed with something that does not even have a general consensus in the community with how or what should be in markdown files, let alone what kind of markdown will be supported (not specifically "CommonMark", which is what that issue was about originally btw if you read it).

These are critical questions that should be answered before something like this is implemented. What happens when that policy is adopted and existing project pages break because the the format isn't the same? It is foolish of us to proceed just for the sake of "getting it done" without an actual plan in place.

You shouldn't tie the completion of this task to your own CommonMark project.

I'm not.

While I understand that you want to see this more widely adopted (there is currently 2 installs resulting from 848 downloads)...

No, I don't. It's a small simple project, one that was created solely as a POC for that issue. So yes, it does have a very small install count.

... your project is irrelevant here until CommonMark is given official approval.

No, it's not. You really are missing the entire point of that issue. Just because it's "my module" is really of no consequence to me.

The only reason I created it is because there was push back in that issue to use a more standardized markdown library (which is why "CommonMark" was chosen, by others, not myself). Nothing existed, so yes... I'm the one that stepped up to help push along that issue... I don't see anyone else helping with that issue.

This isn't some attempt to blanket my name on things and quite frankly, I take offense to that. I'm here to help and it's inevitable that sometimes I'll create projects to move things along. Use it or don't, but again, not the point of that issue.

---

You can refute everything I said in #41, but it doesn't change the fact that #2191525: [PP-1][policy, no patch] Extend Markdown coding standards to support API module (DOXYGEN) needs to be addressed first. We cannot implement something that doesn't even have any standards in place within the community... like, at all.

markcarver wrote:

These are critical questions that should be answered before something like this is implemented. What happens when that policy is adopted and existing project pages break because the the format isn't the same?

Do you think people who write MarkDown sit down and look up the syntax before creating their README.md-file?

If so, you're wrong. People use MarkDown because there is no such thing as "syntax error" in MarkDown. And the MarkDown you'll find in README.md-files (which is the only thing relevant here) is just basic stuff (headings, single level lists, non-nested emphasis and boldface). At that basic level, there is no significant difference between John Gruber's MarkDown and CommonMarc.

Do you think that no README.md-files exists in repos yet, and that people with patiently wait for #2191525: [PP-1][policy, no patch] Extend Markdown coding standards to support API module (DOXYGEN) to be decided before committing any?

If so, you're wrong. The Advanced help module ( 128547 installs) has for several years successfully rendered README.md-files that is packaged with a project (leveraging on the Markdown filter module ). These README.md-files are not going to be rewritten if you ever manage to conclude on what syntax to "officially" adopt in #2191525: [PP-1][policy, no patch] Extend Markdown coding standards to support API module (DOXYGEN).

This project is not about new README.md-files that people are going to postpone writing until #2191525: [PP-1][policy, no patch] Extend Markdown coding standards to support API module (DOXYGEN) has decided on a "policy". It is about making the thousands of README.md-files that has already been written usable on the Drupal.org project page.

The fact that #2191525: [PP-1][policy, no patch] Extend Markdown coding standards to support API module (DOXYGEN) seems completely oblivious to this huge installed base of MarkDown on Drupal.org is to me clear indicator that this project is poorly aligned with the community and whatever "policy" that project end up with choosing will not be enforceable.

However, I am not going to enter into a status war here by setting status back to RTBC, but for the record: I think your actions here are disruptive and are based upon poor judgement.

I see nothing in the proposed implementation that should be contingent upon one markdown flavor or another. Please set this back to RTC.

The issue summary is pretty much non-existent (there is a first/opening post, but not really a summary). We shouldn't RTBC before the summary at least reflects the current state of the issue.

The branch specifity issues raised in #14 and addressed in #22 may pose UX issues and I did not see anyone comment on those yet. The setting this patch introduces handles the branch issue from a maintainer's point of view, but does not do anything to clear up potential confusion on the visitor's side. I recommend we request for at least one UX review before going ahead with this.

Some documentation feedback:

1. +++ b/sites/all/modules/drupalorg_markdownizer/drupalorg_markdownizer.install
   @@ -0,0 +1,138 @@
   + * @param array $fields
   

   @param array is not explicit about the array contents. This should be @param array[], because the values are arrays themselves too.

2. +++ b/sites/all/modules/drupalorg_markdownizer/drupalorg_markdownizer.install
   @@ -0,0 +1,138 @@
   + *   An associative array where keys - field names and values - definition.
   

   Please write full sentences: Keys are field names and values are field definitions.

3. +++ b/sites/all/modules/drupalorg_markdownizer/drupalorg_markdownizer.install
   @@ -0,0 +1,138 @@
   + *   When cannot create a field.
   

   This is not a correct English sentence. It should be something like When any of the fields cannot be created.

4. +++ b/sites/all/modules/drupalorg_markdownizer/drupalorg_markdownizer.install
   @@ -0,0 +1,138 @@
   +  foreach ($fields as $name => $data) {
   

   $data should be $field_definition to be self-descriptive and to match the documentation.

5. +++ b/sites/all/modules/drupalorg_markdownizer/drupalorg_markdownizer.install
   @@ -0,0 +1,138 @@
   + * Completely delete fields.
   

   The one-line summary must be written in the third person singular tense.

6. +++ b/sites/all/modules/drupalorg_markdownizer/drupalorg_markdownizer.install
   @@ -0,0 +1,138 @@
   + * @param array $field_names
   

   @param array is not explicit enough, as explained before.

7. +++ b/sites/all/modules/drupalorg_markdownizer/drupalorg_markdownizer.install
   @@ -0,0 +1,138 @@
   + * Attach existing fields into entity.
   

   One-line summary must be written in the third person singular tense.

8. +++ b/sites/all/modules/drupalorg_markdownizer/drupalorg_markdownizer.install
   @@ -0,0 +1,138 @@
   + * @param array $fields
   

   The type is not explicit enough.

Just want to note that while we're still debating, people have started rendering Markdown in README.md files already: http://cgit.drupalcode.org/purge/tree/README.md?h=8.x-3.x

@anavarre: People have been putting MarkDown in README.md for years - see comment #45 above.

The driving force behind this issue project to make this installed base of README.md usable on project pages.

As a maintainer in the Drupal Commerce ecosystem, this would be super helpful. We have to maintain a README in code and a project page. I don't care what we use to parse the Markdown, because any library should be OK. I've never had issues, except for when using some GitHub flavored Markdown-isms.

Also, the community could then contribute to a project's page and not rely on maintainer to manually make the edits.

IS updated, feel free to edit if I missed something

My 2 cents: I would probably keep the readme.md up to date, I think most of my projects have a very outdated project page, mainly because it is disconnected from the normal workflow.

For custom module for clients we used gitlab, and it least the readme contains useful info.

This should not be postponed on the markdown issue, we have a markdown filter that we can use, if needed we can always switch later.

Back to NW so the issue of linking can be addressed.

Agreed on not postponing this issue on the CommonMark discussion. I have written at least five or ten million words in Markdown, and 99.9% of what I've written would work exactly the same between the existing Markdown module and CommonMark. Plus, it's already in use in many places around the *.drupal.org ecosystem, and on thousands of Drupal sites.

All the main bits (bold, italic, headings, links, basic images, etc.) work exactly the same, and for almost all the more exotic formatting, I use HTML because no matter what markdown flavor you use, one library or another chokes on a code sample inside a list inside a table inside a table, or something to that effect. This is true even with CommonMark.

@Xano, I'm ready to fix all meta information, but after first commit (as additional patch). We definitely need to start from implementation and then fix issues.

If you, or anyone, have some blocking notes, related to the code architecture, please tell us about them.

Thanks to all, who want to make this possible ASAP.

@Xano, I'm ready to fix all meta information, but after first commit (as additional patch). We definitely need to start from implementation and then fix issues.

I am not sure what you mean exactly, but if you are referring to the documentation issues I raised, then then will need to be fixed before this can be committed. It's just as much a part of our QA process as security and performance are.

I've moved drupalorg_markdownizer as submodule for https://www.drupal.org/project/drupalorg project. Now, to apply a patch, we need to be at the sites/all/modules/drupalorg directory.

@Xano, I've tried to take into account your comments, please, when you'll have a free time, review this one.

Also, all of you know that I've used a Markdownizer module to implement this task. It was created by me earlier than a similar one by @markcarver. All that it does - provide a Drupal layer for Parsedown library. I've chosen the Parsedown because, at that moment, it had more stars on GitHub than any other library.

To make sure that markdown syntax will be parsed correctly, I've created a SimpleTest which could be easily extended.

When come a time when someone propose better library then it could be changed in a moment.

Guys, we need to move on!!!

Guys, we need to move on!!!

We're not all guys ;-)

Can you please provide an interdiff so we can more quickly review the changes that have been made since the previous patch?

I know that you'll ask me about it. Unfortunately, I cannot provide an interdiff because made a mistake previously. Initially I've created a module wrongly: initialized a new repo on d.org dev instance, create first commit to have a log, wrote custom code and create a patch. Now, I've moved "custom code" to the https://www.drupal.org/project/drupalorg prject and, from now, we will be able to have the interdiffs.

On September 6 I'll have a birthday. Maybe somebody apply a patch, test the functionality and implement it as a gift for me?

Seriously, how I can help to move this feature on?

Tagging this as d7dx (DX?) as definitely effects developer experience.

We should be trying to minimize frustrations such as these as a lot of developers maintain mirrors on github.

Love to help with whatever I can to get this moving :)

September is coming...

Unfortunately, we recently lost our dev server. I set up a fresh site with minimal configuration, but stopped when I saw the issues below. If you are still interested in helping, please let me know and I'll re-add your SSH account. Sorry for the inconvenience and delay in reviewing this.

$data = $repo->exec("show $value");

Executing shell commands on project node view is at best slow. And shell commands should never be added for general security; we also block those off with Suhosin whenever possible. (The Git servers can run different PHP configuration as needed.)

hook_drupalorg_package_release() was recently added to the packaging process, http://cgit.drupalcode.org/drupalorg/tree/drupalorg_project/plugins/rele.... That’s a good time to look for README.* files and put their contents into a field.

@drumm, I definitely want continue!

Please try out ssh BR0kEN@devwww.drupalsystems.org. The host has changed, it should now resolve to 140.211.169.87.

I forgot to mention the web nodes don’t have any access to the Git repositories on disk too. When packaging and other things needing the files happen, that’s generally a fresh checkout.

To trigger packaging, run drush release-package {tag|branch} {project name}, like drush release-package tag openatrium. To force it to run again, remove the packaged tarball from htdocs/files/projects/… to make it think packaging hasn’t happened. Currently in production, packaging is queued up in and run at 5 minute intervals; longer packaging tasks, like drush make for distributions can clog the queue, but usually for not more than 30 minutes.

There is also hook_versioncontrol_code_arrival() which happens on Git servers quickly after code is pushed.

Despite the current patches using @BR0kEN's https://www.drupal.org/project/markdownizer, I think both @gisle and I feel very strongly that the Drupal ecosystem (including d.o) would benefit greatly from an existing and familiar project/namespace (my https://www.drupal.org/project/commonmark module was really only ever developed as a POC anyway).

So, as of now, @gisle and I are both maintainers of https://www.drupal.org/project/markdown and will be merging https://www.drupal.org/project/commonmark into it.

I'm also postponing this issue on #2952616: Adopt CommonMark spec for Markdown files since this is ultimately the first step for harmonizing all markdown in the community.

Also semi-postponed on #2949200: Add drupalorg_versioncontrol_repo_changes hook and #2952435: Merge in the CommonMark project

Tangentially related, but I thought folks here would be interested: krakjoe, one of the php maintainers, has been working on a commonmark php extension:

http://docs.php.net/cmark

https://github.com/krakjoe/cmark

Not really something drupal.org can use because we cant php7 yet, but worth noting.

Yes, I've seen that and have re-opened #2180993: Add support for PECL cmark extension.

I also opened https://github.com/thephpleague/commonmark/issues/315 and commented on https://github.com/krakjoe/cmark/issues/1.

Whatever/however the PHP community standardizes on a PHP extension for CommonMark, I'll be ready.

FWIW, https://pecl.php.net/package/cmark has been created and I have added it to https://markdown.unicorn.fail/benchmarks

It is, indeed, much faster but lacks any sort of extensibility, see https://github.com/thephpleague/commonmark/issues/315#issuecomment-37779... and https://github.com/commonmark/cmark/pull/123

Of course, this would all be dependent on when d.o can php7 :D

Regardless, thephpleague/commonmark is still a viable option for php 5.6.5+ and we'd likely want to use it for extensibilty reasons anyway (at least until the native cmark can as well).

Not sure where this issue is at, but if it's helpful at all, I've been piping my module READMEs through https://matteobrusa.github.io/md-styler/. For example, at Site Quota Enforcer, click on the External documentation link, and you'll see how it gets rendered.

I thought about this today when I had to manually edit the description of a module to just re-create what was already written in markdown but in HTML.

It really feels outdated and backwards. Anything, from Github/Gitlab to almost every modern tool is able to read and parse a README.md file to create a welcoming page for projects.

Also, this encourages developers to create a single point for basic documentation and info instead of having several, often outdated and/or conflicting sources.

Surprised to see that this post is 11 years old...

For what is worth, +1 to having this. I guess tooling has evolved during these 11 years and standards on markdown have settled, so I think this could be re-considered.

Unrelated to drupal.org, but I would also happily extend this and make the hook_help() also read from the markdown file. I don't get what's the use of having three different sources of info (the README.md, the hook_help, the project description) that usually need to tell the same information: what's the project, how to use, how to install, requirements.

UI-wise, we could have a button that tries to fetch the contents and overwrites whatever is in the description (transforming markdown to html), all ajax driven.

That way we make it a one-off fetch operation and give the user the possibility to refresh when editing and we would not need an additional field on projects or do constant external requests.

Suggestion:

Yep, @flgarlin proposal could be a good solution and probably easier to get done.

I still think on the long run Drupal modules should move towards a single source for module documentation (and that single source should be fetched from elsewhere: drupal.org, hook_help, etc), but that's another discussion.

I'll work on it.

Yay, excited to see this moving forward!

In general, I'm all in favor of a single, canonical copy of something, instead of duplicate, out of sync versions. However, for something like a README.md file, the requirements, instructions, functionality, etc, might change across different branches. So long as it's a README.md file in Git, it all works great, and you can easily maintain the differences as needs change. But for a d.o (or any other) project page, you might need different information that's true across all branches.

The solution @fjgarlin came up with of an AJAX-driven import where you select a specific branch seems like a great compromise. If a project is simple, and everything can be said exactly once, yay. If it's complicated and needs different instructions in different branches, it can either not use this feature at all, or start with an import, then customize the results to be more general...

Sounds like this is no longer postponed, so removing the "[PP-3]" from the title.

Thanks again,
-Derek

This is ready to review.

See code changes here: https://git.drupalcode.org/project/drupalorg/-/merge_requests/157

Here is a demo to see how it looks and how it works: https://www.drupal.org/files/issues/2023-02-21/Screen%20Recording%202023...

Happy to address any feedback.

Feedback: As i read #80, this is about a button to copy a readme of some arbitrary branch to the description.

Sorry, but i can not see that this addresses the need addressed in the issue.
As a module maintainer i want to
- be "DRY" and not maintain a text in different locations, and most important, toolsets.
- to update stay in my toolset of code editor and git.

Stated differently, if i'm updating a module, i have it open in my IDE anyway. If i have to open drupal.org, copying the text from my IDE is simpler that hitting a button, and in the end i know it's the correct version.

As i understand this issue:
As a module maintainer, i want to update the project page by pushing updated file(s).

Agreed with #87.

Given that gitlab supports this out of the box, and that d.o is moving to use gitlab for project management, should this be closed as a "won't fix"?

(I did not read the whole discussion, just the IS, but) What about this:

As a module maintainer, i can put [tokens] into the description text that embed a project file (spcedified in the token from a branch specified in the token) as rendered HTML.

We can bikeshed about the details, but tha would address my need as multi-maintainer. E.g. if i want, i can add readme files of different branches, and wrap them in details tags.

That would work for me.

#88:
> and that d.o is moving to use gitlab for project management, should this be closed as a "won't fix"?

This depends if gitlab will be the main project frontend.
(Btw, there is an analogous issue for the description text of gitlab projects, the text that is displayed in project lists.)

d.o is moving to use gitlab for project management

Even tho many things are moving to gitlab (issues, CI...), there are many other parts of d.o projects that won't. Project pages will stay in d.o in the foreseable future as far as I know.

As discussed in earlier comments, not everybody might actually want to pull the info from readme files into the description (for different reasons), so this additional feature just gives them the possibility of a "quick" copy/paste into the body field.

My only concern with the tokens would be the performance as we'd need to make external requests to bring that in when displaying the project.

#89 could be good for me - or just links to the rendered readme at Gitlab.

However, to me the real problem is that the project pages often contain too much information. As already mentioned, we have doc guides and readmes for projects. The project page shouldn't repeat this (unless it's automatically maintained for example with tokens as suggest above in #89).

One idea: We already select a default branch. Used that information to select readme that is automatically added to the project page.

> My only concern with the tokens would be the performance as we'd need to make external requests to bring that in when displaying the project.

Page cache and/or caching in the token provider should be relatively simple. Invalidating on every commit also, as we have the build logic on commit anyway (and maybe we have cache tags by then ;-).

@fjgarlin: Amazing, thanks! Fantastic progress. 🎉

@everyone who thinks this is totally insufficient and wants something else: Please remember that d.o can be a slow moving site, change can be hard, and when someone makes a solution that helps (but doesn't totally solve) your use case, and still supports other people's use cases, I recommend leading with gratitude and appreciation, not complaints that it's not good enough. 😉

If comparing this to Gitlab, keep in mind that the UI works there since the "project home page" is itself a Git filesystem browser, including a branch filter with a pre-selected value, and a list of files/directories in the root of the repo. Not everyone browsing / using the project page necessarily wants/needs to interact with it as a Git repo, wants to see the codebase of that project (if any), or even understands branches and how to pick one. Yes, we can specify a default branch in the d.o project, so we could use that, but not every project page can always === the contents of the README.md on the default branch. Some projects have no code, and potentially no Git repo at all.

If we're going to automagically slurp in the README.md from the default branch on every commit to that branch or something, and cache that in the d.o DB, I'd vote we stick that into a separate field from the project body, and give maintainers the choice to show either the README, body, or both (but not neither 😅).

And, I think this is already great progress, could probably be committed and deployed ASAP, and we can open follow-ups to explore further improvements / changes. I just haven't looked at any of the code, so I can't RTBC this.

Thanks again,
-Derek

Thanks for the feedback. I'll try to work on those things soon.

Addressed some of the feedback and followed up with some questions on some of the comments.

The latest round of feedback has been addressed.

Adding a quick demo video to see how messages display, showing that the escaping is correct, etc: [video]

Looks fantastic, many thanks!

Great work @fjgarlin, it's nice to see this feature getting very close to being a reality.

This has been merged & deployed!