Problem/Motivation

Some things packaged on drupal.org aren't Drupal modules or themes. For them, the [core_version]-[major].[patch]-[extra] version strings don't make sense, since they're not bound to a specific version of core. The prime example is drush although lots of other scripts, add-ons, drush plugins, etc, are forced to either live elsewhere, or be subject to the core-compatible version strings which aren't appropriate for them (often leading to confusion for end users who think they need a specific version of core to use them).

Proposed resolution

  • Create a new project type on drupal.org (just via the taxonomy UI) called "Other" (or something)
  • Give all projects using this type a different version string format. The current proposal is:
    [major].[patch].(-[extra])?
    

    (e.g. "1.0", "2.0-beta1", etc).

  • Teach Project* and Git about these new version strings (see below for details)

Remaining tasks

User interface changes

- There will be a new project type, so we'll need a new listing page for those at the downloads section.
- New project type radio button on project add/edit pages
- Possible new admin setting UIs for #1434476 and #1434500

API changes

Nothing user-facing. There will probably be some changes internal to project_release and versioncontrol_release modules.

Original report by IT-Cru

// Text of original report here.
(for legacy issues whose initial post was not the issue summary)

Hello,

is it possible to create projects for scripts?

I'm currently working on a little automated drupal update bash-script, but this doesn't match with modules, themes, translation or so. A categorie script for new projects would be nice ;)

thx Gos77

CommentFileSizeAuthor
#24 OtherDownloads.jpg42.17 KBjuan_g
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

fgm’s picture

Side node: this category could also be relevant for drush-based scripts.

dww’s picture

Right. At DCDC, I ran into something like 3 different things that should really be in a separate script project type, although foolishly I didn't post an issue about it at the time. :( Drush itself should be one. cvs-release-history.php is another (currently just lives in "tricks", there's no issue queue, etc). I'm sure there were others, but I don't remember what they were. So yeah, we should probably do this.

However, see also #462078-1: New project type: database driver.

killes@www.drop.org’s picture

Maybe simply name it "Other" or "Misc" then that db driver would also fit there. But maybe we think too much inside the box:For most people "module" probably means "something I can extend Drupal with". If they aren't coders they probably do not think "has a .module file".

Maybe rename the category "modules" to "extensions"?

dww’s picture

http://drupal.org/project/drupal_php_ext is another clearly mis-categorized project. However, that's not a "script". Not sure the best way to handle that...

webchick’s picture

I'm with Gerhard: "Other." Stick theme engines here too, while we're at it.

moshe weitzman’s picture

drush could really use this.

dww’s picture

Yeah, drush was the initial use-case that prompted this whole approach. ;)

I'm -1 to folding theme engines in here, since those actually *are* tied to specific versions of core, so the Drupal core version element in the version string is useful and necessary.

So, what do we want for version strings for "other"? Just a standard major.minor.patch(-extra)? Or exactly like core: major.patch(-extra)?

anarcat’s picture

Aegir's Drush extension (provision) could also use this, as it's core-independant.

We are using "provision-0.4-rc1" for the current provision release here. We're going to be using "provision-0.4.0" for the first release (to avoid confusion with the branch). "provision-0.4" is the branch name. Once we head for 1.0, this will be something like "provision-1.0-rc1", "provision-1.0" and "provision-1" (for the branch).

I wouldn't mind stripping the "provision-" prefix from there.

I don't want to bug people too much with this, but it would be unfortunate for us if we would have to add the silly 6.x prefix to our provision module (and drush) as they are really core-independant. But I guess it's the *only* "con" in our list of "pros and cons" to switching back to drupal.org, so it's not a big blocker for us.

I don't really understand why we want to have such restrictions on branch names and tags in the first place anyways, but that's another thing... :P

dww’s picture

@anarcat: CVS has certain restrictions on what can even be in a branch or a tag name, hence the inability to have CVS tags that look like "6.x-1.0". Thankfully, Git has no such restriction, so there's no longer weird mapping. The "DRUPAL-" prefix is just insanity from how Drupal's been using CVS for the last 10 years, which I kept around when working on the migration to the "new" release system back in 2006.

Once Git goes live, there will be no restrictions on what your branches and tags are named. However, there will continue to be restrictions on the branches and tags that can be used for release nodes on d.o. That's because the version info has to be encoded in the tag name. The version has to be well-formed for things like update status (and by extension, drush) to work. If it was totally free-for-all version numbering, no one would have a clue WTF is going on, no program could make any sane attempt at telling you if you're missing an update, the project pages on d.o couldn't automatically point people to the right thing to download, etc, etc.

Anyway, yes, we need an "other" category of project that isn't tied to a specific version of core and has a different version numbering scheme. I'm leaning towards just doing it like core does: [major].[patch][-extra]. Any objections to that?

Cheers,
-Derek

anarcat’s picture

Thanks for the clarification. The only objection I have to [major].[patch][-extra] is that it doesn't cover our current use case of 0.[major].[patch][-extra] (ie. 0.4-rc1, 0.4.0-beta1 soon), so I would be more in favor of major.minor[.patch][-extra], with stuff in parenthesis optional. In fact, that would be:

\d+\.\d+(\.\d+(-[\w\d]+)?)?

... as a regex, I believe.

(As a aside, note that Debian (dpkg --compare-versions, actually) considers x.y-z to be a higher version than x.y, which is problematic if we want to package Drupal software in Debian. The "dash" in Drupal is really the equivalent of the ~ in Debian, which means "null", so that x.y~z is actually *smaller* than x.y, so you can upgrade from 1.0~alpha1 to 1.0 without problems. But I guess this is not the right space to debate that... :)

apaderno’s picture

[major].[patch][-extra] seems fine, to me.

Freso’s picture

I'm with #11.

Also, subscribing. =)

anarcat’s picture

Title: Contribute scripts as projects » Contribute scripts as core-version-independent projects (drush and similar)

I hereby retract my objection to [major].[patch][-extra], in case there was a feeling I was blocking something... :)

Otherwise what's blocking this from getting online? We are going to make a provision release soon and i would like to call it 0.4-rc1, not 6.x-0.4-rc1... ;)

jonhattan’s picture

I'd like to see two separate terms for drush: "integrates with drush" (ej backup_migate and many modules) or "drush extension" (ej drush_make, provision). At present they're all mixed in "drush".

fuzzy76’s picture

I have a munin plugin that logs both authenticated and anonymous users for a Drupal installation, just waiting for a place to flourish. :)

Gerhard Killesreiter’s picture

So, it seems we need a plan on how to move this forward?

Should we simply rename the "template engine" term to "other"? And then what?

rfay’s picture

Title: Contribute scripts as core-version-independent projects (drush and similar) » Package and version non-modules for download

Renaming. I think this is about packaging of things that are not modules and not themes, right?

This seems like a really important initiative. There are so many things that are not modules that belong here. A good example is the Grammar Parser "module" which is not a module at all, but rather a PHP library, which is forced to have a core version number in order to provide a downloadable package. Yuck.

scor’s picture

+1 for new project category "Other" or "Misc" and [major].[patch][-extra] like core does.

@Gerhard Killesreiter check #7 re theme engines.

lisarex’s picture

+1 for adding Other to the categories, or another word/phrase that better describes the contents. Could it also contain Courseware? (see #489392: New Project Type: Course?

We have several issues open that are related to not having an 'Other' category, e.g. #1239994: Change category for 'drupal.org project' sandboxes that shouldn't have it.

mlncn’s picture

Category: feature » bug
Status: Active » Needs review

#1154000: Create "Other" project type for projects that are not modules, themes, installation profiles, etc. is marked as duplicate, but envisions a very simple solution: Create an Other type. Deal with Theme engines later. I don't see an need for any changes to the packaging or versioning; it should work just like Modules which is where most non-module, non-module theme projects are ending up. And don't themes and theme engines and Drupal itself work the same?

#1166546: Projects are assigned to the wrong category meanwhile properly reports this as a bug. Projects are incorrectly listed, and currently without an "Other" categorization are necessarily incorrectly listed.

Proposal: Add "Other" to the "Module", "Theme", etc. list.

Edit: I understand the appeal of not being tied to Drupal version numbers but that can be a follow-up; projects are already set up with that now and it will take a bit of thought to fix that problem, while allowing for an Other category already has consensus.

fuzzy76’s picture

A lot of these "other" projects will be tied to specific Drupal API versions any way (my Munin plugin will, and Drush do too, to some extent). Who can nudge this issue further?

dww’s picture

Title: Package and version non-modules for download » META: Package and version non-modules for download
Category: bug » feature
Status: Needs review » Active

Given #1432296: Create a release node for Drupal.org Drush or provide documentation on usage and my desire to Do The Right Thing(tm), I just created a detailed issue summary for this, including links to sub-issues I just filed in the relevant projects that would need to happen for this to move forward. If anyone wants to work on any of them, please assign yourself. ;)

dww’s picture

juan_g’s picture

FileSize
42.17 KB

So, maybe we are going to something like:

Other Downloads

juan_g’s picture

Since some of them are core version independent, maybe an option to have simply "x.x-1.0" (or similar) instead of "7.x-1.0" would be enough for the generic ones.

dww’s picture

Re: #24: Yeah, something like that. Looks nice.

Re: #25: x.x-1.0 would require the same work (if not more) than the plan I outlined in the summary, and it's more confusing, so I don't see how that helps us at all. But thanks for brainstorming. ;)

Cheers,
-Derek

juan_g’s picture

It seems you are right, looking at the preg_match regexp in the patch for a drupalorg_versioncontrol plugin, from #1019244: Add a versioncontrol_release label_version_mapper plugin for d.o-specific mapping of Git branches and tags to d.o releases:

+      if (preg_match('/^((\d+)(\.(\d+))?\.x)-(\d+)\.(\d+)(-((unstable|alpha|beta|rc)\d+))?/', $tag_name, $matches)) {

So, core versions for contrib releases must start with a digit, and probably this is expected in more parts of drupal.org code. Therefore, options such as "x.x-1.0", "all-1.0", or similar, in the cases that are not "7.x-1.0", would need patches here and there, unless some convention such as "0.x-1.0" was used meaning compatibility with all core versions. As confusing as x.x, probably.

(But, surprisingly, Drush has releases such as All-versions-4.x-dev, etc.; not sure how that works, probably only for development releases).

Those options were just a possibility to have both core version independent ("all-1.0", etc.) and dependent ("7.x-1.0") projects in the same "Other" project type, with the same version string structure, that is including [core_version].

However, if most projects in "Other" are going to be core version independent, then your proposal seems better. And the few dependent ones can always include notes such as "Theme engine Smarty 2.0 works with Drupal 6.x". Probably there will be few of these, if features, apps, etc. (now modules) are not going to be under "Other".

dww’s picture

Features and apps are definitely modules. Those have no business being in the "Other" category. If we need a separate category for features, so be it, but that's a totally different issue.

This issue has always been about a way to package stuff with version strings that aren't tied to core, which is exactly what my proposal addresses.

Cheers,
-Derek

juan_g’s picture

Agreed then, it seems a reasonable plan, and the main candidates for this new "Other" project type are largely core version agnostic (scripts, plugins, libraries, etc.). For example, Drush, Drupal.org drush, Ctools Plugin Stub, Provision, Barracuda, Octopus, Grammar Parser...

RobLoach’s picture

Closed #1893436: Introduce a Drush Extension project type on Drupal.org as a duplicate... Would love to use a 1.0.4-alpha1 versioning convention for some of these projects that arn't tailor to a specific version of Drupal/Drush.

fuzzy76’s picture

Seems like Drush (rightfully) got tired of waiting for this… http://www.mosheweitzman.me/post/58712856560/drush6-announcement

fuzzy76’s picture

Issue summary: View changes

Created a full summary of the roadmap for getting this done, including links to sub-tasks

ergonlogic’s picture

Since we're working on re-factoring infrastructure components to support semantic versioning in core and contrib, it'd be nice to tackle this one too. So I'd propose we look at the following tagging format:

[major](.[minor]).[patch].(-[extra])?

So, basically support:

  • 1.2
  • 1.2-alpha3
  • 1.2.3
  • 1.2.3-beta4

Would that be sufficiently flexible for most "other" projects?

th_tushar’s picture

Status: Active » Closed (works as designed)

Please re-open if necessary. Closing due to inactivity.

fuzzy76’s picture

Status: Closed (works as designed) » Active

Sorry, but closing an issue with active subtasks and a lot of agreement is not very helpful.

apaderno’s picture

Project: Drupal.org site moderators » Drupal.org customizations
Version: » 7.x-3.x-dev
Component: Site organization » Miscellaneous
drumm’s picture

Status: Active » Closed (duplicate)
Related issues: +#2474007: Add a “General” project type to Drupal.org
fuzzy76’s picture

@drumm this issue talks about shell scripts, commandline tools and monitoring apps... Are you sure that a "library" node type is a duplicate of this?

apaderno’s picture

Despite the title, the other issues is not merely about libraries. (See #36.=

This is for code, in any language, that does not currently fit the 'module', 'theme', 'distribution', or the seldom used 'templating engine'. It is also for subtree splits of core, and new code not hosted elsewhere.

The same comment also says:

This is not intended to be used so that code, that exists somewhere else, can be hosted on drupal.org so that its 'easier' for end users install process or anything like that.