Problem/Motivation

We have added versioned dependencies to Drupal 7. It relies on the info file having a version. If you use git clone then the version information is simply not there. Also, dev tarballs only have the branch information (7.x-3.x-dev) but not the latest tag (7.x-3.5). You cannot tell if Rules 7.x-2.x fulfills the requirement of Rules ( >= 2.3).

Proposed resolution

Before generating the version = line in the .info file, the packaging script will run git to find the latest tag, and use this to determine the version string. If we are exactly on a tag then we'll use that as the version. If not, then we add a .1-dev suffix. So dev tarballs will contain version = 7.x-3.5.1-dev instead of 7.x-3.x-dev, allowing us to fulfill dependencies like ( > 3.5 ).

To support site builders who use git clones of contributed projects in their sites, we will also add a mechanism to allow core to use a local version of git to determine the version string, as the packaging script does.

Remaining tasks

At this moment everything but check back in two hours ;)

User interface changes

Users will see very little change. They will see 7.x-2.x-dev less times and versioned dependencies will work.

API changes

We do not plan API changes. Unless something relies on

Original report by fago

I try to make use of a versioned dependency for modules relying on the entity API. However, Drupal says the version is incompatible with the version checked out directly via CVS/git, as there is no "version" entry in my .info file. According to [#542202] this is correct. It also says:

Because Drupal core uses a slightly different packaging process than contributed modules, the core modules have a version line predefined. This is an exception to the rule, not the model for contributed modules.

However, adding the version = VERSION line make it accept my CVS/Git version. So, is the documentation wrong and we should add version = VERSION or is this a bug?

Next, I was not able to define a versioned dependency compatible with a d.o dev snapshot containing the version string "version = "7.x-1.x-dev"" at all - I tried stuff like >= 1.x, >=1.x-dev, =7.x-1.x-dev, .. Nothing worked. So what should I declare to have compatibility with >=1.0-beta6 + but also the dev versions? E.g. I'd like (>=1.0-beta6, >=1.x-dev) to work.

Files: 
CommentFileSizeAuthor
#63 #1013302-db_update_impossible.png29.68 KBklonos
#64 1013302-db_update_impossible.png29.68 KBklonos
#44 Selection_010.png40.49 KBchx
#43 1013302_43.patch1.09 KBchx
PASSED: [[SimpleTest]]: [MySQL] 33,448 pass(es).
[ View ]
#42 1013302_42.patch965 byteschx
PASSED: [[SimpleTest]]: [MySQL] 33,450 pass(es).
[ View ]
#41 Selection_009.png20.16 KBchx
#38 1013302_pretty_regexp.patch974 byteschx
PASSED: [[SimpleTest]]: [MySQL] 33,446 pass(es).
[ View ]

Comments

You should probably be using git_deploy or cvs_deploy, since that information is added by the packaging script for modules on d.o. Those modules fill in the missing information from the git or CVS metadata if you're not using a downloaded package. The version = VERSION is just a method of keyword expansion for core modules, so that they all contain the default version.

hm, I don't think it is very practicable for developers requiring cvs|git_deploy in order to be able to enable a module at all (due to the versioned dependencies).

Version:7.x-dev» 8.x-dev
Priority:Normal» Major
Issue tags:+needs backport to D7

A similar problem occurs when you add a new dependency. People upgrading from an old version will not know to add and enable the newly-required module, and neither update.php nor admin/reports/status provide helpful information as to why the newly-upgraded module crashes.

To resolve both problems I find myself writing elaborate hook_requirements() tests that react to missing dependencies by either enabling the required module, or disabling the dependent module and redirecting to the module selection page with an appropriate drupal_set_message().

This works, but it strikes me as particularly messy.

See for example htmlmail_requirements()

And using Git Deployment is totally impractical until the following issue is resolved:

#1165694: Remove default nomask from file_scan_directory()

Title:versioned dependencies and dev versions?Versioned dependencies fail with git checkouts

Re-titling, this is a regression from D6, in that you used to only need cvs_deploy for update status. However I don't see it being fixed without moving git_deployment or at least some of it into core.

Priority:Major» Normal

Additionally, cvs_deploy was a contrib module, gitdeploy is a contrib module. This practice was never recommended or officially supported by Drupal core. Thus, there's no way this issue can be major.

Priority:Normal» Major

Sun, this is about not being able to install modules at all, D6 did not have versioned dependencies so it is a new bug. Update status is optional, installing modules is not.

As I could really need versioned dependencies now to prevent an entity-API update mess, I had another look at this and figured out that putting manually a version like version = 7.x-1.x-dev in the repo fixes things. Once the packaging system adds a second version line, it looks like the second line wins.

Thus, it looks like that way finally versioned dependencies would work - event with Git checkouts. Any thoughts whether doing so might cause troubles somewhere?

Can glip read tags? If yes (i think so -- it can read history for sure) then add glip to core and problem solved.

Edit: no time to properly tear glip apart but getTypeID has else if ($name == 'tag') return Git::OBJ_TAG; very promising.

Edit2: I think git_deploy has this already in _git_deploy_version_alter

Title:Versioned dependencies fail with git checkoutsVersioned dependencies fail with dev versions

grml, I don't know why it seemed to work previously, but now it doesn't. Maybe because of different PHP versions? (5.2 vs 5.3)

unmet dependency:
> Entity API (>=7.x-1.0-beta9) (incompatible with version 7.x-1.x-dev)

That would mean users that want to upgrade entity api to dev version, would have to update all modules depending on it also to dev versions, where the dependency could be removed after rolling release? Not really an option.

That way, dependencies are unusable :(

No, git is the unusable part because it does not have a hook firing post clone (guess what, bzr has). We can and will fix this with glip.

That's the one problem, the other one is that #11 appears even with packaged dev-snapshot releases.

subscribing

packaged dev should die a fiery death.

iirc there was a conscious decision to not support versioned dependencies for dev releases - in that it'd imply some kind of support for running dev releases.

But stopping git checkouts from being installed at all without a contrib module looks more of an oversight to me and we should move that part of git_deploy to core.

>iirc there was a conscious decision to not support versioned dependencies for dev releases - in that it'd imply some kind of support for running dev releases.

This is not about supporting dev releases, it is about being able to use them. If one cannot use them, #15 would be the more honest solution.

theres friction here right? if this is enabled or fixed, that friction to develop will disappear :)

some thoughts:
some hardcore developers use git to checkout modules for testing, its reasonable to support them in the core, not to wait until they finished a packaged release.

some motivation to those hard core devs/ module devs that porting low version to 8.x-dev, because they are appreciated and recognized by the core!

Title:Versioned dependencies fail with dev versionsVersioned dependencies fail with dev versions and git clones

This issue is stopping me from being able to install the services module alongside the latest git checkout of ctools. #1219980: services Requires: Chaos tools (>=7.x-1.0-beta1), yet it doesn't work with the latest 7.x-1.x git branch

I echo the sentiments of others here, in that git checkouts need to be supported. Often in a drush make file you will need to lock down versions of modules to git checkouts from specific dates. You can't be restricted to using official releases.

thanks for the comment #22, the best way is to work with git versions if your a developer,
if your a troll. then its another story ;))

imho, dont be affected if your strucked :))

Encountered #22's as well, makes that Drush can not "enable" as well, see #1212746-12: Services Menu.

Migrating git_deploy to core is not going to happen. The module depends on a fork of the Git PHP client (glip), and that's quite a very significant code base.

why does one could copy the git pull of drush?

It is really important to support dependencies but on the other side it seems to be that it's not possible to use or anyhow detect this with git checkouts in drupal itself.

What about making this checking optional so people using git can disable it.
With disableing it, i mean make it not validate but still show the information on the modules page.

What do you think about this?

The reality is that having even the major version in the checked-in info file is quite a lot of trouble, as it means that a 7.x and 8.x version of a module immediately diverge, even if there is no change to the code. So we end up maintaining multiple branches for no good reason.

The real answer here is probably to figure out some new way to inject version information (including major version information) into a module. Hmm. I don't know what it is. But I do hate our static versioning in the info file.

This one just bit us in Media. We wanted to specify that Media requires File entity 2.x but users of the dev release will now get warnings.

I fail to see what using a dev release has to do with this issue. If you are using a dev release, you just need to adjust your dependencies to depend on the dev release or higher. I'm pretty sure there are tests for this and that specifying a dev should work.

Because doing dependencies = file_entity (2.x) fails dev versions (7.x-2.x-dev) because 2.x-dev is actually 'less' than 2.x.

I've even tried (2.x, =2.x-dev) and it also fails.

@Dave Reid: I think (2.x, =2.x-dev) fails because when listing multiple dependecies, they are ANDed together, not ORed.

One way to fix this issue would be to introduce an OR syntax. For example: (>=2.0-beta1 or 2.x-dev). Beyond that though, while I think it's reasonable for 2.x-dev to always be considered less than 2.0-alpha1 (although in reality sometimes it's earlier and sometimes it's later, depending on when the person downloaded it, but that's the challenge in a version identifier whose contents changes), I don't understand at all why 2.x-dev fails a (2.x) check. If I want to force someone to a non-dev 2.x release, I can easily write (2.x, !=2.x-dev).

By the way, for anyone interested, the way we currently implemented the Media related dependency checks is Media 2.0-unstable2 now depends on file_entity (>1.99) (i.e., all 2.x including 2.x-dev) and Media Gallery 1.0-beta7 now depends on media (<1.99) (i.e., all 1.x, including 1.x-dev, but not 2.x-dev or 2.0-*). Ugly, yes.

Erm , you want (>= 2.x) or ( < 2.x) and you do NOT want to hijack this issue further with this.

Issue summary:View changes

Issue summary by chx

Issue summary:View changes

Expanded on the versions and dev tarballs

Issue summary:View changes

Re-wording proposed resolution to be a bit more clear.

Issue summary:View changes

Another sentence to clarify the problem being solved here.

chx and I spent about an hour discussing this issue in IRC tonight.

The version changes to the packaging script sound interesting, as long as they don't break Update Status, Drush, PIFR, and other tools that use and depend on this information. Being able to tell what version a dev release is between would be useful info, and not just for this issue.

Where I'm not as keen on is putting an exec() call to git inside Drupal core. I don't believe it's a good idea to couple our website software to the arbitrary version control tool that Drupal.org happens to be using at this time (imagine if we'd done this a year ago and coupled D7 to CVS, for example). That's why we created http://drupal.org/project/cvs_deploy and http://drupal.org/project/git_deploy in the first place, to de-couple these things, and support advanced use cases of people who want d.o clones of their projects.

I asked chx for information on why Git Deploy wasn't sufficient for this, and he said because it was "bloated" and that this would be an easy one-line-of-code way to find out the same information. He also said that we basically "beg" for people to clone their stuff from Git because of the "Version Control" tab on project pages and that this is a common thing that will come up, moreso than it did in CVS. The fact that 317 sites out of 550K run http://drupal.org/project/git_deploy I think blows this argument out of the water, however. (Interestingly, 1791 sites are running http://drupal.org/project/cvs_deploy still. But that's still 1791 of 550K.)

So basically, I'm not really convinced, and would prefer energy being dumped into fixing whatever the problems are with Git deploy instead. But I told chx to ask catch if this approach passed his sniff test. If it does, we need to make damned sure that it doesn't break on shared hosts and other places without a git client installed, but who may have Git clones thanks to local development environments.

Issue summary:View changes

Change from +1. -dev to -pl.

Issue summary:View changes

We will use 7.x-3.5.0-dev ChX

Status:Active» Needs review
StatusFileSize
new974 bytes
PASSED: [[SimpleTest]]: [MySQL] 33,446 pass(es).
[ View ]

So I agree with webchick it's not nice to have git-specific code hard-coded in common.inc. However when more projects are using versioned dependencies I also think we'll see a lot more people run into this (in fact I think what's happening now is contrib developers just aren't using versioned dependencies because they keep running into this issue and users aren't running git-deploy). Before we had versioned dependencies the only feature you missed out on in core was update status, but not being able to install modules at all is a lot more severe.

However, having that logic directly in common.inc makes me wince. Could we move this to a pluggable class, and ship core with a git implementation of it? That way if we ever do switch version control again we could potentially just add a new implementation and change the default. It also makes that available to other projects (like potentially git_deploy) if they need it.

generally speaking, i'm +1 on the patch, though with some qualifications. i haven't tested it directly, but i've been over the regex and consulted with chx on the approach.

@chx, the only case we may have missed on the dev release thing is if there a) are no tags in the repo at all or b) there are no tags on the major branch version. in the first case, it seems like this should be fine - it'll just miss on the match and so do nothing to further manipulate the string. in the second case, i think it would still just pick the most recent tag that could be found, which would be on a previous major version branch. so i think these cases are fine.

note also that this approach, being decoupled from the data provided by update status, could also be a bit fouled up if someone pushed a tag but didn't actually tie a release to it yet. we'd get a count back to a tag that doesn't have any real release associated with it yet. in practice, though, i don't think there's actually any problem with that.

git_deploy's inappropriateness for this task have nothing to do with "bloat." it's worthwhile because *not* having git deploy but having some things, even temporarily, cloned down with git can grind your site to a halt, and without particularly good explanatory messages as to why. if the call fails, it fails - not a big deal, beyond the fact that the dependency checking will fail. unless, of course, i'm missing something, as i'm operating under the assumption that the patch here is really only run when we do version checking, not when we do normal update status inquiries. given that it's part of drupal_system_listing(), it seems it might touch both. if that's the case, i'm more inclined to side with @webchick as this really does get into the domain of stuff that git_deploy really ought to take care of.

StatusFileSize
new20.16 KB

That patch does not work but if I move the code into system_system_info_alter it most certainly does.

StatusFileSize
new965 bytes
PASSED: [[SimpleTest]]: [MySQL] 33,450 pass(es).
[ View ]

OK, now we see how pretty it is :) do we want to add a new module , enabled by default? I feel like that's an easy way to plug this in. This patch (unlike the one in #38) works. It is in system_system_info_alter but it could be in any other system_info_alter.

StatusFileSize
new1.09 KB
PASSED: [[SimpleTest]]: [MySQL] 33,448 pass(es).
[ View ]

Cleanup, way more resilient. Changed format to 7.x-3.0-rc1.228-dev (it was 7.x-3.0-rc1-228-dev).

StatusFileSize
new40.49 KB

Here's another screenshot, this time with the 7.x-1.0-beta1.47-dev format also showing that subdirs won't find the .git in the parent dir. Not a big deal IMO.

RE: #39, #37 outlines that we're talking here of 0.005% of active sites who are using clones/checkouts from drupal.org as opposed to tarballs. That number's probably off, but probably not by like a factor of 100. And even if it were, we're still talking < 1% of sites.

If those sites installed Git Deploy module, then this problem goes away, IMO. It's pretty easy to add a pointer to Git Deploy on the Version Control tab if we're concerned about people not finding it. We could even hook into Update Status info and display a big-ass warning on the modules page if we know there are modules which should have a version and do not, in an effort to highlight this problem more for site builders.

So are we sure that this why contrib developers aren't using versioned dependencies? Or is it because dev releases aren't supported ala #30-36 (which is a much, much more common scenario)? Or is it even because of some far less nefarious reason, like "They haven't read the documentation at http://drupal.org/node/542202 to realize they exist."

Adding additional metadata to the packaging script to -dev releases to try and pinpoint which tag/version they're between would solve the main problem here, and would solve it regardless if people are using git clones or tarballs.

Sure we can do this in a contrib and point, I will talk to halstead about git deploy "lite". I have filed basically the same code against the packaging script under #1314180: Make dev release numbers useful. This would only solve it for dev tarballs, I fail to see what webchick means by "would solve it regardless if people are using git clones or tarballs.".

Sorry, I meant that the problem that bites more like 90% of sites is not knowing whether 7.x-2.x is > or < 7.x-2.3. And if we fixed this in the packaging script, then both the tarballs themselves, as well as Git Deploy, would have that information (I think?).

No, git clone is not affected by the packaging script. I have committed this code into git_deploy 7.x-2.x and I am working with eliza411 (IRL, yay!) to get a handbook page and a link from the version control tab to git deploy and the handbook page up.

When it's time to update docs and the text of the Version control tab, the unpublished handbook page is at http://drupal.org/node/1314752.

The proposed text for Project Git instructions is visible in context on git-dev.drupal.org ... I can make the changes to production when it's needed, but chx can, too.

Right after "Not working for you? See Troubleshooting Git clone." we thought the following sufficed:

Once it works, you need the Git deploy module. See Versioned dependencies and Git for an explanation.

Has there been any update on this issue?

I'm currently thinking of removing version requirements altogether in a module I'm working on, seeing as I otherwise cannot even enable it...

As catch wrote very nicely in #39:

So I agree with webchick it's not nice to have git-specific code hard-coded in common.inc. However when more projects are using versioned dependencies I also think we'll see a lot more people run into this (in fact I think what's happening now is contrib developers just aren't using versioned dependencies because they keep running into this issue and users aren't running git-deploy). Before we had versioned dependencies the only feature you missed out on in core was update status, but not being able to install modules at all is a lot more severe.

Bolded some bits for truth/emphasis.

Yes there was massive progress, git deploy is updated once i can get hold of dww we will add the same code to the packing script without patching core this will be fixed.

#1314180: Make dev release numbers useful is now fixed. Yay! So can we close this one, or at least reduce the priority?

Well, if it works now. :) Let's find out!

I will work eliza411 to get a new version of the "Version Control" out strongly urging using git deploy. Then we can close this.

I confirmed that I can now use a module with a versioned dependency both with a dev build and the latest HEAD from git. It seems to work without git deploy, because it understands the 7.11-dev version string that is there? Am I mistaken?

Priority:Major» Normal

Ok, well if all we're missing is some instructions on the version control tab to encourage the use of a module that people already know about, this doesn't get to block D8 features anymore. :) This should probably also be moved entirely to another queue, but I'm not sure which one so I'll leave it here for now.

Status:Needs review» Fixed

I published http://drupal.org/node/1314752 and updated the version control instructions on drupal.org for maintainers and non-maintainers of full projects. I believe this should be closed now. If I'm mistaken, please reopen.

Status:Fixed» Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

XREF to #1247476: drupal_parse_dependency fails on major version ops (>7.x) and (<=8.x) ... not sure whether some remarks made here fits to it.

Component:base system» install system
Status:Closed (fixed)» Active

There's still a pain point here - git_deploy doesn't work when doing db updates (see #1558158: wordpress_migrate.info "dependencies[] = migrate (>2.3)" not correctly detecting latest migrate.). In my situation, I have wordpress_migrate dependent on migrate (>2.3), and a git clone of migrate. This is just fine on the modules page, but after adding an update function to wordpress_migrate, update.php won't let me run it: "Migration from WordPress requires this module and version. Currently using Migrate version". The problem is that the system info data is fetched before doing the full bootstrap, so git_deploy's system_info_alter() hook doesn't get called.

Can we get to full bootstrap sooner? Or is it even worth it to accommodate this scenario?

How is the system info data fetched before the full bootstrap? Does adding function git_deploy_boot() {} help?

This feels very familiar, I think I've seen this someplace before... Basically, an error causes hooks to get cached prematurely:

drupal_bootstrap(DRUPAL_BOOTSTRAP_SESSION)
drupal_session_initialize()
drupal_anonymous_user()
ip_address()
$ip_address = $_SERVER['REMOTE_ADDR']; // Note that REMOTE_ADDR is not present
_drupal_error_handler()
_drupal_error_handler_real()
_drupal_log_error()
watchdog()
module_implements()
drupal_alter()

The hooks get cached before we get to the drupal_alter('system_info', ...) and git_deploy never gets a peek at it.

No, implementing git_deploy_boot() doesn't get it in there early enough, must be before bootstrap_invoke_all()...

We can fix the narrow issue simply by checking the existence of $_SERVER['REMOTE_ADDR']. A more general solution would be... watchdog() forcing a full bootstrap before invoking its hooks? Otherwise, contrib modules don't get a shot at catching early watchdog messages...

#319844: Watchdog should work, even if the error is too early, before module.inc is loaded appears related, although in that case (which seems to have started with D6) it was erroring out completely.

StatusFileSize
new29.68 KB

I'm mostly using dev versions. The main reason being that they include latest patches. Here's the situation today where I simply cannot run db update:

db update impossible

StatusFileSize
new29.68 KB

...hmm

Status:Active» Closed (works as designed)

If you use dev tarballs , they now contain something like this:

; Information added by drupal.org packaging script on 2013-06-27
version = "7.22+68-dev"

If you use git and are bothered by this please take the code from git deploy (or the patch in this issue) , put it into a script and use it as a post-checkout git hook script. The git deploy queue would be a good place for such a script.

I use dev versions and still got #63 in D7 dev. Perhaps #1405212-16: Requirements problem (Unresolved dependency, System Version >=7.12 required) is why but I cannot tell since I had to comment out all the offending dependencies[] = entries from the .info files of complaining modules in order to work around not being able to run update.php.

Status:Closed (works as designed)» Active

This is indeed still a problem. Unbelievable :(

I also can't believe so few people are apparently using git_deploy. And probably most that do use drush updatedb.

#65 is not an actual solution, the git_deploy module should be able to inject its version information — see #62.

Finally! The whole "works as designed" status in this issue really started getting on my nerves.

Issue summary:View changes

.1-dev instead. It doesnt matter.