The newly ported package-release-nodes.php script uses a sophisticated query to determine if a package *might* be considered for being repackaged. While we probably won't get rid of the query, we can make stuff more clean and robust by not checking the mtime of exported files for triggering a repackage, but rather remember when changes are committed to a project. versioncontrol_project already retrieves affected projects on hook_versioncontrol_operation(), we just need a way to relay that information to versioncontrol_release.

The project_release module provides a nice table column named {project_release_nodes}.rebuild which formerly specified whether the CVS "tag" referred to a branch (rebuild = 1) or a tag (rebuild = 0). We don't use that anymore currently, since we're now using {versioncontrol_labels}.type instead for retrieving if a release. So I thought, let's reuse the rebuild field and fill it with values that actually indicate whether the release tarball should be rebuilt *just on the next* run of the packaging script.

Let's have a little example to demonstrate what I mean:

  • User creates a release for branch DRUPAL-6--1. The "rebuild" property is set to one.
  • Packaging script runs, selects the new release for packaging, makes a tarball out of it, and sets "rebuild" to 0.
  • Packaging script runs again (cron), and doesn't even consider the release for repackaging as "rebuild" is 0.
  • Consequently, no rebuilds will happen as long as no commits happen to that branch. That saves us about a million cycles of "cvs export" & comparing exported file timestamps with {package_release_file}. As a bonus, we also can skip (left) joining {project_release_file} and {file} into the SELECT query.
  • User commits into DRUPAL-6--1. versioncontrol_release takes note of that commit, and sets "rebuild" to 1.
  • Packaging runs again (cron), and repackages the release because "rebuild" is 1 again. Makes a tarball, and resets it back to 0.
  • Continue ad infinitum.

Actually, I currently prefer not to reuse the "rebuild" column from {project_release_nodes} because it's a branch/tag/packaging-script specific property and thus should go directly into {versioncontrol_release_labels}. Let's leave it to cvs.module to store its VCS specific data in other modules' tables.

Comments

jpetso’s picture

jpetso commented

One thing I forgot: in the above example, the only difference for "tag" labels would be that any commit to the tag (ok, it's not possible to commit to tags anyways in most version control systems) would just be ignored and "rebuild" would not be set to 1 once a release tarball is built. Even if stuff like SVN makes it possible to commit into tags after creating them, not rebuilding the tarballs ensures that all downloaders always get the same file for the same version number. (If developers want to correct some bug, they'd better be rolling a subsequent patch release.)

dww’s picture

dww
we/he/they
commented

This would still be a win. It'd also solve the kinds of problems discussed at #1331390: Packaging of dev tarball based on commit date instead of push. -dev rebuilding should be triggered on push, not trying to do datestamp computations via polling.

xano’s picture

xano
he/they
Primary language English
Location Southampton
commented

Currently DrupalorgProjectPackageRelease::createPackage() decides whether or not to re-package a dev release based on the latest release's datetime and the datetime of the files to be packages. This worked well for CVS, but as @rfay pointed out, committing a change to a Git repository does not mean the change is directly available to be packaged:
1) Change A is committed locally.
2) Change A is pushed.
3) Change B is committed.
4) Because of change A, the files are newer than the latest dev release, so a new dev package is built that contains change A, but not change B.
5) Change B is pushed.
6) During the next cron run, no new dev package is built, because the last package is newer than change B.

Instead, we need a solution that is triggered the moment a change is pushed to the server. versioncontrol_git.module offers hook_versioncontrol_git_refs_update(). We can use this to flag a module for a dev rebuild. For Drupal 7 we can use the queue, but for Drupal 6 we probably need a separate table to store these flags in.

(Shameless copypaste from my comment in #1331390: Packaging of dev tarball based on commit date instead of push)

rfay’s picture

rfay
he,him
Primary language English
Location Palisade, CO, USA
commented
Title: Mark project releases as "to be updated" only if commits happen there » Mark project releases as "to be updated" only if pushes happen there

Retitling to cover the issue closed-as-duplicate #1331390: Packaging of dev tarball based on commit date instead of push, which mentions the fact that currently rerolls are based on commit date, which causes releases not to be rolled sometimes when they should be.

drumm’s picture

drumm
he/him
Location NY, US
commented
Status: Active » Closed (duplicate)

#1878418: Queue release packaging at git code arrival does this in D7.