Dev releases not packaged?

Is this related to #2100727: Increase frequency of dev package runs?

This has made D.O. very unusable without git. Other fails from popular modules include...

http://ftp.drupal.org/files/projects/date-7.x-2.x-dev.tar.gz
http://ftp.drupal.org/files/projects/views-7.x-3.x-dev.tar.gz
http://ftp.drupal.org/files/projects/webform-7.x-3.x-dev.tar.gz
http://ftp.drupal.org/files/projects/pathauto-7.x-1.x-dev.tar.gz

Lots of others too:

http://ftp.drupal.org/files/projects/nodequeue-7.x-2.x-dev.tar.gz
http://ftp.drupal.org/files/projects/nodequeue-7.x-3.x-dev.tar.gz

-dev releases are currently being rebuilt ... working alphabetically, and there are a lot of packages, so it will take a while. :)

Noting that file_entity was affected as well.

I think what happened was a permissions change or something on util which caused packaging to fail. When packaging fails, cleanupFailedBuild() in drupalorg/drupalorg_project/plugins/release_packager/DrupalorgProjectPackageRelease.class.php is called and it removes any packages that exist.

Closed the following issues as duplicates. Perhaps a small PSA via Twitter would have been nice?

#2101331: Can't download any contrib dev versions
#2101473: Problem with FileDepot Module download links
#2101493: Views data export module: 404 for development releases files

A PSA would be nice, yes, but I first wanted to figure out what actually happened. Also, I didn't think about doing so.

So here is a summary:

1) dev releases are not considered fit for public consumption, for this reason the build script simply deletes files for which the build fails.

2) the dev release build process is currently quite unstable, as a lot of distros tend to fail and then the build process occassionally stops. It is then aborted after 4 hours.

3) when it stops at say, "m", all the projects from n-z don't get built and need to wait for the next run.

4) I've long wanted to be able to run the job more often per day (currently twice) in order to alleviate the problem somewhat. #2100727: Increase frequency of dev package runs

5) To this end it was neccessary to create another build job that would clean up after failed dev branch build jobs and regain diskspace.

6) Since the script has usually died, I am not easily able to get the exact directory it created, so I delete all the matching ones.

7) Usually, this shouldn' t be a problem since the packaging job is configured to not run in parallel.

8) I was assuming that this would also apply to any downstream projects.

9) This was not the case, this needs to be configured separately.

10) My best guess at what happened is:
- In the morning I enabled the cleanup script after testing it the previous night.
- I scheduled two extra runs so that any backlog might get caught up with.
- The first run didn't finish in time and was automatically aborted.
- The next run was probably started before the downstream cleanup project of the first had started.
- The cleanup script then deleted the first and the second build directory
- The 2nd build faild for all projects and took all the existing dev releases with it.

Dev releases are now being built at a rate of several tarballs per minute, but we are only at "field*".

New error: File field_group-7.x-1.x-dev.tar.gz?date=1380114329 is corrupt (wrong md5 checksum).

Still problems:
File filebrowser-7.x-2.x-dev.tar.gz?date=1380099648 is corrupt (wrong[error]
md5 checksum).

Same problem with video and videojs projects.

That is because the XML only gets rebuild when a successful packaging run occurs.

A work around for drush users:

drush dl git_deploy
drush en -y git_deploy
drush dl <project-name> --dev --package-handler=git_drupalorg

Substitute <project-name> with the contributed project you want to download.

Edit: Ignore this comment. Killes explained the reason in #7.

releases have been rebuilt.

Thanx.