I tagged and created a release for spark 8.x-1.0 alpha3 last night, at http://drupal.org/node/1827460

The package is failing with the following:

Packaging error messages

ERROR: /usr/bin/php /var/www/drupal.org/tools/drush5/drush.php --no-cache --include=/var/www/drupal.org/tools/drupalorg_drush make --drupal-org=core --drupal-org-build-root=/var/tmp/tag.27613/spark --drupal-org-log-errors-to-file --drupal-org-log-package-metadata=metadata-core.json /var/tmp/tag.27613/spark/spark/drupal-org-core.make spark-8.x-1.0-alpha3 failed with status 1
ERROR: Build for spark-8.x-1.0-alpha3 failed.

I can't confirm these fails locally using the --drupal-org argument.

I asked about this in #drupal-infrastructure today, and dww thought it was maybe because we don't have an up-to-date version of Drush? #1301978: Upgrade Drush to v5 on all servers Though I'm not sure, cos unless the command output there is lying, it seems to be running drush5 (not sure exactly which 5 version though, to be able to test... apparently elsewhere on d.o we're running 7.x-4.5, though).

dww also suggested it might have something to do with applying patches, but we've been applying patches in Spark since the beginning, and never had a problem with it until now. But for reference, here are our make files:

http://drupalcode.org/project/spark.git/blob_plain/refs/tags/8.x-1.0-alp...
http://drupalcode.org/project/spark.git/blob_plain/refs/tags/8.x-1.0-alp...

I don't see anything terribly different in there from alpha2, which packaged just fine:

http://drupalcode.org/project/spark.git/blob_plain/refs/tags/8.x-1.0-alp...
http://drupalcode.org/project/spark.git/blob_plain/refs/tags/8.x-1.0-alp...

The only other thing that I saw in the issue queue that looked a little similar was #1793426: 0 sites reported usage for a distribution packaged on Drupal.org, but that seems to be specific to a checksum error, which we're not getting here.

Any help on this would be appreciated, as it's currently blocking people from testing D8 functionality we're trying to get into Drupal core by feature freeze. :(

Comments

webchick’s picture

Oh, and as a basic troubleshooting measure (since at #1472052: Packaging issue for distribution while using dev releases it mentioned this might be intermittent), I did try deleting/re-creating the release, but no cigar.

I also find it a bit strange, that the output of the errors over in that issue contained more stuff after "failed with status 1" but this one does not. Hm.

Anyway, I am stuck. :(

greggles’s picture

At least on util the file /var/www/drupal.org/tools/drush5/drush.php exists and is a Drush 5.4 installation which seems pretty recent.

Did you try building this locally using a similar command?

greggles’s picture

Ahem...

I can't confirm these fails locally using the --drupal-org argument.

So, maybe see if your version is different than the 5.4 and try again. Otherwise I don't have other ideas :/

webchick’s picture

Title: spark 8.x-1.0-alpha3 distro failing to package » Failed patches output as cryptic "failed with status 1" without further detail from Drush in distro packaging
Project: Drupal.org infrastructure » Project
Version: » 6.x-1.x-dev
Component: Packaging » Packages

Ok, thanks! Drush 5.4 helps narrow it down.

So I did git checkout 7.x-5.4 from my ~/drush directory, which normally points at the master branch.

Then I did:

$ drush make --drupal-org=core drupal-org-core.make foo
drupal cloned from http://git.drupal.org/project/drupal.git.         [ok]
Checked out revision 3b4548f90a192bac1226bef1cabeaf7945e6bb1f.       [ok]
Unable to patch drupal with 1137920_responsive-toolbar_184.patch.    [error]
drupal patched with set-theme-for-layout.patch.                      [ok]
drupal patched with region-module-7.patch.                           [ok]
drupal patched with pages-28.patch.                                  [ok]
drupal patched with grids-in-core-18.patch.                          [ok]
drupal patched with rlayout-1.patch.                                 [ok]
Generated PATCHES.txt file for drupal                                [ok]

Ah-ha! Well that'll do it.

Had that shown up in the output instead of a cryptic "status 1" I'm sure I could've worked that out myself. Moving to the Project queue.

webchick’s picture

Incidentally, I believe the reason this failed is because that patch was done as git format-patch rather than a traditional git diff. Fortunately, looks like later versions of Drush don't have this problem. But let me know if that's another packaging system/infrastructure bug/feature request I should file.

jessebeach’s picture

I think the problem is git am and binary data. The patch that failed had a deleted PNG file in it. The file was included in the patch as encoded, changing the file format from UTF-8 to binary. git am failed to apply the patch with the deleted PNG. The problem doesn't exist with git apply. I rerolled the patch without the deleted file and posted that to #1137920: Fix toolbar on small screen sizes and redesign toolbar for desktop.

jbeach-macbook-2:d8 jbeach$ git am ~/code/drupal/patches/d8/1137920_responsive-toolbar_205-d.patch
tr: Illegal byte sequence

Are we using git am to apply patches with the build system? I don't think that's possible because any patch created with git diff would fail with the message Patch format detection failed. This message occurs when the patch is created with git diff instead of git format-patch and applied with git am.

If we're using git apply to apply patches, then I'm not sure why it failed. It works fine for me on the original patch that failed

jbeach-macbook-2:d8 jbeach$ git apply ~/code/drupal/patches/d8/1137920_responsive-toolbar_184.patch 
jbeach-macbook-2:d8 jbeach$ git st
# On branch 8.x
# Changes not staged for commit:
#   (use "git add/rm <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#	modified:   core/modules/shortcut/shortcut.base.css
#	modified:   core/modules/shortcut/shortcut.module
#	modified:   core/modules/shortcut/shortcut.theme.css
#	modified:   core/modules/system/system.module
#	modified:   core/modules/toolbar/templates/toolbar.tpl.php
#	deleted:    core/modules/toolbar/toolbar-rtl.css
#	deleted:    core/modules/toolbar/toolbar.css
#	modified:   core/modules/toolbar/toolbar.info
#	deleted:    core/modules/toolbar/toolbar.js
#	modified:   core/modules/toolbar/toolbar.module
#	deleted:    core/modules/toolbar/toolbar.png
#	modified:   core/modules/user/user.css
#	modified:   core/modules/user/user.module
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#	core/modules/toolbar/config/
#	core/modules/toolbar/css/
#	core/modules/toolbar/js/
no changes added to commit (use "git add" and/or "git commit -a")

Whereas git am fails like the build process did:

jbeach-macbook-2:d8 jbeach$ git am ~/code/drupal/patches/d8/1137920_responsive-toolbar_184.patch 
Applying: Issue #1137920 by jessebeach, kathryn531: Toolbar completely broken for small screen
error: patch failed: core/modules/toolbar/toolbar.png:1
error: core/modules/toolbar/toolbar.png: patch does not apply
Patch failed at 0001 Issue #1137920 by jessebeach, kathryn531: Toolbar completely broken for small screen

We might just need to go in and delete that PNG file by hand instead of by patch.

jessebeach’s picture

Here's the bit with the messed up data:

diff --git a/core/modules/toolbar/toolbar.png b/core/modules/toolbar/toolbar.png
deleted file mode 100644
index f2c7f35..0000000
--- a/core/modules/toolbar/toolbar.png
+++ /dev/null
@@ -1,4 +0,0 @@
-<89>PNG
-^Z
-^@^@^@^MIHDR^@^@^@d^@^@^@^C^@^@^@<F7>,~O^@^@^@<B7>PLTE^@^@^@<FF><FF><FF>fff<FF><FF><FF>fff<FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF>fffffffff^@^@^@^A^A
^A^B^B^B^C^C^C^D^D^D^H^H^H===>>>GGGHHHPPPQQQXXXfff<80><80><80><82><82><82><84><84><84><85><85><85><87><87><87><88><88><88><8A><8A><8A><8C><8C><8C><8D><8D><8D><8F><8F><8F><90><90><90>
<92><92><92><94><94><94><95><95><95><97><97><97><98><98><98><9A><9A><9A><9C><9C><9C><9D><9D><9D><9F><9F><9F><A7><A7><A7><AE><AE><AE><AF><AF><AF><B7><B7><B7><B8><B8><B8><C1><C1><C1>
<C2><C2><C2><F7><F7><F7><FB><FB><FB><FC><FC><FC><FD><FD><FD><FE><FE><FE><FF><FF><FF><A5>s^D<D8>^@^@^@^NtRNS^@^@^P^]<80><88><8B><8E><91><95><99><D0><E0><F0>O<BC>^z^@^@^A^XIDATx<DA><ED><D3>[O<C2>@^P<86><E1>o<B0>
-^T<A8>^HV^N<A3><A2><82><E7>^S^T^Q<C4><F9><FF><BF>K7<8D>1i<86>^P<92><U+E10B>}.<F6>f.<DE><DD>I^V<EF>^F<F0>f^@<AF>^F<F0>b^@<CF>^F<F0>d^@<8F>^F<F0>`^@<F7>^Fpg^@<B7>^F01<80><B1>^A<DC>^X
<C0><B5>^A\^Y<C0><C8>^@.^M<E0><C2>^@<CE>^M^@<88><9A>\ԌP<EA><A0><D2>`M<A3>R<E2>^@^Q<EB><A2>^R^Gh<B1><AE>U<E2>^@l DB<C4>(<D2>gŰ<9B>8<DD>!+2VL<97><E2>,<A7>Z$MNY1<E8><FC>6:^CV,<E4><93>^U
<B3><95><88><AC>f<DA>Kz<ED><E4><F8><8C><8B><F2>A<BB>NJ<F9>Z<BE>?6^L<D6>se]<F9><85>Oԍ<A5>I<BA><F1><C2>_<99><FE><C4>^E^W"[W<DF>ߺ<FA><A2>,<FC><93>^P     <91>}<8C>8D<D8>^A<ED>^L<CE><C1>!y
<8F>P,G<E4>9BU^Q<A9><92><D7>^H<D5>ĩ<91><C7>^H<D5>%W'^?<91>X<FE><C4><FE>"<F2><CF>_<C4>U<90><9F>!<B2>^_<91>^_<C0>N<*<D9>"<85><F0>^@^@^@^@IEND<AE>B`<82>
\ No newline at end of file
dww’s picture

Is this issue about debugging the specific packaging problem with spark, or is it about the thing in the title about the cryptic error messages?

If it's about the specific problem, I believe it's a bug in drush make that's been fixed and just needs to be deployed, in which case this should be in the infra queue and perhaps just marked duplicate with #1738358: Upgrade drush used for distro packaging to 5.7+ to avoid bugs in drush make.

If it's about the general problem of the sucky error propagation between drush make and end users, it can stay here in the Project queue, but then let's not get lost in the weeds about the specific problems with the patch that's failing to apply. ;)

Sorry for the troubles. Sadly, d.o distro packaging isn't as high a priority for me right now as the D7 port of d.o...

jessebeach’s picture

@dww, I assume this is a general issue with patches that include deleted binary files. I'm trying to understand why it failed. If the make system is using git apply, then the patch with the deleted PNG should not have failed. But it did fail to build. I assume we're not using git am because any patch created with git diff would fail to apply. So, I'm just confused and trying to put as much data around the issue as possible.

dww’s picture

Cool, thanks. If you suspect this is a bug in how drush make applies patches, you'll have much better luck with an issue in the drush issue queue. Sounds like this is probably a different bug from #1528422: Patches no longer apply because of info file re-writing (which is linked from #1738358: Upgrade drush used for distro packaging to 5.7+ to avoid bugs in drush make) so maybe we should just move this issue there for now to get the drush maintainers eye's on this. I honestly don't have the bandwidth to follow drush development closely, so I have no idea what it's currently doing to apply patches.

There's probably still value in a project.module bug report about improving error propagation, but since this issue is already full of useful info about this specific packaging problem, maybe we should just start a new clean issue about the error propagation and leave this one for figuring out why drush make can't apply this patch. ;)

Cheers,
-Derek

jessebeach’s picture

Cool, I'm fine with moving the conversation over to #1738358 as a first step in attempting to resolve the build failures. I've made a note there and linked back to this issue.

#1738358-6: Upgrade drush used for distro packaging to 5.7+ to avoid bugs in drush make

I'm totally agnostic about tracking the error propagation in this issue or a new one. I'd probably just use this one since it's here already.

dww’s picture

Category: bug » task
Priority: Major » Normal

spark 8.x-1.0-alpha4 is built now that #1738358: Upgrade drush used for distro packaging to 5.7+ to avoid bugs in drush make is done.

Do we still care about trying to do a better job propagating errors here?

Thanks,
-Derek

webchick’s picture

I would say yes, because then people can solve their own problems without pinging busy d.o D7 sprinters. :D

dww’s picture

Project: Project » Drupal.org customizations
Version: 6.x-1.x-dev » 6.x-3.x-dev
Component: Packages » Code

Okay, sure. Technically, all the distro packaging code is a d.o-specific plugin from drupalorg_project. Moving to a more appropriate queue.

I doubt anything's going to happen here until after the D7 launch of d.o, but we'll see what happens.

Cheers,
-Derek

drumm’s picture

Status: Active » Closed (cannot reproduce)

Seems like this hasn't been a problem lately. If it is still an issue, #1854444: Upgrade drush used for distro packaging with drush make has a good chance of improving it.