My makefile for Profiler example passes verification when I paste it into the verification tool, but when I try to create a release, I receive the following error:

The drupal-org.make file for project Profiler Example failed verification for CVS branch HEAD.

http://drupal.org/node/642116 -- to learn more about correcting these errors.

The 'core' attribute is required [error]

An error occurred at function : drush_drush_make_d_o_verify_makefile [error]

Once these errors are fixed, commit them to the branch, then resubmit the release.

My makefile is pretty simple, not sure what the problem is:

core = 6.16

; Profiles
projects[profiler] = 1.0

; Modules
projects[admin_menu] = 1.5
projects[advanced_help] = 1.2
projects[cck] = 2.6
projects[date] = 2.4
projects[features] = 1.0-beta6
projects[filefield] = 3.2
projects[globalredirect] = 1.2
projects[imageapi] = 1.6
projects[imagecache] = 2.0-beta10
projects[imagefield] = 3.2
projects[install_profile_api] = 2.1
projects[markdown] = 1.1
projects[pathauto] = 1.3
projects[token] = 1.12
projects[transliteration] = 2.1
projects[views] = 2.8
projects[wysiwyg] = 2.1
CommentFileSizeAuthor
#52 751242-3.patch748 bytesezra-g
#30 751242-2.patch3.86 KBhunmonk
#28 751242-1.patch3.09 KBhunmonk
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

q0rban’s picture

Title: Drush makefile verification is failing on release » Drush makefile verification should verify the filename
Category: bug » feature

Ok, I figured out that the name of the file was incorrect. My makefile wasn't entitled drupal-org.make. This really should be part of the error text IMO. Thanks!

q0rban’s picture

Priority: Normal » Critical

I think this justifies critical.

hunmonk’s picture

Priority: Critical » Normal
Status: Active » Postponed (maintainer needs more info)

i don't think it's a critical issue -- it's very clearly documented to name the file drupal-org.make at http://drupal.org/node/642116, which is referenced in the error message.

however, i'm curious to know how you even got to that section of the packaging script with a .make file named something other than drupal-org.make -- that should not be possible. can you send me a listing of all the files in your main profile directory at the time the error was occurring?

q0rban’s picture

"i'm curious to know how you even got to that section of the packaging script with a .make file named something other than drupal-org.make"

I guess that's why I was thinking it should be critical. It's very confusing the way it currently works b/c you have no idea what the real problem is (filename incorrect).

README.txt
profiler_example.make
profiler_example.profile
profiler_example.profiler.inc

q0rban’s picture

It's also very frustrating, b/c you can't make a release with a non drupal-org drush makefile, meaning if you just wanted to include a makefile along with your install profile, tough luck.

hunmonk’s picture

It's also very frustrating, b/c you can't make a release with a non drupal-org drush makefile, meaning if you just wanted to include a makefile along with your install profile, tough luck.

this is absolutely not the case, or if it is, it's a bug. :)

here's the relevant section of code in the packaging script:

    // In order for extended packaging to take place, the profile must have a
    // file named drupal-org.make in the main directory of their profile.
    $drupalorg_makefile = 'drupal-org.make';

    if (file_exists($drupalorg_makefile)) {
      // Search the .make file for the required 'core' attribute.
      $info = drupal_parse_info_file($drupalorg_makefile);
      ...the rest of the drupal.org-specific packaging logic...
    }
    else {
      wd_msg("No makefile for %profile profile -- skipping extended packaging.", array('%profile' => $release_file_id), $release_node_view_link);
    }

given that, i'm quite confused how you ever got a packaging error message in the first place if you didn't have a file in your profile named drupal-org.make.

in order to do anything about this issue, it would probably be necessary to have the problem happening again somewhere, because there's no obvious flaw in the code logic that i see.

q0rban’s picture

That is very odd. I'm not sure how to test this again without creating a dummy project and trying it all over again. It happened many many times, though. I kept trying thinking it just wasn't looking at the most recent commits or something.

nedjo’s picture

Status: Postponed (maintainer needs more info) » Active

I get the same error when trying to create a dev release for the Open Outreach profile:


The drupal-org.make file for project Open Outreach failed verification for CVS branch HEAD.

http://drupal.org/node/642116 -- to learn more about correcting these errors.

The 'core' attribute is required [error]
An error occurred at function : drush_drush_make_d_o_verify_makefile [error]

Once these errors are fixed, commit them to the branch, then resubmit the release.

Steps to reproduce (as drupal.org admin with ability to create releases):

* Visit http://drupal.org/node/add/project-release/831878
* Enter the following values:
* CVS Branch: Head
* Core compatibility: 6.x
* Major: 1
* Patch-level: x
* Extra identifier: dev
* Click Next

I don't want a drupal-org.make file, just the regular openoutreach.make file I've committed.

hunmonk, I'll delay making any changes to give a chance to look at this.

Thanks!

hunmonk’s picture

Status: Active » Postponed (maintainer needs more info)

i'm confused why you wouldn't just simply include a drupal-org.make file with the profile. even if you have to strip it down, there's no penalty in doing so.

nedjo’s picture

Status: Postponed (maintainer needs more info) » Active

The profile includes many external dependencies and breaks without them. So there's no point in including a drupal-org.make file. I want the release to include just the profile, not the profile packaged as a distribution. Then e.g. users can include the profile in a drush make file and get the full distro built. E.g.

core = 6.x

projects[] = drupal
projects[] = openoutreach

Am I right in understanding that if I include a drupal-org.make file a distro gets packaged for the release, not just the profile?

hunmonk’s picture

Am I right in understanding that if I include a drupal-org.make file a distro gets packaged for the release, not just the profile?

yes, that is correct.

this bug is not something i have time to troubleshoot right now, as i'm buried in work for the redesign. you're welcome to investigate for yourself and submit a patch -- the relevant code is in the 6.x-1.x-dev branch of drupalorg module, in a subfolder called 'project_verify_package'

q0rban’s picture

Title: Drush makefile verification should verify the filename » Install profiles cannot create releases with a non drupal-org makefile
Category: feature » bug

It's also very frustrating, b/c you can't make a release with a non drupal-org drush makefile, meaning if you just wanted to include a makefile along with your install profile, tough luck.

this is absolutely not the case, or if it is, it's a bug. :)

I just ran into this again. I renamed my drupal-org.make file to profiler_example.make and when trying to create a release based on that branch, the verification errors in the initial post display, blocking me from creating a release. Switching to a bug report.

q0rban’s picture

Priority: Normal » Critical

Bumping this to critical, as I can't make a stable release for my Install profile, even though it doesn't have a drupal-org.make file.

hunmonk’s picture

Priority: Critical » Normal

i don't see this as a critical bug. you can absolutely make a release by including a drupal-org.make file -- it's not ideal in your situation, but it's not preventing you from making releases.

apaderno’s picture

What happens when the file drupal-org.make is empty, or contains the minimum lines necessary to avoid the function returns errors?

q0rban’s picture

> What happens when the file drupal-org.make is empty, or contains the minimum lines necessary to avoid the function returns errors?

I think if it is empty, the release will yell at you that you need to specify a version of drupal. If I just specify a drupal-org.make file that contains minimum lines, my release will be broken and misleading. I literally /can't/ include a drupal-org.make file that will make a functional release, as Profiler needs to live at profiles/profiler_example/libraries/profiler, and drupal-org.make doesn't currently accept libraries[].

nedjo’s picture

I looked in _project_verify_package_verify_release_node() in project_verify_package.pages.inc and didn't find the source of the error. The code looks like it should only process a .make file with the drupal-org name (as defined in the PROJECT_VERIFY_PACKAGE_MAKE_FILE constant).

seanr’s picture

I ran into the same problem with ngpprofile (ngpprofile.make, ngpprofile.profile and rebuild.sh). Even if I only tag ngpprofile.profile and try to make a release, I still get the error which is absurd because I'm not even releasing a make file.

q0rban’s picture

Priority: Normal » Critical

I'm going to try to bump this back up to critical. It really is a critical bug. If I cvs rm all make files and try and tag and create a release, it fails.

dww’s picture

Ugh, sorry to hear all this. I hate to say it, but tomorrow is the Git launch and I'm kinda busy for the next few days... ;) I'll look into this early next week if no one else beats me to it, but I can't deal with this before then. Sorry/thanks! -Derek

dww’s picture

Heh, I ran into this with the 6.x-2.x series of drupalorg_testing since we did this:
#1045768: Remove drupal-org.make file

My current theory is that project_verify_package is constructing a URL to download the drupal-org.make file from gitweb, for example:

http://drupalcode.org/project/drupalorg_testing.git/blob_plain/master/dr...

Gitweb returns an empty file, but not an actual error. So, project_verify_package tries to parse the empty file and fails.

So, maybe all we need is to check if the drupal-org.make file is empty, and if so, skip this validation step (and don't try to do the fancy packaging in the packaging script).

hunmonk’s picture

Gitweb returns an empty file, but not an actual error.

i verified this with both wget and curl, gitweb does indeed return an empty file.

So, maybe all we need is to check if the drupal-org.make file is empty, and if so, skip this validation step (and don't try to do the fancy packaging in the packaging script).

we *could* do this, but wouldn't the best solution be to fix gitweb to behave properly? it should be returning a 404 or something of the like in these cases. it feels like a brittle fix to work around this deficiency instead of fixing it properly.

dww’s picture

We *could* try to change gitweb, but that's going to be a major PITA. So, we can leave this critical bug broken for another few weeks or months while we fight with gitweb, or we could add basically this:

  if (file_get_size($filename) == 0)) {
    return;
  }

and be done with it. Honestly, relying on gitweb at all is a brittle hack. Given that, I don't see why relying on a 404 is any more robust than checking the filesize.

dww’s picture

Hrmm. Upon further inspection, I appear to have crafted the wrong URL above. The thing project_verify_package is really using looks more like this:

http://drupalcode.org/project/drupalorg_testing.git/blob_plain/master:/d...

Which indeed results in a 500, not just an empty file. Here's the same URL but with a file that exists:

http://drupalcode.org/project/drupalorg_testing.git/blob_plain/master:/d...

and that happily works and gives you the file.

When testing this locally on my laptop, the 500 is triggering this else case:

      else {
        form_set_error('title', t("Pre-packaging verification failed -- unable to retrieve %makefile from %url", array('%makefile' => PROJECT_VERIFY_PACKAGE_MAKE_FILE, '%url' => $url)));
      }

That's itself a bug (since we shouldn't assume every project has a drupal-org.make) but that's not the error I'm hitting on d.o when trying to create a new release of drupalorg_testing. Weird. It's late now, so I'm going to have to go to sleep, but this issue clearly needs more investigation, both on and off d.o, to figure out all the ways this validation is not working when there's no drupal-org.make file...

Cheers,
-Derek

dww’s picture

Interesting. My laptop (which fails nicely) is using file_get_contents() while d.o (at least one of the redesign dev sites) is using curl. So, that explains the different behavior. We should try to teach project_verify_package_get_remote_file() to be more consistent about how it behaves, regardless of the underlying mechanism it's using to get the file contents...

nnewton’s picture

subscribe

hunmonk’s picture

Status: Active » Needs review
FileSize
3.09 KB

attached is a first attempt at intelligently handling the not found issue, and being a little smarter about when it's actually a server error.

not tested, i can move to that if the approach looks ok.

dww’s picture

Status: Needs review » Needs work

Excellent, that's much better. A few quibbles:

A) Generally we use ' for string literals, not ", unless you need " so that variables are expanded (or you're defining SQL, in which case we use ' to enclose the string literals in the query).

B) Returning an array is a bit wonky. Maybe passing in a &$contents by reference is cleaner? I dunno. Slightly sucky either way, but generally returning an array is painful. Either way, the PHPdoc for project_verify_package_get_remote_file() is now all wrong.

But yeah, otherwise, this is definitely what we need.

Thanks!
-Derek

hunmonk’s picture

Status: Needs work » Needs review
FileSize
3.86 KB

A) fixed.
B) array by reference feels even worse than returning an array. this pattern is used in drupal on occasion, and i'm comfortable with it here. doxygen has been cleaned up accordingly

anarcat’s picture

I think we're having an issue similar as this one: the hostmaster project doesn't have a drupal-org.make file, but has a hostmaster.make file. When we try to create a release node, it fails with the following error:

Unable to parse file [error]
An error occurred at function : drush_drush_make_d_o_verify_makefile [error]

The error is confusing because there is *no* file to parse (unless it's trying to parse the hostmaster.make file?)..

See also: http://drupal.org/node/1074876#comment-4204620

dww’s picture

Yup, same bug. I'd love to have time to review and test #30, but that's not really going to happen for me until next week...

anarcat’s picture

Any update here? We're going to head for a rc3 shortly and would love to make the release correctly. ;)

dww’s picture

Sorry, no update. :( I suck. At this point, the soonest I'll be able to review and test this will be next Wednesday (2011-04-20), so hopefully later next week we'll have this fixed and deployed. Apologies for the delays!

anarcat’s picture

No worries! We have a workaround here in the meantime: #1128596: version numbers in hostmaster project are wrong and we'll go ahead with our 1.0 release (*gasp*!).

janusman’s picture

Excuse me, but is packaging working at all on d.o?

Entering the single-line makefile:

core=6.19

on a "Verify drupal-org.make files" page (like http://drupal.org/node/1069814/verify-make-file) outputs:

Project information for drupal retrieved. [ok]
Unable to determine project type for drupal. [error]
Errors were found. [error]
An error occurred at function : drush_drush_make_d_o_verify_makefile [error]
Gábor Hojtsy’s picture

@janusman: no, unfortunately they are not working: #951978: "Unable to determine project type for drupal" when creating release.

anarcat’s picture

Sorry to be nagging here, but: ping! :) If I can be of any help with this, let me know.

I feel that #951978: "Unable to determine project type for drupal" when creating release would be fairly simple to fix if we just copy code from the latest drush make. That error is typical of an API problem between drush make and drupal.org's XML output...

mermentau’s picture

Subscribe

klonos’s picture

anarcat’s picture

No, I think this is a different issue. Here you cannot release your profile at all - and your ticket is about a library.

anarcat’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: +needs drupal.org deployment

I reviewed the patch, it looks sane. I do not have access to drupal.org to deploy/test this. If I would be given access through this process I guess I could test, but in the meantime, I'll mark this as RTBC and this make some kind of a ping here.

Can we please test this patch on drupal.org, pretty please, pretty please with sugar on top? ;)

drumm’s picture

Project: Drupal.org infrastructure » Drupal.org customizations
Version: » 6.x-3.x-dev
Component: Packaging » Code
drumm’s picture

Issue tags: -needs drupal.org deployment

Deployed.

dww’s picture

Sorry I haven't been able to deal with this. Life has gotten quite complicated for me these days, and I haven't had as much time for Drupal as I'd like.

@drumm: I don't see any commits pushed to Git. Can you explain what you did? Did you just commit this to the d.o bzr branch and deploy that?

Thanks for moving this forward!
-Derek

drumm’s picture

Sorry, I was marking things as deployed, and assumed this had been committed. Usually, I like to not use the tag until the initial commits are done.

drumm’s picture

The patch in #30 does look straightforward, so I can get it in the next deployment. It would be good to know someone is around #drupal-infrastructure for help in case it doesn't work great.

dww’s picture

Assigned: Unassigned » dww
Status: Reviewed & tested by the community » Fixed

I investigated bzr and found no evidence there, either. So, I went ahead and committed/pushed to Git, imported into bzr, merged into drupal.org branch, and deployed this for real. With a few other fixes, distro packaging is now working again!

anarcat’s picture

Sorry for misusing the tag, I'm new here. :)

Thanks dww for the hard work, you guys rock!

I'll test this with the 1.2 release of Aegir to see how it goes.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

ezra-g’s picture

Status: Closed (fixed) » Active

I'm getting this error trying to release the 6.x-2.4 tag of http://drupal.org/project/commons

Pre-packaging verification failed -- unable to retrieve drupal-org.make from http://drupalcode.org/project/commons.git/blob_plain/6.x-2.4:/drupal-org... not found

As other folks described in this issue, we're intentionally releasing without a drupal-org.make file so that we can package externally using Drush make.

Perhaps this was a side-effect of the recent (awesome) work on distribution blockers?

In any case, this is a blocker to creating a security update release of Commons that includes some of last week's contrib SAs.

ezra-g’s picture

Status: Active » Needs review
FileSize
748 bytes

Here's a patch that returns simply if the result for the make file is "not found" from project_verify_package_get_remote_file().

dww’s picture

Status: Needs review » Fixed

Sorry about that! This was indeed broken by recent changes:

commit 9ce40ac238079c443de9f5592aba89b3b491d93e
Author: Chad Phillips <chad@apartmentlines.com>
Date:   Fri Jan 13 17:07:45 2012 -0800

    profile release node verification should fail if the drupal-org.make file is not found.

Now reverted, tested, pushed, and deployed live. You should be all set for that 6.x-2.4 release now.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

  • Commit 6751c3a on 6.x-3.x, 7.x-3.x-dev authored by hunmonk, committed by dww:
    [#751242] Fixed bug that broke releases of profiles with no drupal-org....
  • Commit 9472142 on 6.x-3.x, 7.x-3.x-dev by dww:
    [#751242] by ezra-g, dww: Fixed regression for non drupal-org makefiles...