Drush make-generate (generate-makefile) isn't working

I'm using the latest drush make beta 10, seeing these errors all the time.

I confirm this problem

Sub

Apologies for my ignorance. I'm experiencing the same problem and believe the last message from dmitrig01 may suggest that a fix for this is in 6.x.2.x-dev. If that is the case - how do I get my hands on that?

@2xdutch nope, it means the bug is in 2.x-dev as well, and it will probably be fixed there first.

Subscribing.

FWIW, this is the error I get with current 2.x-dev:

WD php: Missing argument 1 for drush_drush_make_make_generate() in /usr/share/drush/commands/drush_make/drush_make.drush.inc on line 412.  

Same here. Are we missing and additional argument we suppose to pass along?

`drush generate makefile /path/to/make-file.make`

Adding the path takes care of that, but in my case - using it on a Drupal 7 installation- I get about twenty of these errors:

Could not retrieve version information for .
Could not retrieve version information for .
Could not retrieve version information for .
...

I think this arises from a change that was made in Drush 4.x.

Drush generate-makefile calls _drush_generate_makefile($file). That in turn calls drush_get_projects().

Now, in Drush v3.x, drush_get_projects() would return an array of objects. This is because it calls drush_get_modules() (and drush_get_themes()) for the relevant Drupal major version environment. For D6 this function is just:

function drush_get_modules() {
  $modules = module_rebuild_cache();
  foreach ($modules as $module) {
    if (!isset($module->type)) {
      $module->type = 'module';
    }
  }

  return $modules;
}

In Drush 4.x, this function appears to be implementation-independent and I've not dug around in it too much, but one thing it is doing, is compiling an array:

function drush_get_projects(&$extensions = NULL) {
  /* ... */
  $projects = array('drupal' => array('version' => VERSION));
  foreach ($extensions as $extension) {
  /* ... */
    $projects[$project]['extensions'][] = $extension->name;
  }

  return $projects;
}

Workaround? Downgrade to Drush 3.x . We've just done that here and it worked fine. Solution? Er. Cast arrays as objects? But the 4.x versions seem to have different keys/attributes from the 3.x versions anyway.

I confirm this is failing with drush 4.1, but works when rolling back to 3.3

Subscribing

jp.stacey, thanks for investigating. I'll take a closer look this weekend

i think i fixed it

Seems to be working. Thanks boss.

Thanks for the fix - however it still does not seem to work with 4.2 (fine with 3.3)

Actually, expandonline and Mtro are right. I did not realized when I tested this I was on 3.3.

Al below is using D7

I tried it in 4.2, 4.3-dev and even 5 dev and it all three cases it gave me:

Could not retrieve version information for <em class="placeholder">standard</em>.     [error]
Could not retrieve version information for <em class="placeholder"></em>.   [error]
Some of the properties in your makefile will have to be manually edited. Please do that now.   [error]
Wrote .make file <em class="placeholder">reel_flow_1.make</em>     [ok] 

The makefile it generate only had the api, the profiles type and all the libraries but nothing else. The subdir entry was there but empty: projects[][subdir] = ""

It only seems to work on 3.3.

----Below this it may just be my noobness ----

Using 3.3 it spit out about 15 errors out of about 40 queries it run, all of them "Invalid version <em class="placeholder"></em> for <em class="placeholder"> a module name</em>. and at the end it threw out a final error: An error occurred at function : drush_drush_make_make_generate

It generated a makefile, but all the libraries and modules entries were out order, adding groups of libraries and modules alternatively about 4 times (eg: 5 libraries, then 3 modules, then 6 libraries, then 10 modules, then 3 more libraries....etc)

I can confirm that the generating make files does work under Drush 3.3 but doesn't work in Drush 4.2.

subscribe

no patch yet

Hi,

here is an experimental patch to make generate-makefile work with Drush 4.

Note that I hardcoded the module type to 'module' because drush_get_modules() does not seem to be exporting the type attribute of extensions anymore. This needs to be fixed in Drush itself if needed.

Regards,
Antonio

EDIT: the patch can be downloaded using this URL:
http://drupal.org/files/issues/Issue-1016968-by-ao2-Make-drush-generate-...

As the '#' char is not escaped correctly in the link below.

Updated patch, based on 6.x-3.x.

Changes since v1 of the patch:
Now, instead of blindly hardcode the project type to 'module' we actually
try to get it from drush_get_projects() and fallback to 'module' only if
we don't find the 'type' attribute.

NOTE: Drush still needs to be changed to export the type attribute properly.

The background of the change in drush is the disambiguation of projects and extensions (modules/themes). As you know a project can contain several modules and a module name don't necesarily match the project name. Ej: cck -> content ...

See the big difference in drush_get_projects() between versions 3.x and 4.x/HEAD:
http://api.drush.ws/api/drush/includes--environment.inc/function/drush_g...

Attached is a straightforward fix. A better integration between drush and _make is possible. The inclusion in drush_get_projects() of the type attributed as stated in #22 and #23 will open the door for simplified implementation of generate-makefile with more accuracy.

#1080492: Minor changes to drush_get_projects() for drush_make

Thanks johnattan, the patch in #24 is enough indeed.

Before marking that RTBC, I just have one last observation, with your patch I am getting:

Could not retrieve version information for <em></em>.                                                     [error]

Which results in these lines added to the .make file:

projects[][subdir] = "modules/block"
projects[][version] = "6.20"

By looking at $project_info I think this happens because core modules do not define $item->info['project'] anymore while drush_make.generate.inc now expects its value to be 'drupal' for core projects; I do not know if this is due to a change in drush or rather in the data from drupal.org.

Anyway we need to add an isset($item->info['project']) check as well:

diff --git a/drush_make.generate.inc b/drush_make.generate.inc
index 041ba4f..21b90e4 100644
--- a/drush_make.generate.inc
+++ b/drush_make.generate.inc
@@ -49,7 +49,7 @@ function _drush_make_generate($file) {
   foreach ($project_info as $item) {
     $name = $item->info['project'];
     $type = $item->type;
-    if ($item->info['project'] != 'drupal' && !in_array($item->info['project'], $used_projects)) {
+    if (isset($item->info['project']) && $item->info['project'] != 'drupal' && !in_array($item->info['project'], $used_projects)) {
       if (($item->type == 'module') && (!$item->status)) {
         continue;
       }

If that's OK I can send a proper patch, I just don't know if we can get rid of the == 'drupal' check.

I was also thinking that we could use $name and $type in those ifs instead of $item->info['project'] and $item->type to save some horizontal space, but that's a cleanup which would be in a separate commit.

Thanks,
Antonio

Note: you need very latest drush MASTER to run this patch succesfully.

Changes are basicly a shift from a module-based code to a project-based code. There're is also other fixes in. Full list:
1. fix update service check (drupal.org, features servers,..). It was not working for other services than drupal.org.
2. leverage _pm_find_common_path() to find project path. More reliable now.
3. leverate new drush_get_projects(): we start working from the projects array already obtained by drush --no reinvent.

@jonhattan, the patch works OK for modules, themes and libraries, thanks a lot.

Here below I tried to review it with http://drupal.org/project/dreditor as an exercise.

• Subject: [PATCH 1/2] Issue #1016968: Fix update service check.
  

  Add an extended commit message too. The short commit message tells briefly what you're trying to accomplish, the extended one should tell also why you do it and what are the behavioral changes the patch introduces, finally the code diff tells how the change is implemented.

• +++ b/drush_make.generate.inc
  +++ b/drush_make.generate.inc
  @@ -43,7 +43,8 @@ function _drush_make_generate($file) {
     }
   
     // Re-build theme and module data to be sure we have the most current information.
  -  $project_info = drush_get_projects();
  +  drush_include_engine('drupal', 'environment');
  +  $project_info = array_merge(drush_get_modules(), drush_get_themes());
     $used_projects = array();
     foreach ($project_info as $item) {
       $name = $item->info['project'];
  

  This could go in a separate commit, maybe? As a quick (partial) fix, so to not pollute the _check_updatexml() patch.

• +++ b/drush_make.generate.inc
  @@ -122,20 +127,30 @@ function _drush_generate_track_version($project, $version_options) {
  -  return $update_check === TRUE && $type == $update_check['type'];
  ...
  +  return $update_check !== FALSE && $type == $update_check['type'];
  

  why the check changed to !== FALSE here it is not obvious to me.

• +++ b/drush_make.generate.inc
  @@ -122,20 +127,30 @@ function _drush_generate_track_version($project, $version_options) {
  Subject: [PATCH 2/2] Issue #1016968: leverage drush_get_projects()
  

  An extended commit message wouldn't hurt here as well, albeit less necessary IMHO.

• +++ b/drush_make.generate.inc
  @@ -37,42 +37,58 @@ function _drush_make_generate($file) {
  -    if (_drush_generate_makefile_updatexml($install_profile, 'profile')) {
  +    if (!_drush_generate_makefile_check_updatexml($install_profile, 'profile')) {
  

  Shouldn't this go in the previous commit?

• +++ b/drush_make.generate.inc
  @@ -37,42 +37,58 @@ function _drush_make_generate($file) {
  @@ -156,12 +172,10 @@ function _drush_generate_makefile_check_updatexml($name, $type, $status_url = ''
  
  @@ -156,12 +172,10 @@ function _drush_generate_makefile_check_updatexml($name, $type, $status_url = ''
   /**
    * Helper function to check for a non-default installation location.
    */
  -function _drush_generate_makefile_check_location($module_name, $project_name, $type) {
  -  // Check for non-default installation location.
  -  $path = drupal_get_path($type, $module_name);
  -  $default_location = 'sites/all/' . $type . 's';
  +function _drush_generate_makefile_check_path($project) {
     $info = array();
  -  if ($path != $default_location . '/' . $project_name) {
  +  $path = _pm_find_common_path($project['type'], $project['extensions']);
  +  if (dirname($path) != 'sites/all/'.$project['type'].'s') {
       // If it's in a subdir of sites/all/modules, set the subdir.
       $subdir = preg_replace(array('@^sites/all/' . $type . 's/@', "@/$name" . '$@'), '', $path);
       $info['subdir'] = $subdir;
  

  Maybe this one could be a separate preparatory commit for the levarage drush_get_projects one.

Regards,
Antonio

Powered by Dreditor.

There're two commits in the patch.

First commit is #24 and aditionally fixes the update service check --that I realized was not working for external projects (I tested it with 'singular' theme from developmentseed's repo).

Second commit is the big one: it reimplements a big part of generate-makefile with a shift from a module-based code to a project-based code as stated in #27. It also uses drush's _pm_find_common_path() to obtain the install location for a project.

Perhaps some line in the second commit fits better the first commit but such is life. I can't easily go back to uncommit-change-commit again. Also changes are not previously planned to the last detail but done in the run.

+++ b/drush_make.generate.inc
@@ -122,20 +127,30 @@ function _drush_generate_track_version($project, $version_options) {
-  return $update_check === TRUE && $type == $update_check['type'];
...
+  return $update_check !== FALSE && $type == $update_check['type'];

To explain this is straightforward: === TRUE was not working. $update_check contains the xml from the update service or FALSE. It never is set to TRUE.

wrt. patch from #27: RTBC, as it is indeed working fine.

REMOVED - WRONG ISSUE

Late night working, was using the wrong commands. Spaced out. My apologies.

#24 worked for me as well (drush 4.2, and drush make 2.0)

Thanks!

I'm still having this problem with Drush 4.4 and 4.5-dev. Should this have been fixed? I can try to debug further if necessary.

Still having this problem in Drush 4.4.

Update: I think this bug has been fixed. When I posted a couple weeks ago I was up-to-date with Drush but not Drush Make. After updating Drush Make the make-generate command works perfectly.