I'm trying to follow this guide: http://mig5.net/node/346 to learn how to handle provision without Aegir.

However, even with a trivial make file as the one listed in the guide, drush dies with:

drush --root='/var/www/localhost/htdocs' provision-save '@platform_test' --context_type='platform' --makefile='drupal.make'
PHP Fatal error:  Allowed memory size of 201326592 bytes exhausted (tried to allocate 130968 bytes) in /usr/share/drush/commands/provision/provision.context.inc on line 139

Fatal error: Allowed memory size of 201326592 bytes exhausted (tried to allocate 130968 bytes) in /usr/share/drush/commands/provision/provision.context.inc on line 139

I tried some variations with relative vs local paths, I cleared out everything from ~/.drush to make sure nothing else was broken and caused some type of conflict. I'm not very familiar with the codebase, but it looks to me like provision get's stuck in loading contexts. It is definitely a recursive loop however.
I counted the length of the backtrace in function get_services(), and it reaches into several thousands if I set php to allow it.

The problem occurs in
[file] => /usr/share/drush/commands/provision/provision.context.inc
[line] => 377
[function] => get_services, which calls back onto itself infinitely.

Below is an excerpt of backtrace of where the problem happens. Number 8 and 9 in this trace, repeat all the way to the top, except the very final call that's always the magic __get function.

  [8] => Array (
    [file] => /usr/share/drush/commands/provision/provision.context.inc
    [line] => 377
    [function] => get_services
    [class] => provisionContext
    [object] => provisionContext_platform Object (
      [parent_key] => server
      [name] => @server_master
      [type] => platform
      [properties:protected] => Array (
        [context_type] => platform
        [server] => @server_master
        [web_server] => @server_master
      )

      [oid_map:protected] => Array (
        [server] => 1
        [web_server] => 1
      )

      [service_subs:protected] => Array (
        [http] => @server_master
      )
    )

    [type] => ->
    [args] => Array ()
  )

  [9] => Array (
    [file] => /usr/share/drush/commands/provision/provision.context.inc
    [line] => 377
    [function] => get_services
    [class] => provisionContext
    [object] => provisionContext_platform Object (
      [parent_key] => server
      [name] => @server_master
      [type] => platform
      [properties:protected] => Array (
        [context_type] => platform
        [server] => @server_master
        [web_server] => @server_master
      )

      [oid_map:protected] => Array (
        [server] => 1
        [web_server] => 1
      )

      [service_subs:protected] => Array (
        [http] => @server_master
      )
    )

    [type] => ->
    [args] => Array ()
  )

  [10] => Array (
    [file] => /usr/share/drush/commands/provision/provision.context.inc
    [line] => 358
    [function] => get_services
    [class] => provisionContext
    [object] => provisionContext_platform Object (
      [parent_key] => server
      [name] => @server_master
      [type] => platform
      [properties:protected] => Array (
        [context_type] => platform
        [server] => @server_master
        [web_server] => @server_master
      )

      [oid_map:protected] => Array (
        [server] => 1
        [web_server] => 1
      )

      [service_subs:protected] => Array (
        [http] => @server_master
      )
    )

    [type] => ->
    [args] => Array ()
  )

  [11] => Array (
    [file] => /usr/share/drush/commands/provision/provision.context.inc
    [line] => 210
    [function] => services_invoke
    [class] => provisionContext
    [object] => provisionContext_platform Object (
      [parent_key] => server
      [name] => @server_master
      [type] => platform
      [properties:protected] => Array (
        [context_type] => platform
        [server] => @server_master
        [web_server] => @server_master
      )
      [oid_map:protected] => Array (
         [server] => 1
         [web_server] => 1
      )
      [service_subs:protected] => Array (
        [http] => @server_master
      )
    )

    [type] => ->
    [args] => Array (
      [0] => init
      [1] => Array()
    )
  )
CommentFileSizeAuthor
#11 provision-1454316-context-loops.patch2.79 KBSteven Jones
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

anarcat’s picture

anarcat CreditAttribution: anarcat commented

Could this be related with this bug #946606: provision-save core dumps on loops?

Letharion’s picture

Letharion CreditAttribution: Letharion commented

Not that I can see, my php doesn't segfault.

I realized a --debug would perhaps also be useful:

$ drush --root='/home/letharion/tmp/tmpbuild' provision-save '@platform_test' --context_type='platform' --makefile='drupal.make' --debug
Bootstrap to phase 0. [0.01 sec, 3.63 MB]                    [bootstrap]
Drush bootstrap phase : _drush_bootstrap_drush() [0.01 sec, 3.91 MB] [bootstrap]
Bootstrap to phase 0. [0.07 sec, 12.96 MB]                    [bootstrap]
Found command: provision-save (commandfile=provision) [0.07 sec, 12.97 MB]                    [bootstrap]
Initializing drush commandfile: drush_make [0.07 sec, 12.97 MB]                    [bootstrap]
Initializing drush commandfile: drush_make_d_o [0.07 sec, 12.97 MB]                    [bootstrap]
Initializing drush commandfile: provision [0.07 sec, 12.97 MB]                    [bootstrap]
Load alias @self [0.07 sec, 12.98 MB]                    [notice]
Load alias @server_master [0.07 sec, 12.99 MB]                    [notice]
PHP Fatal error:  Maximum function nesting level of '100' reached, aborting! in /usr/share/drush/commands/provision/provision.context.inc on line 23

Fatal error: Maximum function nesting level of '100' reached, aborting! in /usr/share/drush/commands/provision/provision.context.inc on line 23
Drush command terminated abnormally due to an unrecoverable error.                                                                                                                                                    [error]
Error: Maximum function nesting level of '100' reached, aborting! in /usr/share/drush/commands/provision/provision.context.inc, line 23 [0.07 sec, 13.02 MB]
Letharion’s picture

Letharion CreditAttribution: Letharion commented

The difference between "Fatal error: Allowed memory size of 201326592 bytes exhausted" the first time and "Error: Maximum function nesting level of '100' reached, aborting!" is because I've tried with both php 5.3 and 5.4, and their ini files differ a bit.

anarcat’s picture

anarcat CreditAttribution: anarcat commented

I think the core dump was happening in PHP 5.2

Anonymous’s picture

Anonymous (not verified) CreditAttribution: Anonymous commented

Is this a duplicate of #946606: provision-save core dumps on loops ?

Letharion’s picture

Letharion CreditAttribution: Letharion commented

@mig5, anarcat asked the same thing in #1 :) I'm not sure. I will attempt to debug the problem some more, and get back with more info.

Letharion’s picture

Letharion CreditAttribution: Letharion commented
Version: 6.x-1.6 » 6.x-1.7

When the provision command inits, we call 

$hash_name = drush_get_option('#name') ? '#name' : 'name';
d(drush_get_option($hash_name, '@self', 'alias'), TRUE);

This always results in d('@self', TRUE)
Since @self is not a known instance at this point, we call

$instances[$name] = provision_context_factory($name);
$instances[$name]->method_invoke('init');
$instances[$name]->type_invoke('init');

Which creates a new provisionContext_platform, and runs init() on it. This in turn calls d('@server_master', FALSE); and creates an instance for '@server_master', and runs init on this object.

During my debugging, I noticed that 

$service_list = drush_command_invoke_all('provision_services');

returns an arrays where all keys 'dns', 'http' and 'db' are NULL. Not sure if that's a problem. As the last step, '@server_master' method_invoke('init'), calls

if ($services) {
  $this->services_invoke($func, $args);
}

leading to:

function get_services() {
  $services = array();
  if (!is_null($this->parent_key)) {
    $services = $this->{$this->parent_key}->get_services();
  }

which is the end of our journey. $this->parent_key, is server, so we call get_services() on the parent instead. Problem now is that d(), returns the '@server_master', which is ourselves. Hence $parent_key is _still_ server, and hilarity ensues.

I lack sufficient understanding of the code base to grasp where exactly this goes awry, and would very much like some feedback on the above.

I tried with the newly released 1.7, as well as 2.x, both of which exhibit the same behavior, although I haven't debugged them in the same detail.

Steven Jones’s picture

Steven Jones CreditAttribution: Steven Jones commented
Assigned: Unassigned » Steven Jones
Category: support » bug
wamilton’s picture

wamilton CreditAttribution: wamilton commented

darthsteven helped me out with this during #aegir office hours. I had no @server_master alias, so he had me generate one with

drush provision-save @server_master --context_type='server'

and all was well. Kind of obvious if you think about it, but also an obvious case for throwing a bold, red warning to the tune of

there is something glaringly wrong with what you're doing: you need an alias for @server_master

Letharion: please confirm if doing this solves your issue, and if so I think we can open a new issue for documentation or error messaging, or switch the tags on this ticket?

Steven Jones’s picture

Steven Jones CreditAttribution: Steven Jones commented

Right, so getting somewhere with this, if I run the following:

drush provision-save '@platform_looping_test' --context_type='platform'  --web_server='@server_non_existing'

Where the web_server alias doesn't exist then this command will eat up all the memory when it enters the infinite loop described in #7.

This is because the '@server_non_existing' alias doesn't exist and it's created, and uses the 'context_type' from the command line, incorrectly assuming that the new server alias is, in fact a platform, which causes the loop.

This is because of the following in provision_context_factory:

  $record = provision_sitealias_get_record($name);
  $options = array_merge(drush_get_context('stdin'), drush_get_context('options'), drush_get_context('cli'));

  if (isset($record['context_type'])) {
    $type = $record['context_type'];
  }
  elseif (isset($options['context_type'])) {
    $type = $options['context_type'];
  }

Which unconditionally creates the '@server_non_existing' context with the context type from the command line. It should only do this if the context was the one specified on the command line I think.

Steven Jones’s picture

Steven Jones CreditAttribution: Steven Jones commented
Assigned: Steven Jones » Unassigned
Status: Active » Needs review
FileSize
provision-1454316-context-loops.patch2.79 KB

Right, so here's a patch that essentially throws an error when referencing a context as a service property that doesn't exist. It seems to fix the problem, and gives a helpful error message too.

Steven Jones’s picture

Steven Jones CreditAttribution: Steven Jones commented
Status: Needs review » Fixed

Worked on this a little more, and tested it out, seems to work great!

Fixed in 6.x-2.x and 6.x-1.x

Letharion’s picture

Letharion CreditAttribution: Letharion commented

Thanks a lot for looking further into this. What you say is indeed obivous, but since I was following along with mig5's blog post, I simply assumed I had done everything that was necessary. I haven't had the time to look into/test this further, but your fix seems, again, totally obvious.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

  • Commit 322cf0b on dev-drupal-8, 6.x-2.x, dev-ssl-ip-allocation-refactor, dev-1205458-move_sites_out_of_platforms, 7.x-3.x, dev-subdir-multiserver, 6.x-2.x-backports, dev-helmo-3.x by Steven Jones: 
    Issue #1454316 by Steven Jones: Fixed Provision recurses infinitely on...
  • Commit 59ec3f1 on 6.x-1.x, dev-drupal-8, dev-ssl-ip-allocation-refactor, dev-1205458-move_sites_out_of_platforms, 7.x-3.x, dev-subdir-multiserver, 6.x-2.x-backports, dev-helmo-3.x by Steven Jones:
    Issue #1454316 by Steven Jones: Fixed Provision recurses infinitely on...
  • Commit f61e716 on 6.x-1.x, dev-drupal-8, dev-ssl-ip-allocation-refactor, dev-1205458-move_sites_out_of_platforms, 7.x-3.x, dev-subdir-multiserver, 6.x-2.x-backports, dev-helmo-3.x by Steven Jones:
    Issue #1454316 by Steven Jones: Fixed Provision recurses infinitely on...
  • Commit 9a3857d on dev-drupal-8, 6.x-2.x, dev-ssl-ip-allocation-refactor, dev-1205458-move_sites_out_of_platforms, 7.x-3.x, dev-subdir-multiserver, 6.x-2.x-backports, dev-helmo-3.x by Steven Jones:
    Issue #1454316 by Steven Jones: Fixed Provision recurses infinitely on...