I am trying to migrate a a site from drupal 7.22 based custom platform to 7.27 for upgrade purposes. Site clone, verify etc are successful but migration is failing with the following two errors:

The command could not be executed successfully (returned: Fatal error: Allowed memory size of 261095424 bytes exhausted (tried to allocate 124 bytes) in /data/disk/e1/static/platforms/ecoms-7.27.1-p001/includes/menu.inc on line 2757 , code: 0)

Drush was not able to start (bootstrap) the Drupal database.
Hint: This error often occurs when Drush is trying to bootstrap a site that has not been installed or does not have a configured database.

Drush was attempting to connect to : 
  Drupal version    : 7.27
  Site URI          : migrate.ecoms.org.uk
  Default theme     : garland
  Administration theme: garland
  PHP configuration : /opt/php53/lib/php.ini /data/disk/e1/.drush/php.ini
  Drush version     : 4.6-dev
  Drush configuration: /data/disk/e1/static/platforms/ecoms-7.27.1-p001/sites/migrate.ecoms.org.uk/drushrc.php /data/disk/e1/static/platforms/ecoms-7.27.1-p001/drushrc.php
  Drush alias files : /data/disk/e1/.drush/migrate.ecoms.org.uk.alias.drushrc.php /data/disk/e1/.drush/server_localhost.alias.drushrc.php /data/disk/e1/.drush/server_master.alias.drushrc.php /data/disk/e1/.drush/platform_ECOMS7271P001.alias.drushrc.php /data/disk/e1/.drush/hs-legal.co.uk.alias.drushrc.php /data/disk/e1/.drush/ecoms.org.uk.alias.drushrc.php /data/disk/e1/.drush/platform_Guardr137271P002.alias.drushrc.php /data/disk/e1/.drush/platform_005.alias.drushrc.php /data/disk/e1/.drush/platform_004.alias.drushrc.php /data/disk/e1/.drush/platform_Drupal6311P002.alias.drushrc.php /data/disk/e1/.drush/platform_OpenAcademy107271P002.alias.drushrc.php /data/disk/e1/.drush/platform_ECOMSSMS7221P001.alias.drushrc.php /data/disk/e1/.drush/sms.e1.web1.ecoel.org.uk.alias.drushrc.php /data/disk/e1/.drush/hostmaster.alias.drushrc.php /data/disk/e1/.drush/platform_010.alias.drushrc.php /data/disk/e1/.drush/platform_006.alias.drushrc.php /data/disk/e1/.drush/platform_ECOMSSMS7271P001.alias.drushrc.php /data/disk/e1/.drush/platform_011.alias.drushrc.php /data/disk/e1/.drush/platform_008.alias.drushrc.php /data/disk/e1/.drush/platform_003.alias.drushrc.php /data/disk/e1/.drush/platform_Drupal6311S002.alias.drushrc.php /data/disk/e1/.drush/platform_hostmaster.alias.drushrc.php /data/disk/e1/.drush/platform_Commerce2137271P002.alias.drushrc.php /data/disk/e1/.drush/platform_Drupal6311D002.alias.drushrc.php /data/disk/e1/.drush/recreate.e1.web1.ecoel.org.uk.alias.drushrc.php /data/disk/e1/.drush/platform_002.alias.drushrc.php /data/disk/e1/.drush/sms.ecoms.org.uk.alias.drushrc.php /data/disk/e1/.drush/platform_Commerce1187221P001.alias.drushrc.php /data/disk/e1/.drush/platform_ECOMS7221P001.alias.drushrc.php /data/disk/e1/.drush/platform_ECOMSSMS.alias.drushrc.php /data/disk/e1/.drush/platform_HSLegal6x.alias.drushrc.php /data/disk/e1/.drush/platform_007.alias.drushrc.php /data/disk/e1/.drush/platform_009.alias.drushrc.php /data/disk/e1/.drush/ecoms.e1.web1.ecoel.org.uk.alias.drushrc.php
  Drupal root       : /data/disk/e1/static/platforms/ecoms-7.27.1-p001
  Site path         : sites/migrate.ecoms.org.uk
  Modules path      : sites/migrate.ecoms.org.uk/modules
  Themes path       : sites/migrate.ecoms.org.uk/themes
  File directory path: sites/migrate.ecoms.org.uk/files
  %paths            : Array

You can select another site with a working database setup by specifying the URI to use with the --uri parameter on the command line or $options['uri'] in your drushrc.php file.

I am trying to migrate the cloned version of the site (clone was successful in the same platform). The site is otherwise working ok. I have also discovered that after the couple of attemts the site looses connection to its db. Platform and site verify task brings it back though. I tried increasing memory_limit in /opt/local/etc/php53.ini but no help. I fact I am seeing 4-5 php-fpm entries in top command output. Where does database disappears? Memory exahaustion??? Don't Know. The server has 1GB RAM and only two sites in production.

The upgrade to 2.2.3 has been a bit tough till now and is a continuation of troubles in #2246691: Maria DB 10.0.10 high cpu usage -site horribly slow before being cached.

Any help is highly appreciated. If guided I may be able to provide more information.

CommentFileSizeAuthor
octopus_log.txt1.18 KBaanjaneyam
e1.octopus.cnf_.txt1.09 KBaanjaneyam
barracuda_log.txt2.14 KBaanjaneyam
barracuda.cnf_.txt1.96 KBaanjaneyam
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

aanjaneyam’s picture

aanjaneyam CreditAttribution: aanjaneyam commented
Title: Migrate Task Fails » Migrate Task Fails - BOA 2.2.3
aanjaneyam’s picture

aanjaneyam CreditAttribution: aanjaneyam commented

I have no modules in site-name/modules folder all the modules are in /sites/all/modules/contrib folder. I was migrating/upgrading the site to deal with slow loading of sites after upgrading to BOA 2.2.3.

omega8cc’s picture

omega8cc CreditAttribution: omega8cc commented
Component: Aegir Provision » Miscellaneous

The answer is right in front of you: "Allowed memory size of 261095424 bytes exhausted".

You either need to increase limits normally set by BOA based on available resources (RAM) or force your own limits with BOND script, or you have memory leaks in your codebase, which means that no amount of extra memory may help. You should find the real source of your problem and not blindly try unrelated changes/actions.

aanjaneyam’s picture

aanjaneyam CreditAttribution: aanjaneyam commented

You either need to increase limits normally set by BOA based on available resources (RAM)

Thanks!! But where should the above be done? As stated earlier I tried increasing memory_limit in /opt/local/etc/php53.ini to 300 MB instead of default 196 MB but no help.

omega8cc’s picture

omega8cc CreditAttribution: omega8cc commented

Please read the release notes: https://github.com/omega8cc/boa/blob/master/CHANGELOG.txt#L378

For PHP-FPM see also #2242519: Unable to effectively Edit Php Memory limit (non caps sorry)

aanjaneyam’s picture

aanjaneyam CreditAttribution: aanjaneyam commented

I am not sure if it is memory leak as drupal 6 sites including hostmaster are working fine blazing fast.

aanjaneyam’s picture

aanjaneyam CreditAttribution: aanjaneyam commented

Well it could be memory leak. I get the following I after increasing the memory to 300M:

The command could not be executed successfully (returned: Fatal error: Allowed memory size of 314572800 bytes exhausted (tried to allocate 79 bytes) in /data/disk/e1/static/platforms/ecoms-7.27.1-p001/includes/module.inc on line 101 , code: 0)

omega8cc’s picture

omega8cc CreditAttribution: omega8cc commented
Status: Active » Closed (won't fix)
aanjaneyam’s picture

aanjaneyam CreditAttribution: aanjaneyam commented

Hi. I have tried other sites also. Including which almost vanilla sites with no views custom menus (just few modules enabled. The followin is common among all sites:

Drush was not able to start (bootstrap) the Drupal database.
Hint: This error often occurs when Drush is trying to bootstrap a site that has not been installed or does not have a configured database.

Drush was attempting to connect to : 
  Drupal version    : 7.27
  Site URI          : recreate.e1.lws01.ecoel.org.uk
  Default theme     : garland
  Administration theme: garland
  PHP configuration : /opt/php53/lib/php.ini /data/disk/e1/.drush/php.ini
  Drush version     : 4.6-dev
  Drush configuration: /data/disk/e1/static/ecoms-7.27.1-p001/sites/recreate.e1.lws01.ecoel.org.uk/drushrc.php /data/disk/e1/static/ecoms-7.27.1-p001/drushrc.php
  Drush alias files : /data/disk/e1/.drush/recreate.e1.lws01.ecoel.org.uk.alias.drushrc.php /data/disk/e1/.drush/server_localhost.alias.drushrc.php /data/disk/e1/.drush/server_master.alias.drushrc.php /data/disk/e1/.drush/platform_ECOMS7271P001.alias.drushrc.php /data/disk/e1/.drush/platform_ECOMSSMS.alias.drushrc.php /data/disk/e1/.drush/platform_OpenAcademy107271P001.alias.drushrc.php /data/disk/e1/.drush/ecoms.org.uk.alias.drushrc.php /data/disk/e1/.drush/sms.e1.lws01.ecoel.org.uk.alias.drushrc.php /data/disk/e1/.drush/hs-legal.co.uk.alias.drushrc.php /data/disk/e1/.drush/sms.ecoms.org.uk.alias.drushrc.php /data/disk/e1/.drush/platform_Commons3917271P001.alias.drushrc.php /data/disk/e1/.drush/platform_ECOMSSMS7271P001.alias.drushrc.php /data/disk/e1/.drush/migrate.ecoms.org.uk.alias.drushrc.php /data/disk/e1/.drush/ecoms.e1.lws01.ecoel.org.uk.alias.drushrc.php /data/disk/e1/.drush/platform_hostmaster.alias.drushrc.php /data/disk/e1/.drush/platform_Guardr137271P001.alias.drushrc.php /data/disk/e1/.drush/hostmaster.alias.drushrc.php /data/disk/e1/.drush/platform_ECOMS7221P001.alias.drushrc.php /data/disk/e1/.drush/platform_002.alias.drushrc.php /data/disk/e1/.drush/server_web1ecoelorguk.alias.drushrc.php /data/disk/e1/.drush/platform_ECOMSSMS7221P001.alias.drushrc.php /data/disk/e1/.drush/platform_HSLegal6x.alias.drushrc.php
  Drupal root       : /data/disk/e1/static/ecoms-7.27.1-p001
  Site path         : sites/recreate.e1.lws01.ecoel.org.uk
  Modules path      : sites/recreate.e1.lws01.ecoel.org.uk/modules
  Themes path       : sites/recreate.e1.lws01.ecoel.org.uk/themes
  File directory path: sites/recreate.e1.lws01.ecoel.org.uk/files
  %paths            : Array

You can select another site with a working database setup by specifying the URI to use with the --uri parameter on the command line or $options['uri'] in your drushrc.php file.

could not bootstrap drupal after updatedb
Template loaded: /data/disk/e1/.drush/provision/http/Provision/Config/Nginx/vhost.tpl.php

Any help would be appreciated.

aanjaneyam’s picture

aanjaneyam CreditAttribution: aanjaneyam commented

If one visits the url it can be seen that nothing done on above site #9.

omega8cc’s picture

omega8cc CreditAttribution: omega8cc commented

You are adding unrelated reports, but let's answer this anyway.

Drush was not able to start (bootstrap) the Drupal database.

The reason can be:

1. The database referenced in the drushrc.php doesn't exist
2. Credentials in settings.php / drushrc.php are incorrect because if previous failed or incomplete migration
3. You can duplicate grants with different passwords for the same site in the mysql database, which confuses Drupal

aanjaneyam’s picture

aanjaneyam CreditAttribution: aanjaneyam commented

The reason can be:

Hi I have checked the steps in #11 above and they seem to be ok. DB exists in chive, I can log into chive using the credentials given in sites's settings.php and I just see two previledges in for the sites database user (one at localhost -USAGE and other 127.0.0.1-USAGE).

Could the below be also responsible for "Drush was not able to start (bootstrap) the Drupal database." error:

The command could not be executed successfully (returned: Fatal error: Allowed memory size of 1048576000 bytes exhausted (tried to allocate 137 bytes) in /data/disk/e1/static/ecoms-7.27.1-p001/includes/menu.inc on line 2757 , code: 0)

However the above memory error keeps on changing in terms of refering to different modules. Below are couple other instances:

modules/help/help.module on line 25
includes/module.inc on line 101

I have kept on disabling modules and retrying if it migrates but no. I disabled taxonomy menu, xml sitemap menu, special menu items, my custom features. Oh I don't know what could be causing memory error. Is it core in some way? The site being upgraded runs Drupal core 7.22.

Could I try manually upgrading the site to new updated platform using the relevant steps in https://omega8.cc/import-your-sites-to-aegir-in-8-easy-steps-109 .

aanjaneyam’s picture

aanjaneyam CreditAttribution: aanjaneyam commented

I just carefully looked at the error below and all other errors in the episode. It is trying to find site db in the new platform to which the site is being migrated. But the site is migrated from from the old to the new platform which is based on Drupal core 7.27. The site migrated site is not yet present in new platform:

Drush was not able to start (bootstrap) the Drupal database.
Hint: This error often occurs when Drush is trying to bootstrap a site that has not been installed or does not have a configured database.

Drush was attempting to connect to : 
  Drupal version    : 7.27
  Site URI          : migrate.ecoms.org.uk
  Default theme     : garland
  Administration theme: garland
  PHP configuration : /opt/php53/lib/php.ini /data/disk/e1/.drush/php.ini
  Drush version     : 4.6-dev
  Drush configuration: /data/disk/e1/static/ecoms-7.27.1-p001/sites/migrate.ecoms.org.uk/drushrc.php /data/disk/e1/static/ecoms-7.27.1-p001/drushrc.php
  Drush alias files : /data/disk/e1/.drush/migrate.ecoms.org.uk.alias.drushrc.php /data/disk/e1/.drush/server_localhost.alias.drushrc.php /data/disk/e1/.drush/server_master.alias.drushrc.php /data/disk/e1/.drush/platform_ECOMS7271P001.alias.drushrc.php /data/disk/e1/.drush/platform_ECOMSSMS.alias.drushrc.php /data/disk/e1/.drush/platform_OpenAcademy107271P001.alias.drushrc.php /data/disk/e1/.drush/ecoms.org.uk.alias.drushrc.php /data/disk/e1/.drush/sms.e1.lws01.ecoel.org.uk.alias.drushrc.php /data/disk/e1/.drush/hs-legal.co.uk.alias.drushrc.php /data/disk/e1/.drush/recreate.e1.lws01.ecoel.org.uk.alias.drushrc.php /data/disk/e1/.drush/sms.ecoms.org.uk.alias.drushrc.php /data/disk/e1/.drush/platform_Commons3917271P001.alias.drushrc.php /data/disk/e1/.drush/platform_ECOMSSMS7271P001.alias.drushrc.php /data/disk/e1/.drush/ecoms.e1.lws01.ecoel.org.uk.alias.drushrc.php /data/disk/e1/.drush/migratesms.ecoms.org.uk.alias.drushrc.php /data/disk/e1/.drush/platform_hostmaster.alias.drushrc.php /data/disk/e1/.drush/platform_Guardr137271P001.alias.drushrc.php /data/disk/e1/.drush/hostmaster.alias.drushrc.php /data/disk/e1/.drush/platform_ECOMS7221P001.alias.drushrc.php /data/disk/e1/.drush/platform_002.alias.drushrc.php /data/disk/e1/.drush/server_web1ecoelorguk.alias.drushrc.php /data/disk/e1/.drush/platform_ECOMSSMS7221P001.alias.drushrc.php /data/disk/e1/.drush/platform_HSLegal6x.alias.drushrc.php
  Drupal root       : /data/disk/e1/static/ecoms-7.27.1-p001
  Site path         : sites/migrate.ecoms.org.uk
  Modules path      : sites/migrate.ecoms.org.uk/modules
  Themes path       : sites/migrate.ecoms.org.uk/themes
  File directory path: sites/migrate.ecoms.org.uk/files
  %paths            : Array

You can select another site with a working database setup by specifying the URI to use with the --uri parameter on the command line or $options['uri'] in your drushrc.php file.

could not bootstrap drupal after updatedb

Well all the errors above are pointing to the code in new platform with migrated site not yet present. Strange?? How could a db exist in new platform.

aanjaneyam’s picture

aanjaneyam CreditAttribution: aanjaneyam commented

You are adding unrelated reports, ....

Well its the same platform. I am trying all the tricks to migrate the relevant site mentioned in #1 or its copy. In #9 above I just tried a different site (with less modules and no customisation) in the same platform to see if it works.

aanjaneyam’s picture

aanjaneyam CreditAttribution: aanjaneyam commented

I mean all the reports are relevant to the problem.

omega8cc’s picture

omega8cc CreditAttribution: omega8cc commented

You have managed to create hidden copies of the site apparently, so you may want to read and follow:

https://omega8.cc/mysterious-sitesfoorestore-directory-228
https://omega8.cc/aegir-task-fails-or-spins-forever-126

The main problem is unrelated to BOA, it is the memory leak in your codebase, so we can't help.

aanjaneyam’s picture

aanjaneyam CreditAttribution: aanjaneyam commented

Could I try manually upgrading the site to new updated platform using the relevant steps in https://omega8.cc/import-your-sites-to-aegir-in-8-easy-steps-109 .

I have been unable to find leak in module codes so possibly manual upgrade to new platform seems to be the way. Hopefully this would resolve the memory leak problem because some older version of some module may be creating the leak.

I looked at sites directory of the new platform and could not find any hidden directories or files.