Drupal module installation process takes too long after/during installing civicrm module while site setup [#2152869]

I am using CiviCRM Starter Kit installation profile to setup CiviCRM Starter Kit on my Ubuntu 13.10 system. There is 8GB RAM.

So far I have tried many distributions like Commerce Kickstart, Commons, Spark etc. and they got installed very quickly. But during the installation of CiviCRM Starter Kit, I noticed that setup is quite fast until it reaches civicrm module installation. CiviCRM gets setup quickly. Few drupal core modules get setup quickly. But after the setup of core modules, when the system try to install civicrm module, it takes too long. After this, all the other module installation takes too long as well.

It takes roughly 20-30 minutes more, just to install remaining 20+ modules. Please see what can be done to resolve this issue.

Comment	File	Size	Author
	civicrm_starter_kit_installation.png	54.16 KB	amitgoyal

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Comment #1

kreynen CreditAttribution: kreynen commented 9 December 2013 at 19:08

Category:	Bug report	» Support request
Status:	Active	» Closed (cannot reproduce)

Just because a system has 8GB of RAM, doesn't make that memory available to PHP or MySQL. This has been reported before on a distribution that builds on the CiviCRM Starter Kit...

#1830446: Install is very slow

I'm not denying that it's possible that on some configurations it could take up to 30 minutes to install, but I can't reproduce that on my servers or Pantheon and I'm not really interested in creating slower systems.

While the modules that are enabled after CiviCRM are enabled slower, that is most likely because of the increase in memory required when bootstrapping Drupal once Civicrm is enabled. In https://vimeo.com/53877092 you can jump to the 3:00 mark and see that is takes < 2 minutes to enable the rest of the modules. That's on my laptop w/ 8GM of RAM running MAMP.

20-30 minutes points to something really wrong with the environment.

I'm willing to help you troubleshoot what part of your configuration needs to be changed to run the install in a reasonable amount of time, but I need more information from you.

Do you have APC or any other PHP cache enabled?
Have you changed the php and mysql configuration files? If so what did you change?

Comment #2

amitgoyal CreditAttribution: amitgoyal commented 10 December 2013 at 06:12

@kreynen - If you are not able to reproduce the issue, it doesn't mean bug doesn't exist. Bug is still there and we have faced it on both Linux and Windows systems.

Yes, we have both APC and memcache installed on our Linux system. PHP is having 128M memory.

The problem starts occuring during CiviCRM Starter Kit setup. As I mentioned earlier, CiviCRM gets setup along with Drupal must have modules quickly. But as long as it reaches, CiviCRM module, installlation becomes very slow.

Not sure why have you marked it closed without further inputs..

Comment #3

kreynen CreditAttribution: kreynen commented 10 December 2013 at 14:49

Status:

Closed (cannot reproduce)

» Closed (works as designed)

I closed the issue because this isn't a bug w/ the distribution. I've installed this a dozen times yesterday alone troubleshooting errors w/ the 4.4.3 update.

Telling me the box running the stack has 8GB of RAM is really useless information until you get into how many authenticated users can use the site concurrently. You didn't start this issue telling me that you've run CiviCRM on this environment without an issue and only have an issue when using CiviCRM with the distribution. This leads me to believe that you don't have much experience w/ CiviCRM and this may actually be the first time you are using it.

Is that correct?

If you installed CiviCRM using the normal install process and then enabled modules after that, my guess is you'd see the same slowness while enabling (or even disabling) any module after CiviCRM is installed.

128MB is the minimum amount of memory required to run CiviCRM. Less than that and the install won't even let you attempt the install. The CiviCRM code base is MUCH larger than Drupal "out of the box" and it creates 4x the number of tables. There are a number of things you can do to optimize your stack for CiviCRM, but that has nothing to do w/ running CiviCRM as part of a distribution.

I've changed the issue from Closed (maintainer needs more information) to Closed (works a designed).

It shouldn't come as a shock that if you run software w/ the minimum requirements, it will run slower than if you allocate more than the minimum resources.

I think I've made it pretty clear on the CiviCRM Starter Kit project page that I'd redirect support requests to the appropriate place. If you aren't already running on stack configured to run CiviCRM well, you need to start there. Start w/ a normal CiviCRM install and get the performance issues figured out by posting to http://forum.civicrm.org/index.php/board,6.0.html.

So we are 100% clear... your problem is running CiviCRM on your environment in any configuration. The fact that it's enabled during the install of a distribution is a symptom of that configuration... not the root cause.

Comment #4

amitgoyal CreditAttribution: amitgoyal commented 11 December 2013 at 06:06

Looks like you did't understand the problem yet. The problem is occuring when I am trying to install the distribution.

So concurrency of users doesn't come into picture. Same for enabling the modules after distribution installation. Both these points are irrelevant as the problem is occuring during setup of ditribution itself.

I had separate installation of D7 and then installed CiviCRM on top of that and it worked perfectly fine.

As I guessed correctly, the problem starts occuring when installation of distribution try to install civicrm module as part of installation process. So I tried adding a blank/dummy module (abc.module) which has dependencies, in it's info file, for all the core/contributed modules except civicrm related modules. Now during the installation of distribution, Drupal will pick this module, abc.module, first for installation than civicrm related modules so all the modules get installed quickly. At the end, it installed civicrm related modules and just took couple of min. extra.

Thanks for your time and quick replies. Please let me know if you need more info. on this.

Comment #5

kreynen CreditAttribution: kreynen commented 17 December 2013 at 18:06

I understand that concurrency isn't an issue at this point of the install. Do you understand that telling me a box has 8GB of RAM is almost meaningless to troubleshoot a slow install and would only apply to concurrent use?

When you use a module with CiviCRM as a dependency to change the order modules are enabled to install more quickly, how long does it take to enable modules after the install? Do you have the same delay?

I don't know how Drupal determines the order it enables modules from the profiles.info. Drush make packages the dependencies in the .make in reverse order. It's possible that simply changing the order of the dependencies in the .info would make CiviCRM to be one of the last modules enabled.

If you can test that, I'd be happy to make that change.

Comment #6

amitgoyal CreditAttribution: amitgoyal commented 26 December 2013 at 06:43

As I mentioned in my last comment that I already tried this solution and it worked fine for me.

I created a blank module with files abc.info and abc.module. abc.module is an empty file and abc.info contains following stuff,

name = abc
description = Dependencies defined here will help in installing all required modules before civicrm related modules as civicrm module slows down the installation process.
core = 7.x

; Core
dependencies[] = block
;dependencies[] = blog
dependencies[] = color
dependencies[] = comment
dependencies[] = contextual
dependencies[] = dashboard
dependencies[] = dblog
dependencies[] = field
dependencies[] = field_sql_storage
dependencies[] = field_ui
dependencies[] = file
dependencies[] = filter
;dependencies[] = forum
dependencies[] = help
dependencies[] = image
dependencies[] = list
dependencies[] = menu
dependencies[] = node
dependencies[] = number
dependencies[] = options
dependencies[] = overlay
dependencies[] = path
dependencies[] = rdf
dependencies[] = search
dependencies[] = shortcut
dependencies[] = system
dependencies[] = taxonomy
dependencies[] = text
;dependencies[] = update
dependencies[] = user

; Contrib
dependencies[] = admin_menu
dependencies[] = admin_menu_toolbar
dependencies[] = backup_migrate
dependencies[] = ckeditor
dependencies[] = ctools
dependencies[] = token

My Installation profile's info file, test.info, contains following stuff,

name = Test
description = "CiviCRM Starter Kit with Drupal 7"
core = "7.x"

exclusive = "1"
distribution_name = "Test"

; Core
dependencies[] = block
;dependencies[] = blog
dependencies[] = color
dependencies[] = comment
dependencies[] = contextual
dependencies[] = dashboard
dependencies[] = dblog
dependencies[] = field
dependencies[] = field_sql_storage
dependencies[] = field_ui
dependencies[] = file
dependencies[] = filter
;dependencies[] = forum
dependencies[] = help
dependencies[] = image
dependencies[] = list
dependencies[] = menu
dependencies[] = node
dependencies[] = number
dependencies[] = options
dependencies[] = overlay
dependencies[] = path
dependencies[] = rdf
dependencies[] = search
dependencies[] = shortcut
dependencies[] = system
dependencies[] = taxonomy
dependencies[] = text
;dependencies[] = update
dependencies[] = user

; CiviCRM
dependencies[] = civicrm
dependencies[] = civicrm_clear_all_caches
dependencies[] = civicrm_cron
dependencies[] = civicrmtheme

; Custom
dependencies[] = abc

With this, all the core/custom/contrib modules get installed first except civicrm related modules. This way 95% installation process runs much faster and takes around 1 min and remaining 5% installation process which installs civicrm related modules takes around 2 min.

As I understand, Drupal installs must have modules first and then installs the remaining modules alphabetically. As abc.module comes before civicrm.module so this will get installed first. As abc.module has all other dependencies including core/contrib modules except civicrm related modules so those module also get installed second. This worked perfectly fine for me.

I need to repeat core modules at both places as the installation was not working correctly otherwise.