Installation failed w/ max execution time of 30 seconds

Perhaps you can provide more info, in which step, what config etc?

For me install never had such problems. I'm having Apache+MySQL+php+FireFox all on the same machine, which is a *VERY* slow, old desktop, limit 30 secs. And still OK. So there must be something different in your config?

I have a test environment setup on a P4 3GHz laptop w/ 1.25 GB of RAM. I'm running Windows XP Home w/ the latest Wampserver and Firefox 2.whatever. Without adjusting my php.ini, install of Drupal 6 failed a couple of times by reaching the timeout limit. I adjusted the php.ini and this was fine. (FYI, it was taking about a minute, maybe a little more, to do the install.)

I decided to roll a patch that instead ups the max_execution_time for a profile install using ini_set() as described in this issue for extending the execution time for another time consuming operation.

Please advise on the patch file's format. This is my first go generating one with Tortoise CVS.

Follow up on this patch. I should've nailed down the context for the fix. Previously it was in drupal_install_profile() in install.inc, but as it turns out, the configuration step in installation was failing due to a timeout also. I've updated the patch to move the code to what seems like the appropriate place... the conf_init() function in bootstrap.inc which gets called for DRUPAL_BOOTSTRAP_CONFIGURATION. This seems like the place to put it since the setting would otherwise be adjusted by a settings.php file. As far as I know, this should cover any step in installation, but I've only tested with the default install profile.

Tested the fix on my localhost again and it worked like a charm.

Upping max_execution_time is not a reliable solution, unfortunately, since many shared hosts won't allow this. The new D6 batch API has been added precisely to adress these kind of issues.

It's surprising that a default drupal install should timeout. It would be best if we could find out what takes so much time and why. Are you using a custom install profile ?

@rszama: you experience this issue, so finally please tell us on what step you find this timeout.

This is probably highly platform-dependent: My test install is on Pentium III / 500MHz (!) / 512MB RAM and the install never failed, no step ever gone above 4-5 seconds. (Linux Mandriva One 2007 / Apache / php 4.4.4 / MySQL 4.1.x / FireFox 2 - sorry don't remember more info, not sitting at THAT machine now.)

I also agree that a solution should go the batch-API way. My site is on a good (I believe) paid hosting, but still, setting the time limit is NOT allowed for us.

The installation failed on the laptop I was using (P4 3 GHz, 1.25 GB RAM, Win XP Home, latest Wampserver) multiple times last night when submitting the form at "Setup database" and at "Configure site". I would time out at either step unless I used the patch I posted above.

Granted, it may be a fluke of that system. I'll try again on my desktop tonight. Using a Wampserver here at work it still took quite a bit of time for the install to happen, but I didn't time out even w/ max_execution_time set to 30 seconds.

I didn't have any trouble last time I installed on our Linux server.

I have the same problem on my macbook pro, windows XP wampserver. Also I get another error related to memory size:

Fatal error: Allowed memory size of 8388608 bytes exhausted (tried to allocate 16 bytes) in C:\wamp\www\d6dev\includes\menu.inc on line 1957

I have changed the memory size limit and the execution time limit and every thing when fine. I think this is related to wampserver configuration.

I can also report the max execution time error problem on a fresh install on a local machine using Windows XP (using WAMP), with nothing else running and PHP set to allow 32 MB of memory. An error like this on a local installation where nothing else is running feels pretty critical to me, since I have to think it will happen even more frequently in real world installations.

I get the following error and the installation times out. I can refresh the page and the installation continues and seems to finish with no errors.

Fatal error: Maximum execution time of 30 seconds exceeded in C:\wamp\www\drupal\includes\database.mysqli.inc on line 147

I altered the code to output the queries it is running to see where it ran out of time, and it is during the creation of the menu navigation. In my case, the last query that ran before the error was the following, which was the last in a very long list of queries related to the menu system (and it was generated by the same line 1957 of menu.inc mentioned in the above post):

UPDATE menu_links SET menu_name = 'navigation', plid = 18, link_path = 'admin/settings/site-maintenance', router_path = 'admin/settings/site-maintenance', hidden = 0, external = 0, has_children = 0, expanded = 0, weight = 0, depth = 3, p1 = 1, p2 = 18, p3 = 50, p4 = 0, p5 = 0, p6 = 0, p7 = 0, p8 = 0, p9 = 0, module = 'system', link_title = 'Site maintenance', options = 'a:1:{s:10:\"attributes\";a:1:{s:5:\"title\";s:63:\"Take the site off-line for maintenance or bring it back online.\";}}', customized = 0 WHERE mlid = 50

I'm not sure exactly what to do next to keep things from timing out.

Karen, does my patch above fix it for you?

A concern was raised about this not solving the problem reliably because shared hosts won't allow this... but it just struck me that this shouldn't affect them at all as PHP will simply ignore the function and they aren't experiencing the problem. At the same time, it should solve the issue for folks who do experience the problem in a WAMP setup (which arguably isn't a lot of us, but more people keep finding this issue).

Maybe the concern was that the query should be fixed instead of just accommodated.

I think the issue is that we need to do the update more elegantly, not try to sledge hammer the timeout value. So far we've only heard from WAMP installations, but I doubt anyone has tried to install D6 in a limited shared host environment where altering the timeout isn't allowed.

We have a way to batch process things, which is probably what is needed here.

Well, an attempt of set_time_limit() might be a bit of help somewhere, but I believe there is a *lot* of hosting services not allowing this (including my one), and then users will get a nasty red error message (which is automatically treated as show-stopper in some parts of the installer!) So:

- If attempting set_time_limit(), we should check whether it's allowed first. There are already three instances of the same in Drupal, but insufficient still. See my patch at http://drupal.org/node/193383 for more related info.

- I'll definitely put it into install.php only, NOT into the main bootstrap. No need to risk issues with disabled functions on runtime; if regular pages took 30secs, then Drupal is completely unusable on such environment.

- Still, it's just a dirty workaround. At first, we need to identify, what takes *so much* time on WAMP/MySQLi (or whichever bit of the config is important). As I already stated, LAMP/MySQL 4.1.19/500MHz CPU is below 5secs. This difference is NOT by design or natural IMO.

Edit: My testing environment is with set_time_limit() disabled. This is on purpose - although I can do anything I want on my desktop-localhost, I copied my shared webhosting config as precisely as possible (because this install is in the first place for testing things for the production site, before going live). Drupal 6 installs fine on that.

Hm, we can use the new batch system to run the install hooks in a batch. That would also help arbitrary install profiles coming in the future, which might need to run three to five times as many install hooks and install more tables. Currently the installer installs all modules in one big sweep. The batch API can be used as soon as the system module is installed though, so getting that first and then running the others in a batch could solve this problem.

It will be no fun if menu_rebuild times out on WAMP. WAMP people, please instal somehow and then set back to default and visit the admin/build/modules page. Does this page time out?

I've got latest HEAD installed and I just double-checked the admin/build/modules in WAMP. It is very slow, but it did not time out.

For some reason it seems to be only when building the menu during the install that there is a problem.

I know menu_rebuild would not win any race but "very slow" -- what's going on there? Does mysql slow log show anything curious? Anyone on WAMP wants to install xdebug and profile? http://www.sitepoint.com/blogs/2007/04/23/faster-php-apps-profile-your-c...

I started working on a patch to install modules in a batch as suggested by Gabor in #13.
Works rather fine, not quite ready yet, but I'm not even sure this will fix the problem at hand here - I can't really seem to figure exactly when the troublesome menu_rebuild() gets triggered (my debugger seems to have a configuration issue and misses the specific page load where it happens...). Are we even sure it's a menu_rebuild() ?

FWIW, I'm using WAMP myself, and never experienced the reported timeout.

yched, i would not work on changing the instller at this point into batch api,that's a lot of work and not sure it'll solve our pbolems. let's hear from the wamp folks.

chx: installing the modules one by one in a batch might not be that big of a code change, but we would only see this if someone tries sure

I have no time right now to try installing and testing the debugger, hoping someone else experiencing this problem can try that.

Here's a patch that installs modules in a batch. System gets installed first, then we install all other modules in a batch.
Still a few missing PHPdocs, but should be ready for review. Careful testing and expert eyes needed, of course - this is my first incursion in install land...

I had the patch mostly ready, and as Gabor stated, it might be a good things for 'rich' profiles anyway, so I post it here. I do agree with chx that this does not save the need to investigate why the initial menu_rebuild seems to take so much time on some configurations...

I tried yched's patch out and it solved the problem for me. Yched said it may also fix the issue at http://drupal.org/node/164458, and as noted above there may be other complex installations that would benefit.

On the original timeout issue, I just realized that I only have the timeout issue on one of my two WAMP installations. I tried to dig through all my configuration code to see what the difference might be and the main difference I am down to is that the installation with the timeout problems is on XP Home while the one that works fine is on XP Professional. I see a mention above that another installation that was timing out was on XP Home, so maybe it is something in XP Home that is an issue. No one would be using that for anything but a local installation, it would certainly not be used as a web server. So if that is the actual problem, we're probably OK to not dig into this any further.

Since this looks like a good patch for all the reasons noted above, *and* it fixes this problem for the few that will have it, I would definitely +1 applying it.

I'd mark it RTBC, but since this affects a lot of installation code, it would probably be good to get one or two others to try it out and be sure nothing broke.

Off topic note: If I did not before, now I surely hate Microsoft for all their fine products.

yched, KarenS: this patch looks like a proof of concept, contains quite some todos and stuff to do and discuss. As I posted at http://drupal.org/node/164458#comment-646123 in reference to this patch:

KarenS: it does not seem like that patch *loads* modules after they are enabled, it just separates enabling them one by one into a batch, so it might make the situation even worse (as it is now), with the previously installed modules not loaded yet, so the API functions are not usable still.

In general, it would be nice to asses the impact of this issue, and concenrate our powers in fixing it if:

- the issue is indeed critical and more widespread then Windows XP Home users
- breaking down the installer to smaller pieces really improves performance (ie. one or two install functions are not timing out in themselfs anyway)
- we can do it cleanly so late in the cycle

@Gabor : the TODOS are PHPDoc, and a minor point I still need to check (but won't be able to before a couple days) about soem drupal_set_message being actually displayed or not.
Both points do not need to stop this patch being reviewed and tested and evaluated - the sooner the better, right, it is late, and this is the installer...Aside from those two points, the patch is ready (as a candidate, I mean)

@gabor - this specific issue may be limited, and possibly therefore not critical, but it appears the fix will correct some things that *are* critical, like the problem with the book installation. More importantly, it's too early in the cycle to know what kinds of problems complex profile installations will run into during installation (since the many of the modules they use aren't ported yet), but if we wait until then to see if we have installation problems that could be solved by batching the installation process it will be very late indeed in the cycle to fix them.

This need a reroll at least in the light of the installer theming fixes committed recently.

Ah, right. Unfortunately, I'm sort of on the road, I won't be able to do that before a couple days, sorry. Er, Karen, maybe ? :-) Looks like a simple reroll.

Here's a re-roll. I also tried to fill out the missing PHP Docs, I left in the other TODOs.

I get this error after clicking "Install Drupal in English":

Fatal error: Call to undefined function: user_access() in .../test/includes/theme.inc on line 1627

This is caused by the user_access('access administration pages') check in theme_preprocess().

It should not be using theme.inc, it should be using theme.maintainance.inc. It looks to me like the problem is that the maintainance theme changes now require that a constant 'MAINTENANCE_MODE' be set to 'install' to tell what theme to use. That constant is set in install.php, but our batch processing loses that constant in the http request processing going on in the background.

Not sure how to fix this though...

Never mind my previous comment, that's not exactly right. I have tried to follow through the new code that uses themes in the installer to see where we need to intervene so it works correctly with batch processing, but can't figure it out.

Inside template_preprocess() where it does the db_is_active() check.. An additional check of !defined('MAINTENANCE_MODE') will fix it.

Not sure mudding it up further is right since it runs pretty often, but here it is anyway. When I altered that function, I considered letting the theme pick up an alternate version of template_preprocess but it would have ended up being more complicated + duplicate code.

That constant is set in install.php, but our batch processing loses that constant in the http request processing going on in the background.

Oh, and if this is still a problem, can that constant be reset from the file doing the http request?

As you noticed, that constant just flags the theme to use. But it also prevents db reads by the theme since some tables may not be available yet.

Only used from list_themes() and _drupal_maintenance_theme(). The constant is also set from update.php to minimize the chance of reading outdated db info while updating.

Tried this change out and it installs fine for me. I was speculating on there being a problem with the constant, but maybe that is not an issue after all.

I did the following:

1) Confirmed the user_access error when trying to install using the patch in #29 and the latest HEAD.

2) Made the change in #33 (a one line change in the patch to fix theme.inc).

3) Installed with no errors and no timeouts.

4) Added the book module to the module list in the profile, and tried another clean install.

5) Installed again, with the book module, and got no errors and no timeouts.

subscribing.
I posted a possibly related issue http://drupal.org/node/197720 which deals with the 8M memory_limit fatal error. This patch doesn't fix it.

KarenS: can you roll an updated patch with the code you found working?

Here's a re-roll, brought up to date. I saw there were a few other changes since my original test, so I re-tested it with the book module added to the list, and it installed successfully with no timeout.

With regard to the 8M memory limit problem, this patch breaks the module installation out into individual batches, but does not break up the core install itself. Finding a way to break the core install into a series of batched steps might be another thing to try, once this patch is applied, assuming the batch engine is available early enough to be of some good.

I don't begin to understand the installation process well enough to propose anything on that, just making a comment in case someone else does.

I read up on the entire thread and:

1. I'm still a bit surprised that this fixes the problem. It feels like we "just got lucky" (just below the timeout) but I might be wrong. Before we commit this patch, I think we need to get more "success reports" for this patch (i.e. from other people affected by this problem).
2. menu_rebuild() continues to be "really slow"
3. The !defined('MAINTENANCE_MODE' check is a bit ugly, IMO.

Dries: installing 1 module at once or 10 to 30-40 modules (in an install profile) at once when compared with the time it takes, is not only about getting lucky or not. It is not impossible that one module install might take 30 seconds, but I would not say it is likely.

I'm testing with lighttpd + pgsql, 8Mb of Memory - 10 sec max execution time, and 1st time failed (on menu.inc) and the 2nd time It worked fine.
I'm using an accelerator for PHP & lighttpd.

There are several different problems being referred to here and this patch is not necessarily going to fix all of them. And I'm not sure it's necessary to create an install that works in an environment that has a 10 sec max execution time (as in #42). I think it's reasonable to assume a minimum 30 sec timeout since that's the default PHP setting (at least I think it is).

The original issue is a timeout in environments that are set to a 30 sec max execution time. I had one of those situations and this patch fixes that problem for me. No one else who has that problem has reported back whether or not it fixes their problem, so we don't have a conclusive answer on that.

However, a different critical issue is also fixed with this patch, http://drupal.org/node/164458, where fatal errors occurred if you install the book module at the same time you install core. This patch fixes that problem by batching the installation of separate modules and enabling each one once it's installed. That issue was marked as a duplicate assuming that this patch would get in to fix that problem.

A third issue referred to here is that systems with only 8 MB of memory sometimes fail before finishing the installation. There is already a report that this patch may not fix that problem, and that issue remains open.

This patch fixes the book installation issue, and possibly will avoid other future problems that might come up as complex installation profiles are created, by separating the installation out into batches, and it appears, so far, to fix the original issue. It won't fix every possible installation problem and we shouldn't hold it up for that reason.

As to 'getting lucky' that this works, I think what we're seeing is that the core installation needs all of the 30 sec timeout window to run in some environments. So long as we push all other processes into batches, we're OK, but we can't try to do the core installation *and* install other things all in a single pass.

It's possible we'll find that the core installation itself also needs to be broken out into batched steps to make things work even more reliably, as I noted in #39, but getting this much done is a great first step. And I don't see any downside to committing this patch.

Karen, this patch actually breaks out the core installation itself (as in all the different modules installed by default) into a batch, only the system module is installed first out of the batch, so that the batch environment is in place and can be used.

regarding Dries' concerns in #40 :
menu_rebuild() is probably slow, but as chx pointed, refactoring it this late in D6 cycle promises to be problematic. Besides, it seems only a small subset of configs (some setups with win xp home ?) are impacted. FWIW, menu_rebuild() takes around 750 ms on my setup.

This patch adresses the most critical aspect, which is at install time. It's the only occurrence where it has been reported to actually cause a timeout WPOD, probably because it comes after a bunch of other processing.

I don't think the patch is being lucky in fixing this - during the install process, the first call to menu_rebuild happens in menu.module's menu_enable(). If this one gets longish, it has much more chances of triggering a timeout in a HTTP request that is supposed to install all of system, required modules and profile modules, than if it stands alone in its own request.

Using a batch here is all the more required that http://drupal.org/node/164458 showed that modules should be installed and enabled one by one, instead of installed one by one and then enabled in one pass as they currently are. This takes more time, since module_list() and module_implements() are refreshed for every module instead of only once.

(Not repeating the other benefits Gabor and Karen already pointed :-)

About the !defined('MAINTENANCE_MODE') check, it is already used by list_themes(), so I guess this is consistent with the logic introduced by the 'themable maintenance pages' patch. If this is to be changed, this could probably be left to a different issue.

PS : the failed install reported in #42 is with a 8Mb memory limit. I'd say this is a different bug, discussed in another thread.

Phew. Another issue which hit the critical queue a few minutes ago also has some parts solved with this patch. The problem there is the hook_requirements() are not callable in install time on modules (due to module_invoke() not being available outside of the bootstrap), so their requirements cannot be checked in an install profile. With installing all modules (except system) in a fully bootstrapped environment, this is also covered.

So, let's get this issue fixed!

- I notice one todo on the messages which is not yet solved
- I notice a whitespace issue in _install_profile_batch_finished()

Let's solve these. The reviews on the behavior of this patch so far are positive, so let's polish this up.

CNW for the todo/whitespace.

Are you talking about the internal TODOs, whether messages are OK and about merging the locale import? Those were yched's questions, so I don't want to do anything with them until yched can respond.

Or was there some other TODO?

BTW, when I talked about breaking up the core install, I was talking about breaking up the system install, which still does take a while to run. No need to hold up this patch for that, of course, it was just a comment.

Merging the locale install complicates stuff to much IMHO. Let's focus on solving the internal TODO, the one about the messages.

About the TODOs :

- merging locale import :
yep, that can be considered in a later patch. No big deal.

- drupal_set_messages about settings.php :
1st thing is I'm not sure of the correct timing for drupal_verify_install_file and the messages. Before the patch, they are issued after all modules are enabled. I'm not sure about the constraints here.
2nd thing is drupal_set_messages seem to be lost when issued right before or during a batch (see http://drupal.org/node/196630). So the messages are also lost in current HEAD with non-english installation. I need to investigate this. Batch progress pages intentionally do not display messages, but I would have expected they get displayed after the batch is finished.
In the meantime, is it OK if we move this part (drupal_verify_install_file / messages) right before the configure screen is called ?

Trying to figure out why some the drupal_set_messages are lost. More complex than I expected - I'm not fully getting the maintenance / install theming logic yet.

Small question : is there a reason why pages displayed by install_tasks() use print theme('maintenance_page', $output);, while all other pages generated by install.php use print theme('install_page', $output); ?

yched, I got burnt too. some themed pages don't display certain types of message, for example, print theme('maintenance_page', $output); will not display the warning messages. use

$messages = drupal_set_message();
var_dump($messages);

to check what messages are actually available. hope that can help.

OK, here's an updated version that should be ready. Here's the deal :
Having messages displayed on batch progress pages is not a good idea, since those pages disappear after a while without any user action.
Having messages defined before a batch is set but displayed only on the next (non batch-progress) page is beyond me for now.

Therefore, in this new patch, the check about the settings.php file and folder, and output of the relevant message, are moved to the 'site configuration' form, after both modules install and locale import batches are done (In current HEAD, this happens after profile install, but then the message is lost if we're doing a localized installation and a batch locale import is started).

No more TODO's, so back to CNR.

PS : the patch additionally fixes a typo in theme_install_page, causing the text prefixed to 'status' messages not to be displayed.

@scor : print theme('maintenance_page', $output); will not display the warning messages
you mean install_page, not maintenance_page ? Yeah, I saw you had to fix that in your patch for http://drupal.org/node/197720 :-)
This is not what is happening here, though.
Anyway, latest patch should be OK.

@yched, yeah sorry, I meant install_page.

scor: so why does it need work?

sorry! of course this needs review. I reloaded the page in firefox which forced the status to what it was before reloading (CNW).
I tested the patch #57 successfully, although it's so fast that all I can see is the progress bar reaching 100% immediately. will try to do more thorough testing.

I tested this patch on a second shared hosting server OK. I read:
Initializing.
Remaining 0 of 10 modules.
Installed database logging module.
Then and only then the progress bar changes from 0 to 100%. (delay maybe due to js?).

Well, I sat down to test this throughly, and all works fine as I expected. I run an install with core only and one with translations too. Messages displayed where they should (previously messages were eaten before the locale batch actually - even forgot to mention this in the commit message, doh). All worked fine. I had one gripe from above: there was a bogus whitespace. Also the patch did not apply cleanly after I committed the code cleanup patch (this one also contained a few cleanup stuff). Rerolled it for testing with the whitespace fixed. Committed the attached patch.

scor: I would say that maybe the image was loading late.

For reference, the hook_requirements issue I mentioned is at http://drupal.org/node/110981 .

Since I am new at this, can someone explain how I can apply the patch? or if the patch will be included in the next update?

In #64, Gábor (the Drupal 6.x maintainer) said, that he committed the patch - this means that all Drupal releases from that moment on contain the change by default. And looking at the date, this means that Drupal 6.x series is fixed from the very begin (6.0). That's also why this issue (thread) is "closed" now, and (nearly) no-one watches new comments here. This is just a history, kept for future reference. If you've any similar problems, you should open a new issue (if there's not another one active already).

Got this error while trying to install new modules on my new Drupal 6.2 Installation. (Newbie here!)

Solved moving the new modules to "/sites/all/modules" folder instead of "/modules" .

Hope this helps to people that, like me, find this post about this error via google.

I have the same issue in installing the Drupal 7.26 on through the wamp server.I get this Fatal error in database.inc on Line 2168 no after entering the system requirements(Database name,Database User name).
Operating System:-Windows 7 64-bit
Drupal Version :-7.26
Server:-Wamp Server2.4

We need to know how to replicate this @Gui-

Let us know and we can try and address it. It might make more sense to open up a new issue though. This one was 6 years old.

@Gui If you're having difficulty installing Drupal then please open a new Support Request issue or better yet seek help in our support forum, https://www.drupal.org/forum/. I can't be certain since you didn't post a lot of information, but it didn't even sound like a server timeout problem which is what this issue was about.

Aww, I was so cute 7 years ago.