Hi there,

I recently installed Boost on one of my sites. I got it to work ok but then started getting WSODs when I was editing nodes and then trying to save them. At first it was for exceeding the maximum memory. So I increased the memory limit in the php.ini file and then the problem moved to the maximum execution time, even though I had increased this to 60 seconds. This didn't happen with all nodes but just some of them. I couldn't make out a pattern. Every time the Fatal error happened it pointed to the database.inc file. Part of the problem may also be that our Mysql database is about 160 Mb.

I did have a duplicate development site (on a different host) and wasn't having the same issue, though obviously it wasn't having all the traffic.

I tried troubleshooting it without success and finally contacted my hosts. They said they had been throttling the site which may have been causing the problem. However, when I tried disabling boost the problem was resolved. I think some of the queries being generated by Boost were causing the problem. We found one query alone for Boost that took 10 seconds to run.

I am not sure if this is really a bug but I thought it would be useful to report on.

Comments

ClaireC’s picture

ClaireC commented

Subscribing

nightlife2008’s picture

nightlife2008 commented

I'm having the same kind of problem here on a +100000 nodes database...

Note: My boost_cache table currently contains over 850000 records, of which 800000 are expired? Shouldn't this be cleaned up?

When updating a node, we found a boost query which took over 500 millisecs to run, after which the page goes blank (WSOD, memory limit exceeded).

The query in question is:

boost_db_multi_select_in()

SELECT * FROM boost_cache WHERE hash IN ('240ef9bc15e603a9bef07abe53f68850', '51debd0aee243782ec8375b3dd19dc32', '8189759ab237483d63a5956a1ab006cf', '240ef9bc15e603a9bef07abe53f68850', '51debd0aee243782ec8375b3dd19dc32', '8189759ab237483d63a5956a1ab006cf', '42b2154ff05a394f117ee9644e7db635', '4ba4819d06d0bd558c0c7cdcda08e439', '74862177541a89d0330376068f6f0835', '42b2154ff05a394f117ee9644e7db635', '4ba4819d06d0bd558c0c7cdcda08e439', '74862177541a89d0330376068f6f0835')

--> Took 586.51 MS to execute

Haven't found the actual source of the problem though...

Anonymous’s picture

Anonymous (not verified) commented
Assigned: Unassigned »

How many nodes do you have ? Also is the crawler on ?

Increasing memory and time in php in your scenario indicates that it was a problem that is between boost and "something" or boost alone. The move to needing to change the max execution time proves that as all that happened is that something is eating memory in a loop and you just gave it more memory to do so resulting in it taking longer to time out.

I got it to work ok but then started getting WSODs when I was editing nodes and then trying to save them.

This bit is interesting though, boost does not work on editing pages in version 6.x, in 7.x it adds the page to a cron run to remove it, but on 6.x there should be nothing related to boost as it should be disabled through admin. It could be that the editing is trying to flush the boost cache and you've so many pages in a queue (in general) that boost can't get through the load.

I can only guess at the moment, but from general experience and the differences between boost 6.x and 7.x I would think that the cache needs flushing and the crawler and all associated functions turned off. Let a search engine or anonymous users build the cache or as you have enough control to change the php.ini you could set a cron job using wget to build the cache. If you cannot flush the cache because of a time out then you'll need to go into the database and ftp/scp remove the files manually because you'll have identified the main fault, the problem will be preventing reoccurrence if it's not the crawler doing this.

joecorall’s picture

joecorall commented

I'm also experiencing this issue.

It doesn't happen every time you edit a node, maybe once out of every 20-50 edits or so.

The site has 20,000+ nodes. boost_cache has over 305,000 entries, 100,000 of which are for nodes. And these are my crawler settings:

http://d.pr/i/Cl4W

Same as @nightlife2008, the boost_cache table is filled with expired nodes. Looking at the stack trace below from one of these errors (and looking at the code called), you can see the timeout came while updating the boost_cache table via a call to boost_db_multi_select_in().

in array_shift called at /DRUPAL_ROOT/includes/database.inc (225)
in _db_query_callback called at ? (?)
in preg_replace_callback called at /DRUPAL_ROOT/includes/database.mysql-common.inc (40)
in db_query called at /DRUPAL_ROOT/sites/all/modules/contrib/boost/boost.module (5567)
in boost_db_multi_update_set called at /DRUPAL_ROOT/sites/all/modules/contrib/boost/boost.module (3579)
in boost_cache_kill called at /DRUPAL_ROOT/sites/all/modules/contrib/boost/boost.module (3464)
in boost_cache_expire_router called at /DRUPAL_ROOT/sites/all/modules/contrib/boost/boost.module (1876)
in boost_expire_node called at /DRUPAL_ROOT/sites/all/modules/contrib/boost/boost.module (1098)
in boost_nodeapi called at /DRUPAL_ROOT/modules/node/node.module (685)
in node_invoke_nodeapi called at /DRUPAL_ROOT/modules/node/node.module (944)
in node_save called at /DRUPAL_ROOT/modules/node/node.pages.inc (455)
in node_form_submit called at /DRUPAL_ROOT/includes/form.inc (781)
in form_execute_handlers called at /DRUPAL_ROOT/includes/form.inc (421)
in drupal_process_form called at /DRUPAL_ROOT/includes/form.inc (120)
in drupal_get_form called at /DRUPAL_ROOT/modules/node/node.pages.inc (14)
in node_page_edit called at ? (?)
in call_user_func_array called at /DRUPAL_ROOT/includes/menu.inc (350)
…cute_active_handler called at /DRUPAL_ROOT/index.php (17)
Anonymous’s picture

Anonymous (not verified) commented

To distinguish between "cron" the program on linux/ unix systems and "cron" part of drupal, I will refer to the latter as "drupal-cron"

This is what I imagine is happening in the background and suggests a solution. The sites are visited by mainly anonymous users. These people only see the cache and so never see a php page. Seeing a php page would trigger "drupal-cron" which would then attempt to clear out all the expired pages from the database, but this has grown so large that it takes many drupal-cron runs to clear it down.

This is suggested by the editing of the page which would be an action only carried out by a logged in user. To solve the problem permanently it would seem that one would need to assign a cron job to run "drupal-cron" as an authenticated user, thereby ensuring that the table never gets to a size where it exceeds the maximum amount of memory.

This theory comes mainly from experience where I have set up some sites that are essentially static in nature and after creation "drupal-cron" doesn't run even to notify the site's owners of required updates. The simplest of solutions is just to visit/ request the log in page, even when not logged in, the rewrite rules will ignore this page and present php and all it's associated triggers for "drupal-cron" will run. Sometimes a search engine will do the job, but not if it following something like the sitemap.xml or robots.txt modules where it is instructed to ignore the log in page.

joecorall’s picture

joecorall commented

Hey Philip,

Thanks for your suggestion, but I don't think that is what's causing the issue in my case. Drupal-cron runs on average about 7-8 times per day on the site in question.

It's as if boost 6.x just isn't clearing the boost_cache for expired entries. I manually cleared the boost cache and all static files this past weekend, and already I have over 6000 entries that are set to expire, but aren't being flushed on a drupal-cron run.