Unavailable drupal.org leads to crashed cron

tomws - March 26, 2009 - 20:03

Project:	Drupal
Version:	6.10
Component:	update.module
Category:	bug report
Priority:	normal
Assigned:	Unassigned
Status:	duplicate

I had problems earlier today with crons getting stuck and failing. Depending upon the calling method, I would either get a 404 or a 'cron failed' message. Playing around with core's common.inc, I found that the hangs were the result of the call to module_invoke_all('cron') in drupal_cron_run(). That suggested disabling modules to find the culprit. Taking down update.module cleared my cron to run freely.

Conveniently enough, at the same time this was going on, I was unable to bring up drupal.org to help with debugging. This afternoon, drupal.org is flowing freely and my cron is working as expected with a re-enabled update.module, That leads me to draw the association that update.module is at fault here.

If drupal.org is slow or unavailable, shouldn't update.module's cron hook detect the issue and kill itself? Instead, it seems as if it currently freezes and cascades the error back up the call stack until crashing the original call to drupal_cron_run() (confirmed by first deleting both cron_semaphore and cron_last in the variable table, and then cron_semaphore re-appearing but cron_last missing). Unless I misunderstand the flow, this would cause subsequent modules with cron hooks to not get their chance to run, thereby killing a potentially significant number of necessary maintenance tasks just because of a slow or unavailable drupal.org.

If this is indeed the case, it presents itself as a possible attack vector through the use of a DoS against drupal.org and keeping thousands of Drupal-based sites from being able to successfully complete their maintenance tasks.

A slippery slope argument might be fun, but I'm really more interested in just having the crons not crash. Can anyone confirm this problem or else point out to me where my reasoning is incorrect?

» Login or register to post comments

#1

bedlam - March 26, 2009 - 22:26

I can confirm this (also based on drupal.org's flaky performance today). At the root, this seems to be a problem with either or both of the following:

Updates's _update_refresh() function, and/or
Core's drupal_http_request() function

_update_refresh() contains the following loop which calls drupal_http_request():

<?php
    foreach ($projects as $key => $project) {
    $url = _update_build_fetch_url($project, $site_key);
    $xml = drupal_http_request($url);
    if (isset($xml->data)) {
      $data[] = $xml->data;
    }
  }
?>

Unless I'm misreading it, drupal_http_request() then attempts to parse the http header received in response to the request. However, if no request is forthcoming, what happens?

I don't have a specific implementation to offer, but I'd suggest that _update_refresh() or drupal_http_request() should time itself out, or check that drupal.org is responsive before initiating the whole http-request-response cycle.