This is a followup to http://drupal.org/node/1567422

All http requests have a 30 second timeout. It never changes regardless of $timeout. My Optimize takes longer than that.

The problem seems to be in the use of drupal_http_request

If I'm reading it correctly, we need to pass in a $timeout parameter or we get the 30 second default which is exactly what I observe.

I'm not sure how relevant or useful it is to set default_socket_timeout. Maybe we need both since drupal_http_request is using low level socket functions.

Comments

nick_vh’s picture

Status: Active » Closed (works as designed)

public function optimize($waitFlush = true, $waitSearcher = true, $timeout = 3600) {

The optimize function has a variable timeout function in D7 and D6.3. Please upgrade and help along with the new version. This is not a critical change for the 1.6 branch

icosa’s picture

Thanks. I will try to test a later version but I looked at the code and I don't think the problem is fixed.

Maybe I'm missing something but looking at Drupal_Apache_Solr_Service.php from 6.x.3.x-dev ...

Optimize calls $this->update($rawPost, $timeout)

Update sticks the timeout in the $options array and calls $this->_sendRawPost($this->update_url, $options);

_sendRawPost calls $this->_makeHttpRequest($url, $options);

_makeHttpRequest calls drupal_http_request($url, $options['headers'], $options['method'], $options['data'])

But drupal_http_request's signature is:

drupal_http_request($url, $headers = array(), $method = 'GET', $data = NULL, $retry = 3, $timeout = 30.0)

The only timeout it uses is that last parameter. It doesn't seem to use $options['timeout'].

jeffsheltren’s picture

Status: Closed (works as designed) » Active

We're experiencing the same error message as described in the referenced issue, http://drupal.org/node/1567422
"0" Status: Communication Error in apachesolr_cron

Since the optimize actually does run, you wouldn't think this would be much of a problem. However, due to the ordering of the code in apachesolr_cron()

$solr->optimize(FALSE, FALSE);
variable_set('apachesolr_last_optimize', $time);

If the optimize times out, the code drops out of the 'try' block and the variable_set never happens. This means that the optimize will run again with each cron run.

One option I've considered is to switch the order of those two lines of code. The benefit is, the variable would get set even if the optimize times out. The downside is the variable gets set even if the optimize fails completely. So I'm not sure if that's a good solution, but at the very least it prevents the optimize from being run repeatedly in our case.

pwolanin’s picture

We should probably remove the optimize on cron feature all together - that was partially a sop to people running their own servers who didn't want to manage a server cron job to optimize if needed. I think we also set the spellcheck index to rebuild on optimize, but we could probably rebuild that on cron instead.

jeffsheltren’s picture

I think that removing optimize is a fine option, perhaps giving a drush command to optimize for those that want it?

dpalmer’s picture

I believe we are experiencing this issue on one of our sites. We have firewalls between our environments, dev, qa and prod. When we migrate our dev site to qa, (along with the solr configs in the db), it fails to establish the connection to the dev solr server (cause of our firewall) and then the page on qa just hangs. Can't even get into the drupal admin pages to change the server, have to do it directly in the db...

theapi’s picture

It seems that $solr->optimize() is not the solution as it requires a connection to the Solr server. The problem I'm experiencing is the connection to Solr itself which has a currently unchangeable timeout of 30 seconds as set by default in drupal_http_request().

Dupal_Apache_Solr_Service::_sendRawGet() accepts an options array and passes that to drupal_http_request(), however Dupal_Apache_Solr_Service::search(), which calls _sendRawGet(), does not provide a mechanism to set any options such as a non default timeout.

Sorry I see this issue is for D6. My issue is with D7.

nick_vh’s picture

Status: Active » Closed (won't fix)

marking as closed and won't fix for 6.x-1.x - If this is applicable to 6.x-3.x or 7.x, please re-open and re-tag

larrylee’s picture

Version: 6.x-1.6 » 7.x-1.0
Status: Closed (won't fix) » Active
StatusFileSize
new1.84 KB

The apachesolr module is still timing out after 30 seconds in 7.x-1.0. This value is hardcoded as the default timeout value for drupal_http_request. I've created a patch that adds a timeout parameter to the Apachesolr admin page. The patch also increases the default timeout limit for solr requests to 10mins.

rares’s picture

I'm getting this error on 6.x-3.x on a solr index with 300,000+ items:
WD Apache Solr: "0" Status: Request failed: request timed out in apachesolr_cron

I determined that this is happening when running

$solr->optimize(FALSE, FALSE); 

in apachesolr_cron.

The timeout it's using is probably around 5-10 minutes, so it's quite large, so the timeout is not the issue, the logic is.
First, it is not possible to set apachesolr_optimize_interval anywhere in the UI. Secondly, it seems to be waiting for the response from solr even though optimize() is called with waitFlush and waitSearcher set to false. Third, I am not seeing anything in my Solr logs related to optimization. Does anyone know if the optimization is even called correctly?

amontero’s picture

There is a proposed feature with patch to expose the index optimization on cron in the UI at:
#2404857: Add index optimization on cron options to settings page

douggreen’s picture

StatusFileSize
new1.55 KB

Attached is an updated 7.x-dev patch that fixes style errors, removes the comment with the user's name (we use d.o and commit logs for that) and rerolls.

douggreen’s picture

StatusFileSize
new1.55 KB

Minor fix, add comma after array element.

See also #2475533: DrupalApacheSolrMultilingualService::_makeHttpRequest not setting timeout correctly. Always 30 seconds. for the same patch for apachesolr_multilingual.

douggreen’s picture

douggreen’s picture

retorque’s picture

Patch in comment #13 works for me.

ram4nd’s picture

Status: Active » Reviewed & tested by the community

Marking RTBC due to #16