I've been trying to get purge working for a site at work, but while I had no problems getting purge+expire to work on a VM, I was unable to replicate this success on production. What I've come to discover is that purge requests are never sent on production, and here is the reason why:

https://bugs.php.net/bug.php?id=63411

According to this bug, the functionality of curl_multi_select() and curl_multi_exec() has changed for PHP versions 5.3.18 and above. It was also confirmed as an issue in 5.4.8, which was released at the same time.

The code here immediate returns, which is not what is expected previous to PHP 5.3.18: (@ purge.inc)

    do {
      $multi_result = curl_multi_exec($curl_purges, $active);
    } while ($multi_result == CURLM_CALL_MULTI_PERFORM);
    // Block until there is activity on any of the handlers. Avoids
    // busywaiting.
    if ($multi_result == CURLM_OK) {
      $select_result = curl_multi_select($curl_purges);
    }

Because of this, the purge_urls() function is returned an empty array, and no purge requests are actually sent out.

I've been working on trying to develop a patch, but so far, my challenge has been making a safe place to do so.

CommentFileSizeAuthor
#26 1892028-26-fix-curl-multi-select-bug.patch1.05 KBPieterDC
#20 1892028-20-fix-curl-multi-select-bug.patch969 byteshanoii
#19 purge-curl-multi-select-1892028-19.patch3.32 KBjmarkel
#16 purge-curl-multi-select-1892028-16.patch3.34 KBdjbobbydrake
#14 purge-curl-multi-select-1892028-12.patch3.22 KBhefox
#12 purge-curl-multi-select-1892028.patch3.23 KBFiremyst
#10 purge-curl-multi-select-1892028-10.patch3.23 KBFiremyst
#6 purge-curl_multi_select_php_version_issue-1892028-6.patch767 bytesmrfelton
#5 purge-curl_multi_select_php_version_issue-1892028-5.patch742 bytesrahulbile
#4 purge-curl_multi_select_php_version_issue-1892028-4.patch712 bytesjaydub
#3 purge-curl_multi_select_php_version_issue-1892028-3.patch710 bytesjaydub
#2 purge-curl_multi_select_php_version_issue-1892028-2.patch710 bytesjaydub
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

SqyD’s picture

SqyD CreditAttribution: SqyD commented

Thank you for this report. For the 2.x release I plan to have a more modular approach that would allow for curl replacements so implementing multiple curl versions should be an option too. I wonder if Guzzle will magicaly take care of these different behaviors in curl...

jaydub’s picture

jaydub CreditAttribution: jaydub commented
Version: 7.x-1.5-rc1 » 7.x-1.x-dev
Status: Active » Needs review
FileSize
purge-curl_multi_select_php_version_issue-1892028-2.patch710 bytes

I just ran into this myself on PHP 5.3.24. I looked up the PHP bug report noted above and there was someone's workaround that did seem to work for me. I've rolled a patch that does a PHP version check for supposedly affected versions of PHP curl_multi_select() which would be 5.3.18+ and 5.4.8+. With the patch my purge _did_ work so it would seem to be at least a temporary fix for those that are running into this.

jaydub’s picture

jaydub CreditAttribution: jaydub commented
FileSize
purge-curl_multi_select_php_version_issue-1892028-3.patch710 bytes

Rerolled as I only tested for > than 5.3.18 and > 5.4.8 rather than including those releases.

jaydub’s picture

jaydub CreditAttribution: jaydub commented
FileSize
purge-curl_multi_select_php_version_issue-1892028-4.patch712 bytes

One more time! I just submitted an unchanged patch :)

rahulbile’s picture

rahulbile CreditAttribution: rahulbile commented
FileSize
purge-curl_multi_select_php_version_issue-1892028-5.patch742 bytes

Here is the patch against 7-x.1.6.

mrfelton’s picture

mrfelton CreditAttribution: mrfelton commented
FileSize
purge-curl_multi_select_php_version_issue-1892028-6.patch767 bytes

Patch updated against latest 7.x-1.x dev code.

Firemyst’s picture

Firemyst CreditAttribution: Firemyst commented

We just pulled this patch into our QA env. It wasn't working, but I'm not sure why. When I tested on local VMs, I was able to make purge work with, or without this patch.

Any suggestions?

jaydub’s picture

jaydub CreditAttribution: jaydub commented

@Firemyst Since the patch is based on specific versions of PHP, would help to know what your environment is running...

Firemyst’s picture

Firemyst CreditAttribution: Firemyst commented

Of course. Here's what we have for apache and PHP:

Server:Apache/2.2.3 (Red Hat)
Via:1.1 varnish
X-Powered-By:PHP/5.4.15

It's where the usleep() should have went into effect, but I don't see how the patch works, since the state doesn't seem to leave state -1. There's no re-test before the exit condition of the do().

I have a task this week to actually convert my patch to a patch file. I'll post it here for review.

Firemyst’s picture

Firemyst CreditAttribution: Firemyst commented
FileSize
purge-curl-multi-select-1892028-10.patch3.23 KB

Here is the patch against v1.6

xavier_g’s picture

xavier_g CreditAttribution: xavier_g commented

Hello,

Regarding the patch attached to #10: I am pretty sure you meant:
$multi_result = curl_multi_exec($curl_purges, $active);
instead of:
$multi_result = curl_multi_exec($curl_pages, $active);

Indeed, applying the initial patch generates the following warning at purge time:
curl_multi_exec() expects parameter 1 to be resource, null given purge.inc:148

Firemyst’s picture

Firemyst CreditAttribution: Firemyst commented
FileSize
purge-curl-multi-select-1892028.patch3.23 KB

Yes, that's correct. Also solves a warning message I was getting, but only on QA servers.

Thanks.

hefox’s picture

hefox CreditAttribution: hefox commented
Status: Needs review » Needs work

Haven't tested it, but there's some minor spacing code stye issues with last patch

hefox’s picture

hefox CreditAttribution: hefox commented
FileSize
purge-curl-multi-select-1892028-12.patch3.22 KB

quickie attempt to update 12 to 1.x-dev

geerlingguy’s picture

geerlingguy CreditAttribution: geerlingguy commented
Issue summary: View changes
Status: Needs work » Needs review

This bug basically broke my local development environment when I was running PHP 5.4.x. I downgraded to 5.3.3 (CentOS default) and now things are working again.

The patch in #14 looks correct, but I can't test it right now :(

djbobbydrake’s picture

djbobbydrake CreditAttribution: djbobbydrake commented
FileSize
purge-curl-multi-select-1892028-16.patch3.34 KB

I've modified the patches from 12 and 14. This patch is rolled against 7.x-1.x-dev. I've tested this patch on 7.x-1.6 in my live environment and it's working successfully. I'm on php 5.3.28 and Varnish 3.0.5. I've wrapped the curl_multi_exec into a separate function.

djbobbydrake’s picture

djbobbydrake CreditAttribution: djbobbydrake commented

I'm going to have to amend the patch in #16, since it only supports 5.3.18+ and 5.4.8+. And will need to test in D6 and D7 in the different php versions.

Pinky Brain’s picture

Pinky Brain CreditAttribution: Pinky Brain commented

Same bug with PHP 5.5.8 and Purge module 7x-1.6. Patch #16 (tested with 7x-1.6+2-dev) works as bug fix.

jmarkel’s picture

jmarkel CreditAttribution: jmarkel commented
FileSize
purge-curl-multi-select-1892028-19.patch3.32 KB

I have rerolled the patch at #16 so that it will apply directly to 7.x-1.6. The one at #16 applies to HEAD on the 7.x-1.x branch but not to the released version.

hanoii’s picture

hanoii
Primary language Spanish
Location 🇦🇷UTC-3
CreditAttribution: hanoii commented
FileSize
1892028-20-fix-curl-multi-select-bug.patch969 bytes

I just faced the same issue and also follow it up a fix before looking into the issue queue (as most times!!), anyway, attached is my version of the patch, I think we did a very similar thing only I changed less lines of code so maybe simpler to review?

brianV’s picture

brianV CreditAttribution: brianV commented

I installed the patch from #19 above, and it did the trick on PHP 5.5.9. This approach is good, however, I haven't tested #16 (which applies against the main dev branch) as we are running the current stable release.

djbobbydrake’s picture

djbobbydrake CreditAttribution: djbobbydrake commented

Would someone be able to confirm that the patch in purge-curl-multi-select-1892028-19.patch works against php version lower than 5.3.18? If so, will close out this issue.

eugene.ilyin’s picture

eugene.ilyin CreditAttribution: eugene.ilyin commented

#19 helps me. Thanks @jmarkel

johan.gant’s picture

johan.gant CreditAttribution: johan.gant commented

#19 worked for me on 7.x-1.6 with PHP 5.4

PieterDC’s picture

PieterDC
Primary language Dutch
Location Ghent
CreditAttribution: PieterDC commented

#20 works for me. I haven't tried #19. Decided to go with the smaller patch right away.

Module version 7.x-1.6
PHP version 5.4.10

PieterDC’s picture

PieterDC
Primary language Dutch
Location Ghent
CreditAttribution: PieterDC commented
FileSize
1892028-26-fix-curl-multi-select-bug.patch1.05 KB

Rerolled patch from #20 against 7.x-1.6, made it comply coding standards and got rid of a temporary variable.

Vacilando’s picture

Vacilando CreditAttribution: Vacilando commented

On drush cc all I was getting

Invalid argument supplied for foreach() purge.inc:179                                              [warning]
WD purge: 1 error has been encountered when purging URLs Array                                     [error]

similar to what was reported in #1874448: Warning: Invalid argument supplied for foreach() in purge_logging() (line 167 of /www/www/htdocs/sites/all/modules/purge/purge.i.

Applied #20 and the problem went away. Could this (or #19 if more acceptable) be released to dev?

  • SqyD committed 2ceb2c7 on 7.x-1.x authored by PieterDC 
    Issue #1892028 by jaydub, Firemyst, djbobbydrake, PieterDC, hefox,...
SqyD’s picture

SqyD CreditAttribution: SqyD commented

Committed to 7.x-1.x branch. Thanks all!.

SqyD’s picture

SqyD CreditAttribution: SqyD commented
Status: Needs review » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.