#1807662: Built-in APCu support in core (PHP 5.5 only) adds APC as a cache backend in core, and of course, a test for it.

The test class implements checkRequirements() already in order to prevent it from running when APC is not enabled, too old, or when the test is executed from PHP CLI (run-tests.sh) but apc.enable_cli is disabled.

However, checkRequirements() actually leads to a failing test case, and does not cause a test to be skipped.

I wonder whether we could enable apc.enable_cli on the d.o testbots to have the test executed as part of the core test suite?

Technically, I don't see why that should cause problems. But if it does, we could inject an enforced cache flush into run-tests.sh to make sure it doesn't run on stale data.

CommentFileSizeAuthor
#18 rfay_screenshot 2012-10-16 at 7.23.08 AM.png103.73 KBrfay
#18 phpinfo_cli.txt28.5 KBrfay
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

rfay’s picture

rfay
Primary language English
Location Palisade, CO, USA
CreditAttribution: rfay commented

I don't see why not. Will try to do this when I increase APC memory for #1808260: Turn off testbot access logging and increase APC space

andypost’s picture

andypost
he/him
Primary language Russian
CreditAttribution: andypost commented

There was a lot of troubles with accelerators, suppose it needs additional step in testing after applying new patch to clean-up apc cache

rfay’s picture

rfay
Primary language English
Location Palisade, CO, USA
CreditAttribution: rfay commented
Project: Drupal.org Testbots » drupaltestbot-puppet
Status: Active » Needs review

This change is deployed on #654. Let's see if it causes any trouble. (I just copied the new apc.ini up there and restarted apache)

Committed in http://drupalcode.org/project/drupaltestbot-puppet.git/commitdiff/9d0f472

sun’s picture

sun
Primary language German
Location Karlsruhe
CreditAttribution: sun commented
Status: Needs review » Needs work

The ini setting is "apc.enable_cli" though:
http://php.net/manual/en/apc.configuration.php#ini.apc.enable-cli

rfay’s picture

rfay
Primary language English
Location Palisade, CO, USA
CreditAttribution: rfay commented
Status: Needs work » Needs review

Whoops, sorry. Glad SOMEBODY is paying attention!

http://drupalcode.org/project/drupaltestbot-puppet.git/commitdiff/769e3e41

Deployed on #654 and #813

sun’s picture

sun
Primary language German
Location Karlsruhe
CreditAttribution: sun commented

Yay! ;)

Hm. Looks like the redeployment caused those two bots to stalemate while they were busy testing patches?

654 Enabled - Reviewing
tim.plunkett's sandbox: VDC - vdc
Environment: 2
48 min 14 sec (Thu, 10/11/2012 - 03:24:03)

813 Enabled - Reviewing
bundle-registration-1786990-21.patch
Environment: 1
49 min 15 sec (Thu, 10/11/2012 - 03:23:02)

(essentially waiting for results that won't ever come back, I guess)

sun’s picture

sun
Primary language German
Location Karlsruhe
CreditAttribution: sun commented

I sent the two bots for client re-test, and also requested a re-test for the two stale tests afterwards. Hope that resolves it.

andypost’s picture

andypost
he/him
Primary language Russian
CreditAttribution: andypost commented

I see no visual speed-up on the bots with apc

sun’s picture

sun
Primary language German
Location Karlsruhe
CreditAttribution: sun commented

That's expected. Performance is not the reason for enabling apc.enable_cli.

In fact, enabling APC on CLI might actually slow down run-tests.sh in a minimal way, since a new cache is created and destroyed in memory for every PHP process that runs (and run-tests.sh spawns a new process for each test case class).

I'll try to re-test the APC cache backend patch manually on QA until it hits one of the bots that has it enabled and will report back whether it worked.

rfay’s picture

rfay
Primary language English
Location Palisade, CO, USA
CreditAttribution: rfay commented

Any reason we shouldn't deploy this on the remaining testbots?

podarok’s picture

podarok
🇺🇦 he/him/his
Primary language Ukrainian
Location Ukraine
CreditAttribution: podarok commented

any reason for enabling this without testing alternatives (FE that is much more preffered xcache for example with integrated single caching storage for duplicated files)

rfay’s picture

rfay
Primary language English
Location Palisade, CO, USA
CreditAttribution: rfay commented

@podarok, this isn't about which PHP cache to use. We've been using APC for years and years. This is about changing one apc setting.

sun’s picture

sun
Primary language German
Location Karlsruhe
CreditAttribution: sun commented
Status: Needs review » Reviewed & tested by the community

I've monitored the test results of the two affected bots in the past days and didn't see any problems.

So yeah, let's deploy to the others. :)

sun’s picture

sun
Primary language German
Location Karlsruhe
CreditAttribution: sun commented

That said, for full disclosure, the APC cache backend patch actually failed with a fatal error, which I can't reproduce locally. I wasn't able to determine the cause for it yet, but I guess it must rather be something with the patch.

rfay’s picture

rfay
Primary language English
Location Palisade, CO, USA
CreditAttribution: rfay commented

Let's wait to deploy until we see what your take on that is... Monday maybe.

rfay’s picture

rfay
Primary language English
Location Palisade, CO, USA
CreditAttribution: rfay commented
Status: Reviewed & tested by the community » Fixed

Deployed on all

sun’s picture

sun
Primary language German
Location Karlsruhe
CreditAttribution: sun commented
Status: Fixed » Needs work

Ahem, sorry, I don't know why I didn't look at the review log right when I mentioned the error... stupid me ;)

/usr/bin/php: symbol lookup error: /usr/lib/php5/20090626/apc.so: undefined symbol: pcre_exec

FATAL Drupal\system\Tests\Cache\APCBackendUnitTest: test runner returned a non-zero error code (127).

So the extension cannot be loaded, since there's a header/symbol mismatch for PCRE.

Searching the net for that error shows plenty of results, commonly talking about

1) PCRE is bundled with PHP 5.3.0+ and should be always enabled -- but custom builds may have force-disabled it.

2) APC binary might have been built with wrong PHP headers or wrong PHP version (are there multiple PHP installs on testbots?)

http://forum.nginx.org/read.php?3,153213
http://serverfault.com/questions/337575/apc-does-not-load-when-calling-p...
http://forums.freebsd.org/showthread.php?t=26850 (somewhat Drupal-related)
http://atomicorp.com/forums/viewtopic.php?f=2&t=4025

Based on these threads, APC should actually not work for Apache either - you'll most likely find PHP startup errors in Apache's error.log.

That's not related or caused by the new apc.enable_cli setting though.

rfay’s picture

rfay
Primary language English
Location Palisade, CO, USA
CreditAttribution: rfay commented
FileSize
phpinfo_cli.txt28.5 KB
rfay_screenshot 2012-10-16 at 7.23.08 AM.png103.73 KB

Apache is not having any trouble at startup; a view of the phpinfo looks sane. This is 654, the only one I looked at. Maybe you can tell me how to recreate the issue you show.

php -r "phpinfo();" works fine; attached.

Edit: I do note that php-apc is not listed in the pinned files for php5.3.5, so it's possible we're getting php-apc from Debian and php5-* from dotdeb. The bit we've done by putting the php5.3.5 packages (from dotdeb) into our own repo could easily cause this kind of behavior.

rfay’s picture

rfay
Primary language English
Location Palisade, CO, USA
CreditAttribution: rfay commented

@sun, all I need is a way to recreate this and I'll get the correct package and fix this issue. If you need access to a testbot I'll set one up for you. Although, as knowledgable as you are on this whole subject, I sure hope you have your own in a virtual machine or something. Creating a drupal testbot explains how.

rfay’s picture

rfay
Primary language English
Location Palisade, CO, USA
CreditAttribution: rfay commented

Should I assume this is no longer important?

catch’s picture

catch
he/him
Primary language English
CreditAttribution: catch commented
Issue summary: View changes

I just opened a duplicate #2250211: Enable apc.enable_cli for core APC cache backend tests because that issue is actively being worked on again.

sun’s picture

sun
Primary language German
Location Karlsruhe
CreditAttribution: sun commented

We should additionally adjust PIFR (or run-tests.sh directly in core?) to force-flush the entire APC cache prior to every test run.

— Otherwise, it's pretty much guaranteed that we'll run into major APC hiccups: The external storage will either run out of space, or random test failures due to a lack of uniqueness and/or test entries that weren't cleaned up due to fatal errors.

rfay’s picture

rfay
Primary language English
Location Palisade, CO, USA
CreditAttribution: rfay commented

I'll leave this alone until you say exactly what you want, and do the issue to adjust pifr or run-tests.sh.

catch’s picture

catch
he/him
Primary language English
CreditAttribution: catch commented

If we going to flush the cache globally, we should do it in run-tests.sh. That's not a bot-specific problem.

Anonymous’s picture

Anonymous (not verified) CreditAttribution: Anonymous commented

discussed this with catch, sun, berdir et al in IRC:

- we don't need to worry about cli APC cache, because it doesn't survive the test run
- as long as the prefix used in the APC backend is random across tests, we don't need to worry about 'leakage' between runs
- we do need to care about the APC cache getting too large over time. not with the APCu backend patch, but later on when it becomes the default and it is used for webtests. even if we write our code to clean up after itself, bugs/fatals etc will eventually eat up the available memory

summary:
- enabling APCu for cli to unblock #1807662: Built-in APCu support in core (PHP 5.5 only) should be safe in the short term
- we will need some sort of garbage collection on bots to handle the APCu cache in the medium term

a question for rfay/jthorson:
- can we schedule an apache restart every N test runs? that would effectively remove any leftovers...

catch’s picture

catch
he/him
Primary language English
CreditAttribution: catch commented

If the bots use standard logrotate for apache, it'll get restarted whenever that happens anyway, so might not be necessary to do it explicitly.

rfay’s picture

rfay
Primary language English
Location Palisade, CO, USA
CreditAttribution: rfay commented

I think we've had a restart after every test in there for a long time.

Anonymous’s picture

Anonymous (not verified) CreditAttribution: Anonymous commented

ok, then i think we're good to go :-)

we just need apc enabled for cli, and #1807662: Built-in APCu support in core (PHP 5.5 only) will be unblocked.

http://us2.php.net/manual/en/apc.configuration.php#ini.apc.enable-cli

rfay’s picture

rfay
Primary language English
Location Palisade, CO, USA
CreditAttribution: rfay commented

This is enabled on #1883. (and puppet is stopped there)

  • Commit 679e516 on master by rfay: 
    Turn on apc.enable_cli for #1809248
rfay’s picture

rfay
Primary language English
Location Palisade, CO, USA
CreditAttribution: rfay commented
Status: Needs work » Fixed

This is pushing out to all testbots now.

catch’s picture

catch
he/him
Primary language English
CreditAttribution: catch commented

Thanks!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.