Problem/Motivation
Make a contrib module in ./module
, give it PHPUnit tests, type ./vendor/bin/phpunit
, and you're happy.
Put that same contrib module in a subdirectory, such as ./module/folder/my_module/
, and its tests vanish from PHPUnit's consciousness.
This is purely an issue with the configuration of PHPUnit.
Note also that there is a problem with bootstrap.php not autoloading any contrib namespaces, but that issue lives here: #2025883: Drupal's PHPUnit bootstrap.php does not register module namespaces out of /core
Proposed resolution
Modify ./core/phpunit.xml.dist
to allow PHPUnit to find the tests.
The main problem with doing this is that SimpleTest also uses ./core/phpunit.xml.dist
for configuration, and a proper configuration of PHPUnit means that SimpleTest dies a horrible death.
Remaining tasks
Change SimpleTest so that it doesn't hijack ./core/phpunit.xml.dist
, or at least can deal with changes to it. That's outside the scope of this issue.
Add a phpunit and/or testing issue category.
User interface changes
n/a
API changes
n/a
Related Issues
#2025883: Drupal's PHPUnit bootstrap.php does not register module namespaces out of /core
Comment | File | Size | Author |
---|---|---|---|
#20 | 2056607-phpunit-config-20.patch | 1.34 KB | Mile23 |
#16 | 2056607-phpunit-config-16.patch | 974 bytes | Mile23 |
#11 | 2056607-phpunit-config-11.patch | 898 bytes | Mile23 |
#4 | 2056607-4-drupal-phpunit_xml.patch | 585 bytes | becw |
#1 | 2056607_phpunit-xml.patch | 760 bytes | Mile23 |
Comments
Comment #1
Mile23And a patch which will fail.
Comment #2
Mile23Comment #3
Mile23Hmm. Maybe not. :-)
Comment #4
becw CreditAttribution: becw commentedI encountered this issue too; #2025883: Drupal's PHPUnit bootstrap.php does not register module namespaces out of /core is related but I think the bugs are separate, since patch #14 from that issue doesn't seem to affect this bug.
If tests aren't found in paths like
modules/contrib/foo
andmodules/custom/bar
, then they won't be found in submodules, either.The patch from #1 in this issue doesn't work for me; I get errors like:
I'm attaching a patch that works for me, but it's very limited; each asterisk in the
testsuite
directory paths inphpunit.xml.dist
only expands to a single directory. This file is interpreted by PHPUnit, so we may not be able to build in as much flexibility at this level.Comment #6
Mile23That's not actually the case. Each directory path tells PHPUnit to start searching in that directory. If you add the trailing
/*
, then it will only expand out to the first level of subdirectories. That's why I removed those in my patch.Well, actually, PHPUnit is very good at finding tests where you tell it to find them. The problem is that our phpunit.xml.dist is misconfigured, and only tells PHPUnit to look at the first level of subdirectories, when it should say to keep going to all the subdirectories.
It looks like you're using Drush to run these tests. How are you running the tests to get this error?
I'm trying to get them to work under plain-vanilla PHPUnit, which you can call this way:
I'm not sure what special cases Drush might need to run PHPUnit tests.
Comment #7
agentrickardThe approach in #4 works for me using ./vendor/bin/phpunit. The patch in #1 fails as expected when running that command.
Not sure why the one patch is green and the other red, though.
Comment #8
becw CreditAttribution: becw commentedI'm not using drush to run the PHPUnit tests--that's just where the error happened to fall in the /modules directory. I run tests in exactly the way you describe (
./vendor/bin/phpunit
).Comment #9
Mile23I think that once #2025883: Drupal's PHPUnit bootstrap.php does not register module namespaces out of /core happens, #1 will be happy from the command line, just running phpunit. The interaction with SimpleTest and Drush seems like a different problem in those areas.
Comment #10
msonnabaum CreditAttribution: msonnabaum commentedThe drush issue is from devel, because it includes a phpunit test for drush in it's root. When you remove the "/*/tests" from the phpunit.xml, it picks up any phpunit test in a module.
I originally added that so that we wouldnt be running vendor tests as well, since it's conceivable that you might use composer and you would want to exclude your dependencies' tests.
I was hoping we could filter by test class, but phpunit doesn't appear to have such an option. I think the best we can do is #1 and then ask that drush tests go in modulename/drush/tests or something, that we can exclude by path.
Comment #11
Mile23Yah, the existence of the test for drush within devel means I can't run any tests for devel without drush installed alongside. Drush should really have its own testing subdirectory and config for PHPUnit, since it's not really Drupal. (This is where I idly mention Symfony's console...)
So if we assume it's OK to break the drush test here, to be solved within drush and/or devel then this is all we need.
Also killed the config and views excludes because PHPUnit does a fine job of not running tests that aren't PHPUnit tests. :-)
Comment #12
Mile23Comment #13
agentrickardDrush / devel can fix itself after we commit this.
Comment #15
msonnabaum CreditAttribution: msonnabaum commentedMoshe agreed to move drush tests to ./drush/tests, so we should probably test with that path being excluded and then let it be fixed in devel.
Comment #16
Mile23#11, excluding ./drush/tests
Comment #18
Mile23Will re-test once #2025883: Drupal's PHPUnit bootstrap.php does not register module namespaces out of /core hits. :-)
Comment #19
Mile23Is there an issue for the ./drush/tests move?
Comment #20
Mile23Excluding /vendor for quicker tests.
Excluding /vendor from coverage reports.
Comment #21
Mile23Comment #23
Mile23#20: 2056607-phpunit-config-20.patch queued for re-testing.
Comment #24
Mile23Would be great to have an RTBC so we can test modules that live inside other folders. :-)
Comment #25
dlu CreditAttribution: dlu commentedQuick before we need a reroll :-)
Comment #26
Gaelan CreditAttribution: Gaelan commentedLooks great to me.
Comment #27
webchickCommitted and pushed to 8.x. Thanks!
One question though:
Do you also need /sites/*/modules/*/tests to be parallel with /modules/*/tests?
Comment #28
Mile23../sites/*/modules/*/tests would bring back the original problem, just under /sites. PHPUnit is pretty good at finding tests very quickly, so a wider net is better in this case.
The ./modules/*/tests line assumes that core modules will never be in subfolders, which could be wrong.
And yay phpunit component. :-)
Comment #29
webchickGot it, thanks. :) And yeah, figured that was easier as a means of finding these. Thanks for all the great work on making it better!
Comment #30
amateescu CreditAttribution: amateescu commentedHere's the Devel issue: #2106943: D8 testing is red again -- with "Class 'Drush_CommandTestCase' not found"
Comment #31
moshe weitzman CreditAttribution: moshe weitzman commentedI put devel's tests right in a ./drush subdir so the exclude should be ./drush. I assume that would also exclude drush/tests in case anyone wants to put tests there.
Comment #32
XanoI'm having trouble adding a PHPUnit test to a contributed module and as most people who helped me were not aware of this issue, I'd like to explain my case here to see if the patch was actually faulty, or if something else is wrong.
I added a PHPUnit test under the name of
\Drupal\payment_form\Tests\Plugin\payment\type\PaymentFormUnitTest
at./modules/payment/payment_form/lib/Drupal/payment_form/Tests/Plugin/payment/type/PaymentFormUnitTest.php
. PHPUnit can't find the file, even though it should be covered by the<directory>../modules</directory>
directive in phpunit.xml.dist. Only when I add<directory>../modules/payment/payment_form/lib/Drupal/payment_form/Tests/Plugin/payment/type</directory>
to it, it finds the file. It then successfully includes it, but PHPUnit_Framework_TestSuite::addTestFile() cannot find any newly loaded classes. However, when simply changing the test class from a UnitTestCase to UnitTestBase, Simpletest correctly finds and runs it.Part of the reason why I spent so much time on this problem already, is that the test structure is highly inconsistent and I haven't been able to find centralized documentation on this:
./core/modules/*/lib/Drupal/*/Tests
./core/modules/*/tests/Drupal
./modules/*
?Comment #33
XanoNever mind my previous post. It turns out PHPUnit does not follow symlinks, which is what I used to insert the contributed modules into my D8 testing setup.
Comment #34
dlu CreditAttribution: dlu commentedSeems like it would be good to document this behavior. Or would it make sense to "fix" PHPunit? The behavior seems like a bug to me…
Comment #35
Mile23There's an issue about it for phpunit-file-iterator: https://github.com/sebastianbergmann/php-file-iterator/pull/21
Ideally, it will just require a composer update. It just looks as though the 1.3.4 release is a phantom.
Comment #36
XanoYes, that's my pull request. I'm waiting for 1.3.4 to be available and then I'll open an issue to update it in core.
Let's set the status back to what this was so my problem doesn't hijack the issue.
Comment #37
Xano#2110153: Upgrade phpunit/php-file-iterator from 1.3.3 to 1.3.4
Comment #38.0
(not verified) CreditAttribution: commentedAdded a next step.
Comment #39
Mile23This issue has a follow-up which is EXACTLY THE SAME PROBLEM because no one cares about contrib. #2499239: Use test suite classes to discover different test types under phpunit, allow contrib harmony with run-tests
Comment #40
webchicks/no one cares about contrib/we don't have an integration test to ensure this behaviour stays working/
FTFY.