setting an installation profile for a test doesn't run hook_install_tasks()

I think it should be simpler and more documented.

Possible

$this->setInstallProfile('web_test_case_profile');
$this->setUp();

....

and on tearDown the class variable $install_profile could be reset.

I like the function name. Useful functionality to have. Not tested yet.

I like this implementation much better although I think in needs to be reset in tearDown so that the next test will always use the default profile.

Personally, I think setInstallProfile is behaving a little funny.

1. The return value can be 3 different things. Because this is a class, we don't necessarily have to do the funny return value thing for setInstallProfile.

2. To reset the install profile, why don't we set it to 'default' using $this->setInstallProfile('default') instead of $this->setInstallProfile(NULL, TRUE)? It seems like reset is not needed?

Why the line:

"+ $this->test_profile = variable_get('web_test_case_profile', 'default');"

?

Currently, there are a lot of assertions relying on the HTML response. For example, drupalLogout checks if the user name field is found when requesting the /user page. If we are going to test alternate profiles that may install multiple modules we should have in mind that such modules may alter the output in any way. For example, consider a module that alters or replace the user login form, then every test that uses drupalLogout will fail due to the absence of the name field in which drupalLogout bases it's assertions.

So, here I see again the real need of having data available to make assertions, and not basing assertions 100% on the HTML response. See #266220: Exposing metadata to tests

@dropcube: I already expressed my concerns about #266220: Exposing metadata to tests. If I understand pwolanin approach here, we want to test profiles themselves (ie. have a default.test and a expert.test), not use existing tests on different profile configurations.

Again: we are testing Drupal in its standard configuration. We cannot even imagine testing every possible configuration out there.

The .test file attached uses a more generic approach by implementing a class InstallProfileTestCase which other profile tests may extend. This class contains common functions, for example, _testInstalledModules(). Also a test for the default profile has been added.

Here is the patch based on pwolanin's #12 with the approach described at #17

Here is the updated patch. The base class does not contain test functions now, but assertions.

Install profiles Tests: 25 passes, 0 fails, 0 exceptions

I am attaching an updated version of the file install.test for review and not a patch because seems to be a conflict in drupal_web_test_case.php due to #231190: Page Cache doesn't work with HEAD requests

In the test now are implemented #22 pwolanin's suggestions and added other tests for the default profile. Will submit a patch later.

Here is an updated patch against HEAD.

subscribe. a worthwhile patch. not time to test now though.

I'd still really like to see this functionality added. I'm trying to write tests for Project Issue module, and it'd be much, much easier if I could re-image durpalorg_testing profile on each testX function, rather than default.

SimpleTest module improvements are still allowed as long as they don't break APIs.

webchick: +1 to that. ;) We're working on Project* tests for d.o unit testing (d.o upgrades, redesign deployment, etc). We need the same thing. Maybe one of us will get this working today.

updated patch. same functionality as the last patch, but a lot has changed in the code since then -- hopefully i got things right. let's see what the test bot says...

bah, forgot to include the install.test file. reroll with that added.

This looks fundamentally good. There's some minor coding style stuff (trailing whitespace, simpletest stuff, etc.) that I'd like to fix up later tonight, but other than that this should be just about rtbc.

Tests for includes are placed in simpletest/tests since all test must be in a module. I moved the install.test test cases to the profiles they relate to and put the base test case in standard.test. This was a lot of fun since I needed to get the registry to parse all install profiles instead of just the active one.

I do not think we want to revert to the standard profile without at least adding a failed assertion or some such, although I am not sure if we want to check it directly or not. With all the other class variables we stay away form get'ers and set'ers so probably keeep with that pattern when working with install profiles.

Since the tests will be closely related to the install profile used a general suite wide variable seems like a bad idea. Tests either want a custom install profile or they don't.

Missing public static on getInfo() and scope modifiers (protected) on other methods. We also no longer put the Implements docblock on setUp() and getInfo(). Few other minor things like t()'s and what not (try to move away form them).

- Test case names should match profile namespace.
- Number of documentation issues.
- Tests don't run since functions are called that do not exists such as hook_profile_modules().
- A number of areas seem to be using outdated API style.
- A bunch of notices in standard.test.

It would appear this was never tested since you cannot run test tests when placed in includes/ and even when placed somewhere will they will run they blow up. After about 3 hours I got everything working, but it was far from done.

Actually so many changes this could use another review.

The way I wrote the patch it will allow the registry to find profiles in places outside of ./profiles which to me seems like something the system should do. So if I get agreement I can file a followup patch that finds the other places in core that hard-code a reference to it in ./profiles.

Laptop isn't configured for spaces properly, fixed patch to remove tabs.

This should fix the number of tests run.

This patch is looking very close. I only saw two minor problems:

A)

+class MinimalProfileTestCase extends InstallProfileTestCase {
...
+  protected function testExpertProfile() {

B)

+class StandardProfileTestCase extends InstallProfileTestCase {
...
+  function testDefaultProfile() {

The test function names don't match the classes. Trivial re-roll, 1 sec.

Like so. I just edited the patch file directly in this case. ;)

Yep, looks good. I changed the class names to match, but forgot the test methods. Do we need another review or should we RTBC this?

Read the patch, tried it. Looks good.

This has the side affect that if you're really evil you can enable two install profiles. Muahahaha. But this is awesome. Something that should be exposed via UI in Drupal 8...

UI? The tests are designed to work with one profile, not multiple.

It would be useful for some modules, for example entitycache, to be able to run all core tests as if they were part of the default profile, since the objective there is to not break core, and core (still) doesn't reliably use it's own API functions for updating entities in all cases. However that's a different issue to this patch.

Subscribing. @boombatower, have you already started a back-port of this feature to 6.x? If not, I can probably throw one together.

Not at the moment...life has been ultra crazy lately.

#39: 279515-39.variable-profile.patch queued for re-testing.

subscribe

Better title. #323477: Increase simpletest speed by running on a simplified profile already focuses on allowing a certain install profile for running tests. This issue seems to be about testing install profile expectations itself.

Closely related: #1215104: Use the non-interactive installer in WebTestBase::setUp()

This blocks writing a test for #1170362-145: Install profile is disabled for lots of different reasons and core doesn't allow for that

I've reread this entire issue once more and looked at the most recent but dated patches. I do not see any new or changed functionality that would not be possible already.

#1373142: Use the Testing profile. Speed up testbot by 50% changed the default profile for running tests to the Testing profile. Individual test cases can still override that decision to use the Standard, Minimal, or whatever else profile being available for running tests by assigning protected $profile = 'standard';

Additionally, #911354: Tests in profiles/[name]/modules cannot be run and cannot use a different profile for running tests fixed the discovery of test cases provided by the installation profile or modules in an installation profile when the installation profile of the test is different to the installation profile of the site that executes the test (i.e., the parent site).

Thus, I don't understand what this issue is about. If I'm mistaken, then please do not clarify in a comment, but revise/revamp the issue summary instead, as it doesn't hold any information at all currently.

Updated summary as requested.

There's no relationship between that issue and this issue, other than that you're dealing with a strange, installation profile related testing system oddity and this issue happens to have that term in its title.

#1215104: Use the non-interactive installer in WebTestBase::setUp() is way more related, as you seem to experience a difference in installation profile behavior between manual testing and automated testing.

Looked once more at the patch of this issue today, and confirming once again, all of the changes exist in the current codebase already - either literally or through similar means. So let's close this issue to prevent further confusion.

D7 also supports protected $profile = 'standard'; already.

Can you clarify what is not working?

Sure this is working in D7, for an example, see SimpleTestOtherInstallationProfileModuleTestsTestCase().

> D7 also supports protected $profile = 'standard'; already.

It does.

But that does not cause the install profile to be used. It causes the modules listed in that install profile's .info file to be loaded.

The install code in the profile is not run. See #2038823: DrupalWebTestCase::setUp() does not run hook_install_tasks() for the selected profile.

Are these issues duplicating eachother? If so, can you mark one of them as a duplicate?

Closed the other one as a duplicate, and changing this to a bug, making title more specific.

Updated summary per sun's request in #55.