yadayada, details follow later.

Check http://behat.org/ and http://mink.behat.org/

#913086: Allow modules to provide default configuration for running tests made me realize that - although the intention and use-case is vastly different - the fundamental code and requirements for introducing default testing configuration are fairly similar to and share a couple of semantics with BDD.

So I hacked together some funny code for default testing config in tests that uses some funny classes and dude even a really fancy (ugly) DI-alike service container that gives you access to default testing config provided by modules, and oh behave, it even works.

A full design, introduction, conversion, and integration of BDD/Behat will take a lot of time, and I definitely do not want to hold off the currently ongoing testing system improvements on that. But it surely can't hurt to explore ways that possibly lead to code that can be more easily migrated in the end.

So let's discuss, Mr. @weitzman, Mr. @beejeebus, Mr. @Sonnabaum! :)

This patch is reduced to the max, so non-functional. Comparing it with the patch above mentioned issue can't hurt to get a better understanding.

CommentFileSizeAuthor
#4 commons_qa.zip63.95 KBmoshe weitzman
drupal8.dbehave.0.patch8.73 KBsun
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Status: Needs review » Needs work

The last submitted patch, drupal8.dbehave.0.patch, failed testing.

sun’s picture

Status: Needs work » Needs review

To clarify what I'm after here:

  1. We don't introduce BDD/Behat. Later.
  2. Modules badly need to be able to setup "dumb" config during tests for themselves.
  3. The current proposal suggests a simple module hook to set that up.
  4. The hook would be ignorant to anything else and just go ahead and set up some config.
  5. BDD/Behat tests use a rather high-level approach of integration testing. They set up dynamic, but not very granular configuration to test a certain scenario.
  6. In turn, those tests will look along the lines of this:
      Scenario: Edit a node
        Given a user "foo"
        And a node type "article"
        And a node
        When user "foo" can "edit article content"
        And I am logged in
        And I visit the node "Test"
        Then I should see a link:
          """
          Edit
          """
    

    (Please ignore exact details of this scenario. Figuring out the proper, modular macro language for those BDD/Behat tests is exactly the reason for why I think this will take a lot of time to get done.)

  7. That direction will require something very similar to default configuration for tests.
  8. But instead of default configuration being ignorant to any scenario and conditions, this asks for individual pieces of things you can set up for a specific scenario.
  9. In turn, it might make sense to at least introduce the default config for tests in a way that would be basically "compatible" with that direction.
  10. I.e., instead or in addition of blatantly setting up default config for all modules, allow modules to expose methods that allow a test to pull in particular pieces of predefined configuration.

    Along the lines of this:

    function setUp() {
      $this->config
        ->createUser('foo')
        ->createNodeType('article')
        ->createNode()
        ->grantDefaultNodeAccess() // <-- Let's have hook_testing_install() in parallel to avoid this
        ->grantPermission(DRUPAL_AUTHENTICATED_RID, 'edit article content');
    }
    
moshe weitzman’s picture

FileSize
63.95 KB

Here are integration tests and step definitions that Acquia uses to test Drupal Commons. We have a similar suite for Drupal Gardens. We've been using BDD for a while now and we think it is a great fit for Drupal. Drupal's modularity and reuse leads to only a few, powerful step definitions.

Please open up the zip and take a look around. The step definitions are written in Ruby since we chose Cucumber a while ago. Drupal 8 would use Behat instead, and thus use PHP instead of Ruby.

This issue suggests smaller steps toward BDD in Drupal 8. Small steps are good. I attach this Zip to show what the end goal could look like.

sun’s picture

Thanks, @moshe. That looks fairly similar to the examples in the Behat documentation.

The missing part for me (and what triggered this issue) however is modularity. An installation profile like Commons is able to rely on the facts that

1) there are certain, limited modules only, and
2) it provides the necessary features/behaviors on its own (instead of the modules themselves).

These assumptions cannot be made for regular module tests. But I believe that BDD would be suitable for many module tests as well. (Not all, but certainly some.)

My work in #913086: Allow modules to provide default configuration for running tests is currently blocked on the question of

A) whether I should go with a simple hook approach to setup configuration/features of installed modules during testing,
B) or whether want to go with a class-based approach as prototyped here.

Meanwhile, I'm personally leaning towards B) - even though it would require explicit configuration/feature setup code in every test case.

fgm’s picture

To some degree, I think that module testing, as the current examples show it, are more likely to be realistic as integration tests (à la RSpec) than full-stack tests, for exactly the reasons you outline, Mocks and seams, might be adjustable by tweaking the hook system, and can help combine individual mechanisms (hence testing integration) while still limiting the impact of the rest of the configuration, making test scenarios realistic.

The problem of course, being that PHPspec is far less legible/handy than its original counterpart, unlike Behat vs Cucumber, where the features almost don't change, and the step definitions are atually not so bad to read than one could have expected. Maybe there are better alternatives ?

nod_’s picture

Issue summary: View changes

A bit less that two years later, seems Drupal is ready #2232271: [Meta] Use Behat for validation testing :p

sun’s picture

Status: Needs review » Closed (duplicate)