Create more extensive guidelines for "mock module" creation and style [#785362]

We use so-called "mock modules" to provide test scenarios for testing. They are just regular modules that provide test situations that we can't get in core, but that are hidden from ordinary users on the web UI modules page. They're completely accessible to the Drupal API and via drush.

These modules have traditionally been seen as just for automated testing. However, I think it's important that we make manual testing accessible to everyone, and hand-enabling these modules is a key part of that. But if we're going to do that we need some guidelines.

In addition, there are parts of our system that can't be tested using automated testing. I'm a proponent of putting manual tests in mock modules for every patch that doesn't have automated testing (and making the mock module friendly to manual testers even when there are automated tests).

Questions to be discussed:

Should menu entries that should be clicked be set to MENU_CALLBACK or MENU_NORMAL_ITEM? (MENU_NORMAL_ITEM would make the specific menu items more accessible to a manual tester)
Is it OK for a mock module to have a feature that can mess up something about the site? If so, how should this be documented or prevented?
Should we consider it best practice to always provide a manual test when automated tests are inadequate?

More?

I'll be happy to add/edit the test guidelines page based on the outcome here.

Comments

Comment #1

rfay

English

Palisade, CO, USA

CreditAttribution: rfay commented 29 April 2010 at 14:56

This is a spinoff from the discussions in #767268: #states doesn't work correctly with radio buttons.

Comment #2

sun

German

Karlsruhe

CreditAttribution: sun commented 29 April 2010 at 15:21

Copying my last reply from aforementioned issue over here:

We should coordinate the addition of "demonstration code for manual testing" a bit more. If I'm not mistaken, not all testing modules can be enabled safely, i.e. without potentially doing harmful stuff to your site. And when I somehow manage to enable a testing module for manual testing, then I'd probably expect some kind of sane structure...

For example, I recently started to work on a "proper" markup_test module in my sandbox, which tries to introduce and build a sane technical structure for manually testing + reviewing issues for the "markup" component, but with the goal to do it in a generic and extensible way in the end. Ultimately, when there's some more code and experience with that module, it will be worth to have its own d.o project, as you will likely be able to use the same testing code to ensure proper styling + behaviors in contributed and custom themes you build.

That markup_test module, as well as the Examples module, as well as the suggested demonstration code here, may be partially but not fully duplicating efforts, but I'm not sure yet. What rather matters for me is that we don't add arbitrary code to Drupal core with no clear vision, structure, and behavior -- at least not at this stage of the release cycle.