Unit testing library
moshe weitzman - June 6, 2009 - 19:14
| Project: | Drush |
| Version: | All-Versions-HEAD |
| Component: | Code |
| Category: | feature request |
| Priority: | normal |
| Assigned: | moshe weitzman |
| Status: | active |
Jump to:
Description
It is becoming clear that we need to unit test drush and its commands. This issue is a discussion about what library to use. Some important points:
- drush is a CLI application. we don't care much about web browsers
- ideally we should test for drupal versions 5,6,7.
- drush aims to be small and so should its test library
#1
I've researched this topic and a leading contender is phpt. Please click that link to learn more. I have not actually done test writing with this yet.
Thoughts on other contenders:
#2
The syntax behind phpt looks pretty nasty... is simpletest that much of a problem? No need to actually package it with drush...
#3
Found shunit2. Looks appropriate for our needs. It does some bash tricks that are above my pay grade. What do others think? Poke around at http://code.google.com/p/shunit2/wiki/HOWTO_TestAbsolutePathFn
#4
doctest appeals to me a lot. I can see how others will gag at code within doxygen but I think thats the easiest way to get devs to maintain tests. Comments?
#5
I fail to see why simpletest is considered "too big" right off the bat... It's used in Drupal, and if it's too big for Drupal, well, I guess we could say Drupal is too big for drush and we still use drupal...
Let's not diverge from the library already used in the Drupal community, it's going to confuse things for everyone. By using simpletest, we make sure that a maximum number of people are already familiar with the testing framework and can contribute tests. Furthermore, we also avoid adding another dependency to drush as, arguably, simpletest is more likely to be installed...
#6
I don't like shunit; it's too simple--or too complex, if you look at it the other way around. There's no substantial framework, and the shell scripts you need to write are inelegant.
I don't like doctest; placing tests inline in the php code itself would make the drush sources unwieldy and hard to scroll through.
I like phpt; seems about right in terms of size and capabilities.
Simpletest is a little more heavyweight than phpt, speaking only in terms of the code you need to write to get started. It has the advantage of being more similar to the Drupal "standard", which has merits. It seems If we used it, code that migrated from drush to drupal might take unit tests with it. But how likely is that? Rare, probably. I still think simpletest is imminently usable, although I like phpt more.
One thing to consider is that a large part of testing drush involves testing the command line parsing and bootstrapping code, not just individual functions. For that reason, something that actually calls drush from the shell would be beneficial. By this metric, shunit looks better -- but again, I'm not thrilled with it. Ideally, something that allowed different config files and environments to be put together would be ideal (use provision to create entire sites for testing?), but that's going down the "build it yourself" road, which is what would end up happening if you started using shunit. That would entail a phenomenal amount of work.
My opinions.
#7
Well, I'm interested in the original simpletest library from http://simpletest.org/. I'm not so interested in the drupal bastard child. My objection to Drupal's simpletest is that it depends on Drupal. Much of drush has nothing to do with drupal, and i dislike keeping a drupal around when running tests on drush_invoke(), context system, error handling, etc.