Replace SimpleTest libraries with custom code

*shock* *love* *awesome* if you can pull this off you are the hero of Drupal 7. Having the various reporters around is useful and necessary, though. Keep up the good work!

Awesome! Drupal simplicity is _more_ than very abstracted code, and I think this should stay as it is.

"BTW, the reason this patch is so gigantic is because there is a lot of file removing going on."

Indeed:

$ diffstat simpletest_replace_simpletest.patch 
 default_reporter.php      |   75 -----
 drupal_reporter.php       |  257 ------------------
 drupal_test_case.php      |   92 ++++++
 drupal_test_suite.php     |  147 ----------
 drupal_unit_test_case.php |   17 -
 drupal_web_test_case.php  |   30 --
 dumper.php                |   80 -----
 errors.php                |  240 -----------------
 exceptions.php            |  171 ------------
 expectation.php           |  144 ----------
 invoker.php               |  119 --------
 reporter.php              |  283 --------------------
 scorer.php                |  413 -----------------------------
 simpletest.css            |    9 
 simpletest.module         |  347 +++++++++++-------------
 simpletest.php            |  369 --------------------------
 test_case.php             |  615 --------------------------------------------
 unit_tester.php           |  178 ------------
 xml.php                   |  641 ----------------------------------------------
 19 files changed, 262 insertions(+), 3965 deletions(-)

What about the browser/web tests?

Keep it up!

I'd support this, I guess. There a number of things to be loved in SimpleTest though -- it does things like 'integrate with Eclipse' and 'helpful reporting'. To some extend, it would be a pity to see some of the 'power user features' go. But then, where do we draw the line? I guess we can add things later on ... :)

subscribing...

Hm, Dries "I'd support this, I guess"? Then let's prepare for work a bit. Sttuff is at https://4si.devguard.com/svn/public/simpletest/drupal-new now in case you want to work on it. Lock the repo when you are working so others know. All the usual suspect already have write access to that and anonymous read only access is also provided. If you need write access, find me, that's easy. As I find time, I will code this, too.

I don't need write access at this point, but I'd like to keep posted about the status of the work. I'll allocate time to review this at the appropriate times.

Any update on this?.... subscribing.

I already did a big rant against this over at http://drupal.org/node/249553. To summarize:

1. We have much higher priorities atm, including getting all core tests working, getting testing.drupal.org working, and fixing testing documentation. Until these three things happen, it will be impossible for our community to 'embrace testing' regardless of how nicely spit-shined the code is underneath.

2. Moving away from SimpleTest prevents developers who already know SimpleTest from transfering their knowledge over. They have to learn the same thing twice, slightly differently. It also makes it more challenging for new developers, because they can't utilize the knowledge in 10,000 existing articles about SimpleTest and how it works are no longer relevant for their Drupal experience.

3. Which leads to a larger point, that in general, moves like this are pushing the burden of supporting our testing framework on the Drupal community's shoulders, and not the SimpleTest community's. Documentation has to be written to supplant the documents widely available about SimpleTest, except for Drupal's testing framework. All of the time spent doing these sorts of activities is actively taking away from other things like reviewing/creating important Drupal patches, working on further test coverage, etc.

4. And finally, since the people most fluent in the testing framework who are working on this effort also happen to typically be the people who generally do a lot of the heavy lifting in the Drupal community, I know I for one would much rather see their time and talent spent on stuff that benefits Drupal more directly than polishing underlying code which about 10 people will ever know/care about.

If we do finish this it would definitely be a good idea to get all the tests to pass first then change the framework and make sure they all still pass.

I'm going to mark this postponed until we have a 100% pass rate with 0 exceptions.

@webchick:

I guess I see where you're heading with you rant, as you put it.

I have worked with Test Driven Development for quite some time now in connection with agile development, especially eXtreme Programming. I couldn't imagine working any other way again; in fact, I didn't imagine—till I began working with Drupal.

From the programmer's point of view, TDD is hard to grasp when you've grown up in an environment that favours quick results by immediate coding, with strict code ownerships and some kind of don't-touch-that-line-of-code philosophy. Even in a community like Drupal's you can see project contributors not daring to change a line of code without prior consent of the project lead or main contributor.

Within Drupal, TDD is a very suitable way to protect the quality of the code by accompanying it with a full set of tests that inevitably fails when something got broken inadvertently.

Here's the catch: from a beginner's point of view (beginning with TDD, not with coding, that is!) Simpletest may seem a good choice, but from a more TDD-experienced point of view Simpletest just isn't a suitable Testing environment for a Drupal scale PHP 5 framework or application. On one hand it is still written in PHP4, on the other hand—more importantly—it complicates coding instead of facilitating it. Those problems range from ignoring common coding conventions to downward poor code.

But rest assured that all those programmers who have become experienced with Simpletest will see a steep progress curve in the more mature testing frameworks to come, and I can almost predict they'll love it.

So I'll just happily wave Simpletest good-bye when its time has come. Can't wait!

Best regards
C.

This was completed by #243773: SimpleTest: Use the Batch API instead of the current all-in-one approach.