Improve and clean up scripts/run-tests.sh

The script is not intended to run from the browser. I think we should update the docs.

When in Paris, we didn't manage to fix all bugs with this script. It needs some more TLC. When I run the script from my command line, I get Test cases run: 42/42, Passes: 2127, Failures: 2744, Exceptions: 324. I haven't investigated the problem yet, but it seems like it isn't smart enough about checking whether it run/configured properly

I'd like to fix this (or see it fixed) so I can run the script automatically as part of my commits. Right now that isn't possible yet.

Another example of why it needs TLC -- I does funny things like:

$ ./scripts/run-functional-tests.sh --help
Drupal Unit Tests
OK
Test cases run: 0/0, Passes: 0, Failures: 0, Exceptions: 0

The output could also be improved. You don't see what tests are running and/or failing.

Incremental improvements are welcome.

See http://drupal.org/node/252920#comment-831489 for more recommendations.

Looks like a great start. :)

I had a close look at this and it looks like an excellent start. After spending 50 minutes on this patch, I've decided to commit it "as is". I get identical test results when running all tests from the command line, or when running all tests from within the browser, and the code looks good -- minus some indentation issues that I fixed myself.

That said, I'm setting this back to 'code needs work' as there might be room for improvement. It sounds as if you have additional plans for it.

This script is important for me as it has the potential of drastically helping me test and review patches. I'm confident that it will help many others as well. Keep up the great work.

I think being able to list all the tests that can be run would be useful.

Cleaned up spacing on help text.

Ah...yes...good eye.

I think we both agree on this patch so we can say one of us reviewed it. :)

I've committed the latest patch. We still need to better check cases where the group name includes a space. Therefore, I'm putting this back at 'code needs work' instead of 'fixed'. Thanks all.

I have a number of things I would like to improve once #274794: Fix run_tests and make it multi-threaded makes it in allowing the script to actually run with new Batch API stuff.

This patch adds color with an optional parameter --color and formats the output to a 80 character max to fit the default console width.

This patch includes coloring of summary messages, message output clean up, and --verbose parameter to suppress all the assertion output if not specified.

Edit: The coloring on summary messages doesn't work with --all looking into.

This now works will --all.

Working on a complete cleanup.

This is a massive code and comment cleanup over the previous script.

The code has been split up into functions that allow for the logic to be condensed and more easily read and understood.

I have added the following features:

• --color argument to display results with color highlighting.
• --verbose display detailed assertion messages.

Things that still need to be improvement:

• Reporter system
• Add XML reporter
• Deal with groups that have spaces
• Give error messages with invalid groups/classes found

Due to the fact that the script is basically completely re-organized and sections re-written I have included and copy of the script as well as a patch.

This looks good and is a small but important improvement to the script. Both failures and exceptions show in red though, so I think there might be something wrong with the exception detection logic and/or the coloring. I didn't get to see any brown lines. Otherwise looks great!

I see brown for notices.

The summary messages are displayed in either green or red just like the web interface. The individual assertions are displayed in either green, red, or brown.

Ah, you're right. After reviewing the code, I was also expecting the summary lines to be brown. My wrong.

Committed to CVS HEAD. Thanks.

I had no time to comment on that before it was committed, but I think this is a very good job, boombatower. We are one step closer to awesomeness!

Reopening, because that patch has not been tested with parallel execution before being committed, and broke everything badly.

Here is a fix. At the same time, I also add a meaningful error message when the user try to use concurrency while the required extension is not available.

Suppose that should have been looked at, I don't run dual core so didn't think about it.

Committed to CVS HEAD. Thanks Damien.

@boombatower: no problem, error handling was bad anyway, so you couldn't have known.