Breakoff from #500280: Provide a verbose mode for SimpleTest

"So if we do separate files...I would recommend creating a verbose directory where the current verbose.html is and then placing each of the files in there named related to the ID and then the anchor can be removed from output."

CommentFileSizeAuthor
#10 529126-split-verbose.patch4.9 KBboombatower
#5 529126-split-verbose.patch3.37 KBboombatower
#3 529126-split-verbose.patch3.22 KBboombatower

Comments

moshe weitzman’s picture

moshe weitzman commented

OK, I can get happy with multiple .html files. If we name our files correctly, you can simulate the next/prev scrolling with https://addons.mozilla.org/en-US/firefox/addon/2933.

boombatower’s picture

boombatower commented
Assigned: Unassigned » boombatower

Working on this.

boombatower’s picture

boombatower commented
Status: Active » Needs review
StatusFileSize
new 529126-split-verbose.patch3.22 KB

This should do it.

dries’s picture

dries commented

I'm confused by the help text change. It tells me tests need to be run one by one, but clearly, it is creating multiple files. It seems like we need to make that a bit more clear.

boombatower’s picture

boombatower commented
StatusFileSize
new 529126-split-verbose.patch3.37 KB

Currently there is no way to distinguish between the verbose output from one test vs another. This was an issue before, but I didn't think about.

There really is SO much information it makes no sense...to run multiple tests, much less the whole suite in this mode as it would dump more data then you could interpret. Given the current design the verbose files output are flushed before each test case to ensure a clean starting point. This could be changed, but I really don't think it is a good idea or that anyone will ever want to debug multiple tests in verbose mode at the same time.

I have added a bit more to the documentation, let me know if it is clear.

moshe weitzman’s picture

moshe weitzman commented

Um, I might have 2 test classes that are failing and so I run just those 2 in verbose mode. You are saying that unsupported? I think thats a bad decision.

One more somewhat related thought. Can I use the verbose method manually, even if verbose mode is disabled? That would be very helpful.

boombatower’s picture

boombatower commented

You can call:

protected function verbose($message) {

But the verbose calls themselves are in drupalPost|Get(), not sure the best way to expose them if we do.

As for not letting it run two tests cases...the reasoning is: we would need to keep track of where the numbers start and end and where we last left off. Creating some logic to separate the two when clicked the next previous, etc. All seems a bit much considering I have yet to debug two tests simultaneously and don't really see it as a useful feature.

chx’s picture

chx commented
Status: Needs review » Reviewed & tested by the community

I think this is good to go

webchick’s picture

webchick
she/they
Primary language English
Location Vancouver 🇨🇦
commented
Status: Reviewed & tested by the community » Needs work

From a user perspective, I found this confusing:

Only local images are allowed.

What I'd ideally like to see here is a "back to test results" link that takes me "up" to the test results. However, in IRC, boombatower pointed out that this would not in fact remember your test results, since the cleanup routine of the various tables happens directly after they're viewed so you'd get back a blank simpletest form with an error. Not nice. :P Let's at least open those links in a new window though, because I think it's very natural to want to see both at the same time.

I also agree with Moshe that not allowing verbose mode across test cases is an unnecessary limitation. What I frequently do with patches is view the test bot results, select each of those test cases, run the tests locally, and then fix the bugs all in one go, then re-run the test cases again. Being forced to do that one. at. a. time. would be extremely annoying.

So let's change the naming slightly from verbose-$id.html to verbose-$testcase-$id.html to support that.

boombatower’s picture

boombatower commented
Status: Needs work » Reviewed & tested by the community
StatusFileSize
new 529126-split-verbose.patch4.9 KB

Alright...this should do it. Tweaked a few things and it clears files when test suite run starts.

webchick’s picture

webchick
she/they
Primary language English
Location Vancouver 🇨🇦
commented
Title: Split up SimpleTest verbose output into sperate files » Split up SimpleTest verbose output into separate files
Status: Reviewed & tested by the community » Fixed

Ok, cool. Committed to HEAD!

moshe weitzman’s picture

moshe weitzman commented

This is very nice.

FYI, the settings page still says "By default SimpleTest will clear the results after they have been viewed on the results page,". I think that needs to says "at the beginning of each test run.

webchick’s picture

webchick
she/they
Primary language English
Location Vancouver 🇨🇦
commented
Status: Fixed » Needs review

Good point.

webchick’s picture

webchick
she/they
Primary language English
Location Vancouver 🇨🇦
commented
Status: Needs review » Needs work
boombatower’s picture

boombatower commented

The results is referring the results in db...as in the assertions. They are indeed cleared at the end of each test.

The verbose...due to the fact that you need to view the pages after the test results...are cleared when next test run starts. That is out of necessity and cannot be disabled/enabled due to the way it is written...and is thus not effected by the setting.

Based on that, this is complete...if anything maybe clarification that verbose is not included.

boombatower’s picture

boombatower commented
Status: Needs work » Fixed

Unless otherwise noted...this is fixed.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.