Provide a verbose mode for SimpleTest

I have started on this.

Initial settings page in preparation for the verbose mode setting, #500292: Provide a settings page for hidden SimpleTest variables.

Once settings page gets in I can modify this patch to add the verbose variable to it, or make a follow patch depending on timing.

Updated to take advantage of #500334: SimpleTest: color debug messages

Assertion:

Log file (after clicking link in assertion):

The last submitted patch failed testing.

Test slave crapped.

This could use some design/visual work, IMO. Like, what's up with the square brackets? :P

The brackets can be removed. The only reasoning was trying to signal that the links take to external page...and arn't really part of the test results.

What other things need work? Suggestions welcome, but lets keep it simple.

The dumb in the log contains numerous pages and there are not a lot of formatting options, and I think we should try and keep it simple since this is for debugging. The log could easily contain 20+ pages so a lot of formatting will just slow down browser (as I experience a bit on larger tests).

This is essentially the method I use to debug tests, except that I never dump as much information (very useful though) and manually dumped pages in test.

There is a bug here:

return $i++;

doesn't return an incremented value of $i;

Nice work! This is a good start in something I think is REALLY important. There is no way to practice TDD the way things are currently setup. Perhaps a better way though would be to ditch the progress bar entirely, OR be able to show the output below the progress bar as it is running. The [verbose message] is a little too hidden for me. I want to edit, debug, go, edit, debug go without clicking around to see my debug message. I'd prefer to just see it inline personally.

If I want to develop from a test, I want to be able to print to the stdout buffer I think. If you use something like junit or ruby::unit, they just spit out to stdout. I don't see why what we've got is an advantage unless you are trying to run every unit test, every time you hit save, which IMO is an impossible way to code.

That is intentional, since $id starts at 0 so the first one returned is 0 and then it is incremented.

1) You an simply refresh the file without clicking debug link...it just puts you are the position in context and gets the file open for you first time. If you leave it in tab you can just refresh it after running test (as I usually do)

2) Getting an stdout functionality is what I am going to try and provide in #296574: Provide debug facility for general and testing or at least something equivalent. Since much of the testing revolves around web pages this verbose mode should be extremely useful since this is something I have been using since the start. So I think it will be great to have it built in.

I'll re-upload patch to make sure everything tests right.

I'll mark it as reviewed per #9.

The last submitted patch failed testing.

@bombatower:

That should explain what I meant. There is a bug, trust me.

Thanks for looking into this, I'll continue to use it and give reviews from time to time, but too busy to help much at this point.

The code works and the proper results are printed and links work right.

I'm really not sure where the bug is, related to this. I can figure out testbots issues as there are bugs this causes, but not related to $id++.

d'oh, my bad. Must have had something funky in my install, but this wasn't what was needed to fix it.

-J

This fails due to php 5.2.9 bug which bot was running.

There is no way to practice TDD the way things are currently setup. Perhaps a better way though would be to ditch the progress bar entirely, OR be able to show the output below the progress bar as it is running. The [verbose message] is a little too hidden for me. I want to edit, debug, go, edit, debug go without clicking around to see my debug message. I'd prefer to just see it inline personally.

While I think the first sentence is an exaggeration, I see value in having something a bit more "streaming" so you don't have to wait for the test results. But as boombatower, mentioned, we can worry about that in a follow-up.

I still think the [verbose message] links are odd. I wonder if we just need to turn messages like: "Valid HTML found on "http://localhost/drupal-cvs/user/3" into links. Or whether we should add an icon somewhere.

While on the topic of that output, I find the 'Status' column to be completely redundant.

The last submitted patch failed testing.

Crazyness with testing clients tonight...this is what I had when we discovered error in PHP.

I had removed the [] and such...I can look into making the link on the previous assert as I think makes sense. It just requires a bit more code.

Added setting since the related patch has been committed.

I've made some minor changes to the labels and committed it to CVS HEAD. Thanks!

Left in a bit of debug code...woops. :)

Still an issue with the variable being not in the context during the test run and it needs to be cached.

Double checked it and it works.

Apparently this code hates me. The link was getting messed up. I have to believe some other core code changed underneath this.

/me crosses fingers.

The last submitted patch failed testing.

Last two patches fixed the issue...but the SimpleTest test brings up a weird edge case...since it actually loads itself within a test...so you get a double depth files folder.

Fixed by adding file_check_directory().

Committed to HEAD.

This is sweet functionality. I turned it on and ran the comment tests and saw nice verbose links in the results. But when i click on the link, my firefox refuses to show the resulting page and safari nearly crashes on it. verbose.html gets huge fast. and it isn't valid html as it is just the body of an html page.

IMO, the logging should be done to separate files, one for each verbose invokation. And we should log to simple text files. the meta info should be in headers like email. that would scale much better. I know that some folks like "seeing" the HTML of the debug message in a browser by default but usually the caller just ends up doing a view source on that anyway. further, all HTML sent into verbose() has to be htmlspecialchars() beforehand. Really, text is best IMO. In any case, multiple files are needed. I can't imagine the hugeness of the file if it was enabled during a run of all tests.

I'm willing to write a patch if folks agree with my approach.

Agreed that the verbose mode is currently not usable in real life, brings browsers on their knees (at best).
Maybe we could just log the latest $this->drupalGetContent() only when a test fails ? (granted, that might be a completely unrelated request and you need to be a little smart when debugging; but the current implementation has the same "drawback": when debugging a failure, the latest drupalPost logged might be completely unrelated)

I'd find it extremely cool if we could preserve the current 'real HTML to see in a browser' feature, though - really more handy in a lot of cases to see whats wrong at once. If I need to investigate the markup, firebug remains my favorite primary tool anyway.

At any rate, separate files per logged page sounds a must.

Yea, I'm not a 100%. I use the functionality (as is) when writing tests because I only run one part of the test. I like being able to just scroll and see the path the browser took.

Before I wrote this patch I used to do it manually so I could only output the pages I need. Perhaps that is the best solution (although would change meaning of verbose)? Add a parameter to drupalGet|Post() to dump verbose...or just allow the verbose method to be called after drupalGet|Post().

Putting each call in a separate file is ok, but being able to scroll is extremely useful. I would be ok with either approach.

well, it seems opening logged pages in separate tabs can compensate the 'scrolling through history' ?

Possibly.

So if we do separate files...I would recommend creating a verbose directory where the current verbose.html is and then placing each of the files in there named related to the ID and then the anchor can be removed from output.

I think we should create separate issue as this is a followup...lets lets this issue rest.

#529126: Split up SimpleTest verbose output into separate files

Automatically closed -- issue fixed for 2 weeks with no activity.