Comments

boombatower’s picture

Assigned: Unassigned » boombatower

I have started on this.

boombatower’s picture

Initial settings page in preparation for the verbose mode setting, #500292: Provide a settings page for hidden SimpleTest variables.

boombatower’s picture

Status: Active » Needs review
StatusFileSize
new4.71 KB

Once settings page gets in I can modify this patch to add the verbose variable to it, or make a follow patch depending on timing.

boombatower’s picture

StatusFileSize
new4.69 KB
new31.36 KB
new18.36 KB

Updated to take advantage of #500334: SimpleTest: color debug messages

Assertion:
verbose assertion

Log file (after clicking link in assertion):
verbose log

Status: Needs review » Needs work

The last submitted patch failed testing.

boombatower’s picture

Status: Needs work » Needs review

Test slave crapped.

dries’s picture

This could use some design/visual work, IMO. Like, what's up with the square brackets? :P

boombatower’s picture

The brackets can be removed. The only reasoning was trying to signal that the links take to external page...and arn't really part of the test results.

What other things need work? Suggestions welcome, but lets keep it simple.

The dumb in the log contains numerous pages and there are not a lot of formatting options, and I think we should try and keep it simple since this is for debugging. The log could easily contain 20+ pages so a lot of formatting will just slow down browser (as I experience a bit on larger tests).

This is essentially the method I use to debug tests, except that I never dump as much information (very useful though) and manually dumped pages in test.

JacobSingh’s picture

Status: Needs review » Needs work

There is a bug here:

return $i++;

doesn't return an incremented value of $i;

Nice work! This is a good start in something I think is REALLY important. There is no way to practice TDD the way things are currently setup. Perhaps a better way though would be to ditch the progress bar entirely, OR be able to show the output below the progress bar as it is running. The [verbose message] is a little too hidden for me. I want to edit, debug, go, edit, debug go without clicking around to see my debug message. I'd prefer to just see it inline personally.

If I want to develop from a test, I want to be able to print to the stdout buffer I think. If you use something like junit or ruby::unit, they just spit out to stdout. I don't see why what we've got is an advantage unless you are trying to run every unit test, every time you hit save, which IMO is an impossible way to code.

boombatower’s picture

Status: Needs work » Needs review
StatusFileSize
new4.69 KB

That is intentional, since $id starts at 0 so the first one returned is 0 and then it is incremented.

1) You an simply refresh the file without clicking debug link...it just puts you are the position in context and gets the file open for you first time. If you leave it in tab you can just refresh it after running test (as I usually do)

2) Getting an stdout functionality is what I am going to try and provide in #296574: Provide debug facility for general and testing or at least something equivalent. Since much of the testing revolves around web pages this verbose mode should be extremely useful since this is something I have been using since the start. So I think it will be great to have it built in.

I'll re-upload patch to make sure everything tests right.

boombatower’s picture

Status: Needs review » Reviewed & tested by the community

I'll mark it as reviewed per #9.

Status: Reviewed & tested by the community » Needs work

The last submitted patch failed testing.

JacobSingh’s picture

@bombatower:


function t_i() {
  static $i;
  $i = 0;
  return $i++;
}

for ($j = 0; $j < 10; $j++) {
  echo t_i($j);
}

That should explain what I meant. There is a bug, trust me.

Thanks for looking into this, I'll continue to use it and give reviews from time to time, but too busy to help much at this point.

boombatower’s picture

static $i;
$i = 0;

is not the same as

static $i = 0;

The code works and the proper results are printed and links work right.

I'm really not sure where the bug is, related to this. I can figure out testbots issues as there are bugs this causes, but not related to $id++.

JacobSingh’s picture

d'oh, my bad. Must have had something funky in my install, but this wasn't what was needed to fix it.

-J

boombatower’s picture

Status: Needs work » Reviewed & tested by the community

This fails due to php 5.2.9 bug which bot was running.

dries’s picture

There is no way to practice TDD the way things are currently setup. Perhaps a better way though would be to ditch the progress bar entirely, OR be able to show the output below the progress bar as it is running. The [verbose message] is a little too hidden for me. I want to edit, debug, go, edit, debug go without clicking around to see my debug message. I'd prefer to just see it inline personally.

While I think the first sentence is an exaggeration, I see value in having something a bit more "streaming" so you don't have to wait for the test results. But as boombatower, mentioned, we can worry about that in a follow-up.

I still think the [verbose message] links are odd. I wonder if we just need to turn messages like: "Valid HTML found on "http://localhost/drupal-cvs/user/3" into links. Or whether we should add an icon somewhere.

While on the topic of that output, I find the 'Status' column to be completely redundant.

Status: Reviewed & tested by the community » Needs work

The last submitted patch failed testing.

boombatower’s picture

Status: Needs work » Reviewed & tested by the community
StatusFileSize
new4.66 KB

Crazyness with testing clients tonight...this is what I had when we discovered error in PHP.

I had removed the [] and such...I can look into making the link on the previous assert as I think makes sense. It just requires a bit more code.

boombatower’s picture

StatusFileSize
new6.03 KB

Added setting since the related patch has been committed.

dries’s picture

Status: Reviewed & tested by the community » Fixed

I've made some minor changes to the labels and committed it to CVS HEAD. Thanks!

boombatower’s picture

Status: Fixed » Reviewed & tested by the community
StatusFileSize
new695 bytes

Left in a bit of debug code...woops. :)

boombatower’s picture

StatusFileSize
new1.09 KB

Still an issue with the variable being not in the context during the test run and it needs to be cached.

Double checked it and it works.

boombatower’s picture

StatusFileSize
new1.5 KB

Apparently this code hates me. The link was getting messed up. I have to believe some other core code changed underneath this.

/me crosses fingers.

Status: Reviewed & tested by the community » Needs work

The last submitted patch failed testing.

boombatower’s picture

Status: Needs work » Reviewed & tested by the community
StatusFileSize
new1.8 KB

Last two patches fixed the issue...but the SimpleTest test brings up a weird edge case...since it actually loads itself within a test...so you get a double depth files folder.

Fixed by adding file_check_directory().

webchick’s picture

Status: Reviewed & tested by the community » Fixed

Committed to HEAD.

moshe weitzman’s picture

This is sweet functionality. I turned it on and ran the comment tests and saw nice verbose links in the results. But when i click on the link, my firefox refuses to show the resulting page and safari nearly crashes on it. verbose.html gets huge fast. and it isn't valid html as it is just the body of an html page.

IMO, the logging should be done to separate files, one for each verbose invokation. And we should log to simple text files. the meta info should be in headers like email. that would scale much better. I know that some folks like "seeing" the HTML of the debug message in a browser by default but usually the caller just ends up doing a view source on that anyway. further, all HTML sent into verbose() has to be htmlspecialchars() beforehand. Really, text is best IMO. In any case, multiple files are needed. I can't imagine the hugeness of the file if it was enabled during a run of all tests.

I'm willing to write a patch if folks agree with my approach.

yched’s picture

Status: Fixed » Active

Agreed that the verbose mode is currently not usable in real life, brings browsers on their knees (at best).
Maybe we could just log the latest $this->drupalGetContent() only when a test fails ? (granted, that might be a completely unrelated request and you need to be a little smart when debugging; but the current implementation has the same "drawback": when debugging a failure, the latest drupalPost logged might be completely unrelated)

I'd find it extremely cool if we could preserve the current 'real HTML to see in a browser' feature, though - really more handy in a lot of cases to see whats wrong at once. If I need to investigate the markup, firebug remains my favorite primary tool anyway.

At any rate, separate files per logged page sounds a must.

boombatower’s picture

Yea, I'm not a 100%. I use the functionality (as is) when writing tests because I only run one part of the test. I like being able to just scroll and see the path the browser took.

Before I wrote this patch I used to do it manually so I could only output the pages I need. Perhaps that is the best solution (although would change meaning of verbose)? Add a parameter to drupalGet|Post() to dump verbose...or just allow the verbose method to be called after drupalGet|Post().

Putting each call in a separate file is ok, but being able to scroll is extremely useful. I would be ok with either approach.

yched’s picture

well, it seems opening logged pages in separate tabs can compensate the 'scrolling through history' ?

boombatower’s picture

Possibly.

So if we do separate files...I would recommend creating a verbose directory where the current verbose.html is and then placing each of the files in there named related to the ID and then the anchor can be removed from output.

I think we should create separate issue as this is a followup...lets lets this issue rest.

boombatower’s picture

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.