Posted by boombatower on July 17, 2009 at 8:50pm
2 followers
Jump to:
| Project: | Project Issue File Review |
| Version: | 6.x-2.x-dev |
| Component: | Code |
| Category: | feature request |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | active |
Issue Summary
When a client detects a failure when testing a "patch" it should re-test on clean version of branch and auto-disable itself if the clean copy fails as well.
Comments
#1
I'm not sure this makes sense..
Are you saying it should test the *branch* and then disable the *branch* if it fails? That makes sense. Disabling the testbot doesn't make sense.
#2
I believe this stemmed from a conversation with webchick about trying to automatically determine if a client is malfunctioning. So re-test with the branch (aka no patch applied) and if that fails as well we assume the client is bad since any branch failure should postpone all patches against the branch.
#3
Assuming the testbot is bad because the branch test fails doesn't make sense IMO.
However, we now confirm against a *tag*, which doesn't change underneath us. So we could reconfirm the testbot every time some *trusted* branch failed. Could be pretty expensive. Means adding the idea of a trusted branch as well, I think.
Examples D7 was broken a whole pile of times by shifts in core D7. That's while D7's tests were still functioning.
I'd rather see us just have more stable testbots, and think we're making at least some progress.
#4
If a client tests a branch and marks it as failed then all patches will be postponed therefore if a client is testing a patch then the last commit to the branch passed so if the client retests the branch and finds it fails then the client has an issue with the branches code or is broken in general. Since the issue may be specific to a client and branch and doing this does not require redownloading simply reverting the patch. git checkout .
Obviously, we can get inconsistent test results or the time period between commit trigger and test that could through this off. Was just an interesting idea.