Automate bug reviews that confirms new tests trigger the existing bug
| Project: | Project Issue File Review |
| Version: | 6.x-2.x-dev |
| Component: | Code |
| Category: | feature request |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | active |
Jump to:
Part of the point of test-driven development is that when you find a new bug, you're supposed to write a test that triggers the bug, verify your test fails, then write your fix for the bug, then re-run the test and watch it pass. I'm doing this manually a lot now as I'm working on update status tests, for example, at #319033-25: Weird order of projects listed on updates page. It'd be slick if this could somehow be automated by PIFT/PIFR. I see two basic approaches:
A) PIFR has some heuristic to decide what hunks of a patch are for new tests. It just applies those, runs the tests, and expects to see at least something fail. Then it applies the rest of the patch, re-runs the tests, and expects to see everything pass.
B) The developers upload separate patches, one for the new/modified test case(s), and one for the code fixes, using some filename convention (TBD). PIFT sends both to PIFR. PIFR does basically the same operation described in (A), only there's no heuristic guess-work involved, there are just *.test.patch and *.code.patch files (or whatever) which the developer is responsible to keep separate.
A would be slick if it could be reliably automated (maybe just searching for *.test and */tests/* would work?), and would be a less drastic change to the workflow. However, the person uploading the patch would still need some way to request this special behavior, so maybe (B) is better anyway, so that it only happens if you go a bit out of your way to upload your work that way...
I mentioned this in IRC and boombatower said he's just now refactoring the abstraction about different "review types", so this is a timely request it seems. So, I'm opening this issue for feedback, and as a place to discuss exactly if/how it should work. ;)
Thanks!
-Derek

#1
actually, A) seems to be doable, as it requires to only apply the patch to the *.test files, (and possily *_test.* files.)
also, when refacoring stuff, only applying he test should also cause fails, that should be fixed when the full patch is applied.
more TTD is always good!
when it notices the patch contains only tests, it skips this phase.