Testbot TNG - Various Automated Testing Tab UI Mockups

Full-size.

This would only be the initial settings page, which would determine whether a particular test plugin is enabled for a project ... There would be a new page for actually queuing a test, which would allow specifying additional test properties (dependencies, patches to be applied, etc.). I suppose you could include another fieldset to include the 'project default' settings on this page for each plugin.

To address the scalability issue with multiple branches, i suppose you could autodetect the number of options available, and change the checkboxes to a multi-select for projects with a large number of branches.

From an implementation perspective, the available options are going to vary with each new testing plugin type ... I'd envision each plugin would define the contents and processing of their settings page fieldset (and 'queue test' page fieldset) inside of a ctools plugin installed into a plugins directory with the module.

I find the 'textbox with documentation' approach somewhat non-intuitive ... if for no other reason than the fact that most people won't read the documentation. ;) My goal here would be to provide a UI for the common options that make the most sense on a per-plugin basis, and then probably an expandable fieldset/textbox for any 'advanced' settings or customization which would be tweaked by the power users.

I also prefer to centralize the testing settings instead of distributing them, so that we can take full advantage of the inheritance ... patches inherit the sensible defaults from their associated branch, and branches inherit from their associated project - I'd be concerned with the ability to track these dependencies from one element to the next (and suspect we'd see a lot of unnecessary settings duplicated in each location as a result). And when we extend the concept to things outside of the realm of 'unit testing' (such as the 'repository health check' test I use as my common project apps example), then we start to run into items which may not necessarily have an obvious home outside of the parent 'project' level - which is where this tab sits now. In any case, we can mock up both approaches and run them by other folks to get a better idea of what is more intuitive from the 'newbie' perspective.

My splitting of simpletest patch versus simpletest branch may have been a little misleading ... my intention was to have the default settings for each 'testing plugin' located on one page; but I split apart branch testing and patch testing as two different plugins. My thought at the time was that some projects might want to run only branch tests, but not patch tests - but not that we'd split branch from patch for every single test plugin. (One use case for the above split is when a contrib project has a test suite that doesn't actually pass. The module author may commit test fixes in an attempt to get them to pass, and will want to see the results after each commit ... but you wouldn't want to turn on 'per patch' testing within the issue queue until you've reached that first 'known good' state.)

I'll work on an update of this mockup, and try to start down the path of how to render the various 'results' summaries - As a 'set and forget' interface, I think having each of the plugin settings forms on one page is workable ... but I've struggled a bit in visualizing an efficient 'testing results' page for the d.o side of things.

New version.

First crack at a 'Git Repo Health Check' results page ...

Updated issue summary.

First crack at a 'Simpletest Branch Testing' results page. (Patch results would be within the issue queues themselves.)

Updated issue summary.

Added simpletest branch result mockup

#12: "Remove from list" is probably a better fit for what I meant that link to do ... but things are starting to run up against real-estate issues now, unless we drop the standard project sidebar.

Updated issue summary.