Provide tarballs/zip files of patches + projects combined for easier review

I'm not sure what's the right queue to discuss this in, either, but here's fine. ;) Could also be considered part of cvs.module, versioncontrol_project (if only we were using that), project/project_release, pift/pifr, and/or d.o infra stuff... But yeah, let's talk here for now.

Yeah, there are a lot of potential issues here, but they're all fairly easy to solve. You enumerated most of them. The only other I'd add is this:

A) What version of the project do you presume to include in these tarballs? Obviously, the version of the issue at the time the comment is posted would be a good first approximation, but what about backports, etc? Like the test bot, we'll have to do some magic based on parsing the filename, too. That's even more complicated for contrib. What about -dev releases? Do we just grab the end of that branch when the tarball is requested? Do we always grab the state of the branch at the time the comment was posted? What about patches that don't apply? Etc. All of this is completely independent of $(D)?VCS de jour.

Otherwise, yeah, the potential concerns are:

B) Do we just do this on demand or do we always create them as patches are uploaded?

C) When/how do we do garbage collection?

D) What kinds of ACLs do we want on these links for blocking search engines etc?

E) (Unimportant in the scheme of things, but perhaps worth thinking about): What's the filename convention for the resulting .tar.gz and .zip files?

Oh, and one other thought:

F) Doesn't the test bot already have to do something like this, anyway? If so, couldn't it just be responsible for this? It's already altering the display of these patch download tables to inject test results. Why doesn't it have its own place to temporarily stash tarballs of the thing it tried to test, and inject links to those, too?

After lectures at length why such a feature is damn hard to do why does everyone have to patch? [2006] The testbot project has proven that at least the technical part of this is now possible in theory...
I still have concerns about the social part - "if you're not comfortable with patches, this code may hurt you"

me thinks [project_short_name]-[issue_num]-[reply_num].tar.gz shows provenance more clearly than misc filenames (I still don't have a good naming scheme for those). As these bundles are auto-generated from a very specific point and place, that is a consistent UID.

I think the demand vs benefit vs server load vs storage and spidering issues point at generation-on-demand rather than build-and-store.
As per point A, This also means that a patch is always fresh, not a snapshot in time. ... which cuts both ways.

Had a long standing idea to just build a live demo site on demand (for quick review especially when away from dev environment and for UX ppl), and given another 10-20 hours should be able to make it happen. Already have a start at pifr_demo and should be trivial to allow a download of the demo environment that will have already been built.

PIFR Demo sounds cool, but it seems like an easy first step is at least the tarball for the tested code. Part of the benefit of decoupling these is that we can get folks with different environments to test the same code that way...

I'll probably just make a two link, and or a link in the setup site.

The demo server will definitely not be running on qa.drupal.org. It may not even have a domain name associated with it. We were thinking of the old t.d.o server..and we can easily use a different domain or straight IP.

The current design involves a separate computer...I never want patched code running on the main qa.d.o server for fear of bring the central server down and such (since it cannot be firewalled and such). I was also planning on ensure each person gets their own site with unique login credentials (most likely auto login with crazy password they can change if they want to test login or what not). Not sure if we want to run some sort of demo module or something, but the site will also only be around for an hour or so before it gets fried.

Virtually done with base: #342509: Add the ability to test a patch in an auto-generated environment.

My thought at the moment is to just provide the tar balls during the build always, and then I just need to figure out how to link to them.

Best filename I could come up with given limited information on client and that would be unique:

Upon the completion of generating the environment, the user is presented with, or a button to login.

Committed.

Now we just need to deploy this baby.

Nice to see this progressing here! A few concerns:

A) Excuse me if I don't understand the PIFT + PIFR architecture since 2.0, but PIFR Demo is part of the PIFR client which runs on the test clients, not on the test server, right? If so, looking at the diff you committed (http://drupal.org/cvs?commit=326234 is what I found), it appears that all that's happening is some .tar.gz and .zip files are getting created on the test clients. Those files need to be transfered back to the test server for them to be useful. Otherwise, what about opportunistic test clients "in the cloud" that come and go? These tarballs need to persist longer than a given test client is alive (although not forever).

B) Speaking of not forever -- what about garbage collection?

C) Re:

Best filename I could come up with given limited information on client and that would be unique:
date('Ymd') . '-' . $this->test['test_id']
example:
20100209-2.tar.gz

Really? Ugh. ;) Can't this be based on issue nid and commit id (and project shortname, for that matter)? Surely *something* knows the correlation between those ids and the patch file being tested, which makes updating the results at the issue possible. If the clients don't currently get any of that info, can we easily pass it along to them? Or, when the files are moved back to the test server, can't we rename them based on the thing that the humans using this feature are going to want to know?

Thanks,
-Derek

A) The clients can run in the cloud, but as I originally intended (and still do) for users to be able to just live demo the site...this client (pifr_demo) will need to have a visible IP and thus the user can just download from there.

B) hook_cron() fries instances older then 1 hour.

C) It would of course be possible, but as of 2.x I have removed all the Drupalisms from PIFR core, so only the plugins are drupal specific. As such it doesn't keep track of issue nid and such. Since these downloads are created on demand, it shouldn't be too bad, but as of now even qa.drupal.org doesn't "directly" know the nid...although it store a "link" field which is an absolute url to the issue nid which will also contain the cid. So I may be able to hack that a bit.

Assuming everyone is good with these details I'll just take a look at the file naming when I have a time block again.

#709580: Add link to demo patch using pifr_demo

A) I don't think that'll do. If this were built on top of a real batch scheduling system, transferring output from the execute clients back to the server would be an integral part of the process. This feature is much less useful if it depends on *which* test client happened to test a given patch. I see no reason why the clients can't have a mechanism to send files back to the server and we host all the tarballs there.

B) Woah. For this to be useful, we need these tarballs to last O(days), not 1 hour. ;)

C) Again, even if the plumbing isn't drupal-specific, the plumbing should have a way to send meta-data along with the jobs... Anyway, short of that, yes, I think a hack is in order here, since filenames like 20100209-2.tar.gz are going to suck for the non-technical audience that's trying to use them.

A) They could easily have a mechanism for sending back files. However I see the more useful side of this to be the live demo, as that makes it infinitely easier for a UX person or whatever to just see it without even having to setup drupal or anything.

This also doesn't depend on which client, as this will only occur on-demand and be sent to our demo client.

B) We can leave them longer, but it shouldn't take long at all to gin one. Personally I don't see any reason to leave them hang around as they may become outdated at any time (commit) and I don't want to build something in to attempt to update them after commits...easier to just rebuild when needed.

C) It does have a method for sending meta data and I've added the information we need in patch. In IRC we decided that including all the necessary information in the filename is too much, so just use a standard filename and add the verbose stuff to a text file (BUILD.txt).

Committed. (as usual re-open if necessary)

boombatower, this seems like a great improvement to me and i just wanted to ask for a quick explanation of the impact of your final commit.

from my understanding, this targets providing test-links for patches on the PIFR demo platform. what about the original idea of providing a download link for the patched module? is it included yet / planned?

i just had outlined a similar proposal at #719882: Enable patches to the average drupal user, including further thoughts on a patch_watcher module which could keep track of installed patches and therefore assist the user with upgrading such modules and provide feedback to the issue queue. thanks to dww for pointing me here...

We can rework this some if we want to generate them and store them...but this method generates the environment on the fly and installs drupal so they can play with it. Upon initially viewing the site they are prompted to download as tar.gz or zip if they so desire.

Assuming the branch the patch applies to can change at any time I think it is more useful to build them on demand...not sure if we want a link that skips the install part (slowest) and just generates download links.

To summarize:
1) Link on patch
2) Wait for build
3) Prompt for download
4) Play with site if desired

i believe that there are two user-stories related to this issue
1) i want to test this patch on my existing infrastructure that i am familiar with - download patched module
2) i want to test this patch online - on the fly generation of test environment

from my personal point of view i see 1) more appealing, because as a site builder, i encounter bugs while having everything set-up yet.
on the other hand side, option 2) might come in handy for non-technical users who just want to help out in the issue queue.

Which this already supports. It is merely implementation details.

Although as I believe I have mentioned #1 seems far less likely since people who are building sites will most likely: a) be using a cvs checkout, and/or b) be able to patch easily.