http://drupal.org/node/325481 demands that we have realtime knowledge of what files PIFT will send for testing.

so the adding of files to {pift_data} needs to be moved to hook_nodeapi() insert op, and hook_comment() insert op, for initial issue attachments and comment attachments respectively. unfortunately, that's not the end of the story, b/c for d.o we still need to provide a way to record the files that pift wants to send that were attached prior to installation of the module.

if all we care about is getting these old files on d.o, then we might be able to leave the existing code in place, wait for the completion of a cron run, then quickly switch in the new realtime code -- not the most foolproof method, but probably close enough.

i don't see a need to change the cron-based nature of sending the files, because this both reduces the number of XML-RPC queries executed to testing.drupal.org, and doesn't leave people in the position where they have to wait for that call to t.d.o every time they submit a patch.

Comments

catch’s picture

if all we care about is getting these old files on d.o, then we might be able to leave the existing code in place, wait for the completion of a cron run, then quickly switch in the new realtime code -- not the most foolproof method, but probably close enough.

This sounds fair enough to me - we're in a fairly unusual position of having active patches going so far back. So does that just leave moving the inserts to hook_nodeapi() and hook_comment()?

hunmonk’s picture

@catch: we have to move the code, test it, and deploy it on d.o in a sensible way.

i don't really have time to work in this right now. would be great if somebody could take up the ball for a little bit.

catch’s picture

Status: Active » Needs review
StatusFileSize
new1.51 KB

Not set up to test this, but here's an untested patch - having spent approximately 10 minutes looking at this module I might be missing something obvious, but it seems like the call to pift_server_add_new_files() just needed moving to the insert $ops and absolutely no other changes (which I think is what hunmonk asked for).

boombatower’s picture

Hunmonk I'd be interested in working on this.

catch’s picture

StatusFileSize
new0 bytes

pwolanin suggested an _update to add files to the queue just after the last cron run and before the next node/comment insert. Seems reasonable so added it in.

catch’s picture

StatusFileSize
new2.35 KB

real patch file.

hunmonk’s picture

Status: Needs review » Needs work

geez, did anybody even read how the add file function works?? ;)

the cron based approach tracked file ids to know what it last checked, but we don't want to do that now. at the time an issue or followup is submitted by the user, we want to examine every file that's attached, compare it against the regex for what pift considers a patch file, and then add the file to {pift_data} if necessary. note that because project, rid, and sid can all change, we cannot use these to filter what files go into {pift_data} for tracking. _all_ files attached to issues or followups that fit the file regex must be put into {pift_data}

the add files function should either completely die, or be converted into a script that an admin would run once to get any old files added to the {pift_data} table before they turned their site back on out of maintenance mode.

sorry guys, i don't think this is going to be just a few lines of change...

pwolanin’s picture

Status: Needs work » Needs review

ah, looked too easy...

hunmonk’s picture

Status: Needs review » Needs work

@pwolanin: i'm not sure why you set this back to CNR -- i won't accept the patch as written, it needs to be adjusted as i outlined in #7!

catch’s picture

Status: Needs work » Needs review

Whoops.

Completely untested again - use upload_load() and _comment_upload_load_files(), pass the $files array into a cut down version of pift_server_add_new_files() - move pift_server_add_files to become the update function itself. If we want a one-off script other than the update then that shouldn't be too hard to add, but doesn't seem necessary for this initial refactoring.

hunmonk’s picture

Status: Needs review » Postponed (maintainer needs more info)

guys, i'm not seeing any patches attached to these followups -- are we having a technical problem w/ attachments?

catch’s picture

Status: Postponed (maintainer needs more info) » Closed (fixed)
StatusFileSize
new7.46 KB

hmm, here's another try.

catch’s picture

Status: Closed (fixed) » Needs review

grrr

hunmonk’s picture

Status: Needs review » Needs work

an update doesn't make any sense here. pift_server_reset_send_queue() is called during initial install with the express purpose of making it so that the server begins to operate on files that are attached after it's initially installed.

i believe this is the most sane way to install the module. the only other sane way to do it would be to use batching to examine and insert all appropriate files related to issues upon initial install -- which is a really bad idea for d.o.

so if operating only on files since the module was installed is our installation strategy, then we don't want an update that grabs any old files, and since we're inserting new files dynamically, then the entire fid tracking system should be removed. this includes a rewrite of pift_server_reset_send_queue(), removing pift_server_get_newest() and pift_server_get_last(), and using update 1 to remove the associated dead variables.

the script which would grab old files would ideally have some settings you could tweak on top for fid-related file insertions. this would make it possible to grab any chunks of old files that the site admin wished for tracking by PIFT. if we don't want to make that script a part of this patch, that's fine. we can follow an upgrade procedure on d.o like this:

  1. take the site down.
  2. run cron to grab any files between the last cron run and the time the site was taken down.
  3. upgrade the code and test it real quick
  4. bring the site up
catch’s picture

StatusFileSize
new7.3 KB
hunmonk’s picture

  1. return array(); needed in update function
  2. $files = _comment_upload_load_files($comment->cid); should be $files = _comment_upload_load_files($comment['cid']);
  3. there's no $file->uid returned from _comment_upload_load_files(), which is needed in {pift_data}
  4. if ($node->type == 'project_issue' && arg(0) == 'node' && is_numeric(arg(1))) { this conditional prevents 'insert' op from ever running in pift_server_nodeapi()
  5. you're only using the issue-specific insert statement in pift_server_add_files(), if you'll look at the original function, the insert statements for original issues and for followups are different.
catch’s picture

Status: Needs work » Needs review
StatusFileSize
new7.88 KB

Thanks for taking a proper look. This one should fix those.

catch’s picture

StatusFileSize
new7.91 KB

Missed a node type check the insert op extras.

hunmonk’s picture

Status: Needs review » Fixed
StatusFileSize
new12.01 KB

the last patch needed a fair amount of massaging to get everything working well. attached patch was well tested for both issues and followups, and has been committed to 5.x and HEAD.

catch’s picture

Thanks hunmonk!

hunmonk’s picture

and to complete this task, i've committed a fancy update script that allows site admins to grab already existing files and add them to the testing queue: http://drupal.org/cvs?commit=150769

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.