With the current API, once a worker has claimed an item, all it can do is:

  • either process and delete it when it's done
  • or die if it cannot process it

But in the latter case the job won't be available for another worker until the item timeout expires.

The attached patch includes a releaseItem() method for this.

CommentFileSizeAuthor
#3 queueRelease1.patch1.18 KBfgm
queueRelease.patch721 bytesfgm

Comments

chx’s picture

chx commented
Status: Needs review » Needs work

Please roll a patch with the -up parameters. (There is a handbook page) For reviewers: we discussed this in IRC and fgm claimed SQS can do a ChangeMessageVisibility(0) for release and beanstalkd has a release op. I am not so sure, the SQS docs say You could call ChangeMessageVisiblity with a value of two hours and the effective timeout would be two hours and 30 minutes. -- so calling it with 0 is a no-op?

damien tournoud’s picture

damien tournoud commented

For what I understand, ChangeMessageVisiblity actually works like set_time_limit(): it simply sets the running timeout counter to the new value. In that sense, if your item is about to timeout, calling ChangeMessageVisiblity(300) actually "extends" its visibility timeout.

The example paragraph in ChangeMessageVisiblity is actually misleading.

fgm’s picture

fgm
Primary language French
Location Paris, France
commented
StatusFileSize
new queueRelease1.patch1.18 KB

Patch rerolled. For reference for reviewers:

fgm’s picture

fgm
Primary language French
Location Paris, France
commented
Status: Needs work » Needs review

Forgot to set status.

chx’s picture

chx commented
Status: Needs review » Reviewed & tested by the community

I am fine with this after all. Given that there is no ordering guarantee if a queue implementation cant do this, there is no harm, after all and it can be useful.

dries’s picture

dries commented

What about DamZ's comment in #2? Is there a documentation improvement required?

fgm’s picture

fgm
Primary language French
Location Paris, France
commented

I think that the documentation DamZ is referring to is the one provided by Amazon on their site.

In our case, releasing would be performed by a SQS implementation of DrupalQueue by setting visibility to 0, anyway.

dries’s picture

dries commented
Status: Reviewed & tested by the community » Fixed

Thanks for clarifying. Committed to CVS HEAD. Thanks.

yched’s picture

yched commented

Just a note that this broke #629794: Scaling issues with batch API, which lies in the RTBC queue for a month.
No harsh feelings, but before updating the patch for the 6th time :-p, I'd like a word of confirmation from drieschick that it has a chance to get in.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.