Job Queue integration
| Project: | Notify |
| Version: | 6.x-1.0 |
| Component: | Code |
| Category: | feature request |
| Priority: | critical |
| Assigned: | Unassigned |
| Status: | needs work |
Jump to:
There are some problems with the Notify module on large/slow sites.
The first problem is that Notify uses time() all over the code, expecting it to return the same value each time. That's not how time() works :-) The end result is that Notify may include certain nodes more than once, or may miss certain nodes.
The second problem is that Notify will try to re-run at each call to cron(), unless it manages to get to the end. It seems to want to re-run from the start of the list.
If sites limit the total runtime of PHP scripts, or the total runtime of cron jobs, then Notify will never make "real" progress, and (worse) will spam users that are early in the table with many multiple copies of everything, whereas later users will get nothing.
I'm proposing this patch to alleviate problem 1. To solve problem 2, it's harder. Cron really should keep track of which user has gotten a "try" for each period, and try to send to the first N (say, 100 by default) users each time cron is invoked within the send period, until all users have been sent. This will allow Drupal to finish, and commit to the database, even on sites with limited runtimes. I am considering a patch, but I'm not versed enough in drupal db core stuff to be sure I do it right, so any feedback on how to solve 2 is welcome!
| Attachment | Size |
|---|---|
| notify_patch.txt | 3.29 KB |
#1
I'm also having problems with the same email being sent multiple times to some of the users because cron fails to complete the list of recipients. Although I'm not sure why, it has gotten worse lately: I went from sporadic cron fails and 1 or 2 repeated emails to having all cron jobs failed and only 3 or 4 email successfully sent (and "perpetually" repeated) per cron job. Might there be some kind of cumulative effect causing this issue?
I also came across this Job Queue project that seem to be a great candidate to solve this problem. Am I correct? Did anyone experiment with it?
Thank you.
#2
Well, I just did: simply replacing the call to "drupal_mail" following the example from Job Queue module works and, I believe, should solve the 2nd problem reported by jwatte.
In my case, it also showed that my problem is not within Notify but it's the smtp module that's breaking the cron job (will try an smtp issue). Still, with Job Queue and a "high frequency" cron all mails get dispatched.
Hope it helps.
#3
A better look at the emails sent showed me that the notify + job_queue option previously referred ends up with wrong usernames and uid in the emails. In order to have it right, some more tweaks are necessary on "notify_mail" function.
#4
This issue is a priority for me, but I'm going to be swamped for the next couple months on some large projects. I'd be thrilled to bring on a co-maintainer to work on this issue.