subscriptions_number_of_mails should be an option on settings page.

thanks
introfini

Comments

salvis’s picture

salvis
he/his
commented

The problem with subscriptions_number_of_mails is that you have absolutely no idea how high you can set this. If you set it too high, you risk a time-out in cron and may lose queue items.

chx intends to look into controlling the cron loop based on a timer, as moshe has suggested in http://drupal.org/node/196771, and then subscriptions_number_of_mails would go away, so I'm not exposing it yet.

introfini’s picture

introfini commented

Yes, I found that the hard way. My server was crashing and I never thought this was the problem.

I had cron running every hour and the last time I checked the subscriptions queue was 40.000 rows!

introfini

salvis’s picture

salvis
he/his
commented

Ouch! Pretty hefty testing environment you have out there...

:-)

salvis’s picture

salvis
he/his
commented
Version: 5.x-2.0-beta7 » 5.x-2.0-beta12
Status: Active » Fixed

The timer-based control is implemented now (#196771: Use a timer to control the cron job), but because of #233079: status of queuing and cron UI or controls? I've also implemented the option to manually limit the number of emails sent, as you originally requested.

A new watchdog message will show you how many notifications are sent and how long it takes. Would you share some of these, so we can some real-world numbers?

introfini’s picture

introfini commented

Hi salvis,

Yes I'm glad to help. I'll update and let you know the results.

Thanks,
introfini

introfini’s picture

introfini commented

Some results:
________________________________
Subscriptions sent 134 single and 0 digest notifications in 39 of 226 (240) available seconds; queue items left.
Subscriptions sent 25 single and 0 digest notifications in 5 of 233 available seconds; queue items lef
Subscriptions sent 408 single and 1 digest notifications in 51 of 192 (240) available seconds; queue items left.
Subscriptions sent 448 single and 0 digest notifications in 61 of 183 (240) available seconds; queue items left.
Subscriptions sent 636 single and 1 digest notifications in 79 of 158 (240) available seconds; 1540356 queue items left.
Subscriptions sent 632 single and 3 digest notifications in 72 of 145 (240) available seconds; 1542735 queue items left
Subscriptions sent 11 single and 0 digest notifications in 10 of 19 (240) available seconds; 1550080 queue items left.
Subscriptions sent 359 single and 0 digest notifications in 63 of 126 (240) available seconds; 1550080 queue items left.
________________________________

The number of items in the queue never got a match to the number of rows in subscriptions_queue table.
The statistics simple stop showing in the log after a couple of days.

Regards,
introfini

salvis’s picture

salvis
he/his
commented

Ah, it should say

SELECT COUNT(*) FROM {subscriptions_queue} ...
       ^^^^^^^^

for !remaining_items. Thanks!

The numbers will never match (unless you've limited Send Intervall to "Immediately"). The items left number is the number of notifications that are actually ready for sending, but haven't been sent because there wasn't enough time left. Due to the bug, the items left numbers are meaningless — please post some more after correcting it.

Apparently, you have some other processing done during cron, that takes quite a bit of time, ahead of Subscriptions.

introfini’s picture

introfini commented

Ok, you are right (of course) about the matching numbers :-)

I’ve patched the module with your fix. Here are some more statistics:

Subscriptions sent 406 single and 0 digest notifications in 118 of 236 available seconds; 10364 queue items left.
Subscriptions sent 447 single and 4 digest notifications in 119 of 238 available seconds; 8693 queue items left.
Subscriptions sent 483 single and 1 digest notifications in 119 of 238 available seconds; 7055 queue items left.
Subscriptions sent 537 single and 0 digest notifications in 117 of 233 available seconds; 4979 queue items left.
Subscriptions sent 576 single and 2 digest notifications in 119 of 239 available seconds; 3423 queue items left.
Subscriptions sent 630 single and 2 digest notifications in 119 of 238 available seconds; 1323 queue items left.
Subscriptions sent 792 single and 4 digest notifications in 103 of 235 available seconds; 0 queue items left.

salvis’s picture

salvis
he/his
commented

Very interesting numbers! Did you run cron repeatedly by hand to get the items left down to 0?

Now you always have 220+ seconds per run? In #6 you had one run with only 19 seconds left for Subscriptions — some other module must be pretty hungry, but only sometimes...

introfini’s picture

introfini commented

Yes, this time I ran cron by hand.

I think it's User Stats module (http://drupal.org/project/user_stats) that is working hard on cron. I've changed the number of users it updates from 200 to 10.

salvis’s picture

salvis
he/his
commented

Going from 200 to 10 made a huge difference. I'm not saying Subscriptions should have it all, but you did have a backlog, so you just might want to run cron more often.

Anonymous’s picture

Anonymous (not verified) commented
Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.