Hello,

Several months ago, when I was sending out regular newsletters, Simplenews + Mimemail were sending out 500 emails in about 90 seconds, using cron.

Now that I'm starting to send newsletters again, the best I can do is 200 emails in 115 seconds. (To stay within a max_execution_time of 300 seconds.)

What's strange is that another domain being hosted on the same codebase is taking 160 seconds to send each batch of 200 emails. And for further comparison, I have a completely different mailing list application (ListMailPro) which sends over 25 emails PER SECOND to the Exim queue.

Site performance overall is good, except in sending newsletters with Simplenews+Mimemail.

My sites are running on a VPS with 1G RAM and equal-share CPU time.

Here's perhaps the strangest part of the situation -- Adding PHPMailer did not change the sending speed at all, even though test show that emails ARE going through that email engine.

The ONLY thing that seems to offer any clue as to what may be happening is that 2 domains running off the same codebase are sending at different rates. There are differences in the modules enabled for each site, and a few differences from when 500 emails would go out in about 90 seconds.

The primary modules that have been added are:

Advanced help 6.x-1.2
Content Construction Kit (CCK) 6.x-2.8
Date 6.x-2.6
Google Analytics 6.x-2.3
IMCE 6.x-1.4
SWF Tools 6.x-2.5

In my testing process, I've added Supercron, which is tracking the time each module spends in it's cron process. This was added after the most recent newsletter issue, so I don't have data on the actual sending process. However, all data collected shows that very little time is spent on any cron process.

I've also installed the Devel module to log query performance. Again, after the last newsletter issue, so no data has been captured from the sending process.

I'm hoping that someone may have some clue as to what may be slowing things down here, and a hint to help restore earlier sending performance.

Thanks.

- Alan

Comments

simon georges’s picture

simon georges commented

Hi, do you still have the problem ? Could you debug something ?

AlanT’s picture

AlanT commented

Yes, I still have the problem. I have set up a multi-site configuration with 1 site dedicated to sending Simplenews mailings. While this helped, the sending speed is still far less than what it used to be a year ago.

Typical mailings go out at a rate of 200 emails in 115 seconds. (This is the site which was taking 160 seconds before.) And this is with practically all modules disabled except for those absolutely necessary to send Simplenews mailings.

If you can give me debug instructions, I can follow and report.

simon georges’s picture

simon georges commented

Can you identify which part is taking long ? Do you have access to your mail.log to see if this is really Simplenews or the communication with the mail server ?

AlanT’s picture

AlanT commented

I have root access to the server, but not sure where to find mail.log, or how to interpret what may be in it.

If you can point me in the right direction, I'd be happy to take a look.

simon georges’s picture

simon georges commented

By looking at the lines, you should be able to see at what times the mail are sent, and understand if this is Simplenews that slows or your server that doesn't process fast enough.

simon georges’s picture

simon georges commented

(mail.log is usually under /var/log on Debian / Ubuntu / Fedora systems)

simon georges’s picture

simon georges commented
Status: Active » Postponed (maintainer needs more info)
AlanT’s picture

AlanT commented

Sorry for the delays in responding. Drupal.org doesn't notify me when there's a new comment, and I don't come here very often.

I don't see any mail.log files, but I see a number of maillog files. maillog, maillog1, maillog2, etc.

I downloaded them and opened with a text editor. So much information, and I'm not sure how to find the specific lines involved with the mailing.

Right now, I'm looking at the logs for the last major mailing I did on 4/21, around 14:45, and most of what I see seems to indicate my desktop computer checking for new messages and various messages being checked for spam. There are a few entries for the CPanel, but nothing that looks like a mass mailing.

Logs for later that day do show a mass mailing sent out by one of my other accounts (my church). I see nothing in the whole file that looks like a mass mailing from the site in question.

Perhaps I don't know what to look for. Any suggestions?

simon georges’s picture

simon georges commented

You should have one line per email indicating that it's going out (with the email adress it is sent to). By reading the logs, you could see if the mails are sent slowly, or if other treatments (anti-spam, anti-virus) are happening that are slowing down the sending (I'm trying to ascertain how this could be Simplenews-related as you seem to be the only one affected).

AlanT’s picture

AlanT commented
Status: Postponed (maintainer needs more info) » Closed (fixed)

I have something new to report, which may explain the situation.

Today, before sending out my newsletter, I created a mail.css file to customize the styling. Specifically, I started with the theme style.css, added the content from the color stylesheet, and added a line to define a maximum width for the body tag. (I got tired of seeing unrestrained text in HTML emails.)

When the mailing went out, it went out a LOT faster. 200 emails in less than 23 seconds, instead of the 115 it was taking before. 5 times faster.

This makes me think back, and yes, I was using a custom mail.css file last year when I had the better sending speeds.

For some reason, it's the process that collects theme styling that was slowing the process down. Using a mail.css file speeds it up.

This points to mimemail as being the primary suspect in this case.

Now that we have this new data, I can see why the 2 different domains had 2 different sending speeds. They used different themes, and the process of collecting theme css files may take longer for one than the other.

I've marked this issue closed, as the answer has been found. Perhaps a note could be added to the documentation to indicate that a mail.css file will speed up the sending process when using mimemail.

Thanks for your help.

simon georges’s picture

simon georges commented

Thanks for coming back to indicate the solution !
I'll keep that in mind for future references. I'll eventually see with mimemail maintainers if there is a reason / something to do about this. Thanks again.