Optimize a SELECT * FROM query that is being killed in a shared hosting server due to execution time exceeds 15 seconds

tcibah - July 8, 2009 - 08:54
Project:Mailout
Version:5.x-1.x-dev
Component:Code
Category:task
Priority:normal
Assigned:Unassigned
Status:active
Description

The error happens only during cron runs.

The error suddenly happened when tables mailout_message_log & mailout_log reach 9,569 records & 307 records respectively. If those tables are truncated, there will be no errors.

What statement causes the error? Any suggestion to optimize it?

The information above is a summar of the following:

Annex 1: Hosting Provier Response
I see the mysql connections being killed for the script as it is running for too long.

[ 1246442407 ] [ mycpid ] [ 1756729 ] [ 19 ] [ mycpid_site] [ Query ] [ KILLED ] [ 'myserver.website.com' ] [ SELECT *,
[ 1246703581 ] [ mycpid ] [ 2703020 ] [ 25 ] [ ] [ Query ] [ KILLED ] [ ] [ SELECT *,
[ 1246703582 ] [ mycpid ] [ 2703021 ] [ 24 ] [ mycpid_site] [ Query ] [ KILLED ] [ 'myserver.website.com' ] [ SELECT *,
[ 1246706642 ] [ mycpid ] [ 2715851 ] [ 18 ] [ mycpid_site] [ Query ] [ KILLED ] [ 'myserver.website.com' ] [ SELECT *,
------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------
Annex 2: The error message thatwe get when the cron is run manually:

Warning: Lost connection to MySQL server during query query: SELECT * FROM system WHERE type = 'theme' in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173

Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'mysql_query() [<a href='function.mysql-query'>function.mysql-query</a>]: Unable to save result set in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 155.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173

Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'MySQL server has gone away\nquery: SELECT ml.mmid, l.mlid, ml.email_to, ml.name, ml.html AS user_html, l.html, l.stat_track, l.opt_out, l.email_headers, l.email_subject, l.email_body, l.email_plain, l.attachments, l.email_from FROM mailout_message_log AS ml JOIN mailout_log AS l ON ml.mlid = l.mlid WHERE sent = 0 AND l.scheduled &lt; 1246895940 in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173

Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'MySQL server has gone away\nquery: LOCK TABLES variable WRITE in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173

Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'MySQL server has gone away\nquery: DELETE FROM variable WHERE name = &#039;cron_last&#039; in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173

Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'MySQL server has gone away\nquery: INSERT INTO variable (name, value) VALUES (&#039;cron_last&#039;, &#039;i:1246895940;&#039;) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173

Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'MySQL server has gone away\nquery: UNLOCK TABLES in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173

Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'MySQL server has gone away\nquery: DELETE FROM cache WHERE cid = &#039;variables&#039; in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173

Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'MySQL server has gone away\nquery: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, &#039;cron&#039;, &#039;Cron run completed.&#039;, 0, &#039;&#039;, &#039;http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node&#039;, &#039;http://www.mydomain.net/mysite/&#039;, &#039;193.188.105.20&#039;, 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173

Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'MySQL server has gone away\nquery: DELETE FROM variable WHERE name = &#039;cron_semaphore&#039; in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173

Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'MySQL server has gone away\nquery: DELETE FROM cache WHERE cid = &#039;variables&#039; in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173

Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'MySQL server has gone away\nquery: UPDATE sessions SET uid = 1, cache = 0, hostname = &#039;193.188.105.20&#039;, session = &#039;current_locale|s:2:\\&quot;en\\&quot;;watchdog_overview_filter|s:3:\\&quot;all\\&quot;;form|a:6:{s:32:\\&quot;eda39568fd2695910891bf50d571111a\\&quot;;a:2:{s:9:\\&quot;timestamp\\&quot;;i:1246815151;s:4:\\&quot;args\\&quot;;a:2:{i:0;s:17:\\&quot;mailout_send_form\\&quot;;i:1;a:6:{s:5:\\&quot;state\\&quot;;a:1:{s:4:\\&quot;mtid\\&quot;;s:1:\\&quot;0\\&quot;;}s:8:\\&quot;template\\&quot;;a:1:{s:4:\\&quot;mtid\\&quot;;s:1:\\&quot;0\\&quot;;}s:2:\\&quot;op\\&quot;;s:4:\\&quot;N in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173

Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'MySQL server has gone away\nquery: UPDATE users SET access = 1246895940 WHERE uid = 1 in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173

Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'Cannot modify header information - headers already sent by (output started at /home/mycpid/public_html/mysite/includes/database.mysql.inc:173) in /home/mycpid/public_html/mysite/includes/common.inc on line 314.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173
--------------------------------------

List of SELECT * FROM:

In file mailout_send.in & mailout_admin.inc:

mailout_admin.inc:671: $not_subscribed = db_query("SELECT * FROM {mailout_list} ml WHERE ml.id NOT IN (". implode(', ', $sub_in) .')');
mailout_admin.inc:912: $all_emails = db_query('SELECT * FROM {mailout_message_log} WHERE mlid = %d ORDER BY email_to', $item->mlid);

mailout_admin.inc:920: $sql = 'SELECT * FROM {mailout_message_log} WHERE mlid = %d ';
mailout_admin.inc:1217: $template = db_fetch_object(db_query('SELECT * FROM {mailout_template} WHERE mtid = %d', $mtid));

mailout_send.inc:54: $res=db_query("SELECT * FROM {address_list} WHERE category='%s'",$catvalue);
mailout_send.inc:80: $res=db_query("SELECT * FROM {address_list} WHERE category='%s'",$catvalue);

mailout_send.inc:572: $res=db_query("SELECT * FROM {address_list} WHERE category='%s'",$catvalue);
- - - -
in file: mailout.module:

333: $result = db_query("SELECT * FROM {mailout_list_addresses} WHERE id = %d", $sub_id);

336: $result = db_query("SELECT * FROM {mailout_list_addresses} WHERE list_id <> %d AND email = '%s' AND list_id = %d", MAILOUT_GLOBAL_OPTOUT, $email, $list_id);
339: $result = db_query("SELECT * FROM {mailout_list_addresses} WHERE list_id <> %d AND email = '%s'", MAILOUT_GLOBAL_OPTOUT, $email);

342: $result = db_query("SELECT * FROM {mailout_list_addresses} WHERE list_id <> %d AND list_id = %d", MAILOUT_GLOBAL_OPTOUT, $list_id);
543: $result = db_query('SELECT * FROM {mailout_template} ORDER BY subject');

#1

tcibah - July 8, 2009 - 15:50

Here are some additional info from the hosting company:

Here are the complete queries that were killed off. The fourth column is the time in seconds that they ran before being killed off.

[ 1246442407 ] [ mycpid ] [ 1756727 ] [ 19 ] [ ] [ Query ] [ KILLED ] [ 'myserver.website.com' ] [ SELECT *,
`TABLE_SCHEMA` AS `Db`,
`TABLE_NAME` ]
[ 1246442407 ] [ mycpid ] [ 1756729 ] [ 19 ] [ mycpid_site ] [ Query ] [ KILLED ] [ 'myserver.website.com' ] [ SELECT *,
`TABLE_SCHEMA` AS `Db`,
`TABLE_NAME` ]
[ 1246703582 ] [ mycpid ] [ 2703021 ] [ 24 ] [ mycpid_site ] [ Query ] [ KILLED ] [ 'myserver.website.com' ] [ SELECT *,
`TABLE_SCHEMA` AS `Db`,
`TABLE_NAME` ]
[ 1246706642 ] [ mycpid ] [ 2715851 ] [ 18 ] [ mycpid_site ] [ Query ] [ KILLED ] [ 'myserver.website.com' ] [ SELECT *,
`TABLE_SCHEMA` AS `Db`,
`TABLE_NAME` ]
[ 1246791605 ] [ mycpid ] [ 3023475 ] [ 18 ] [ mycpid_site ] [ Query ] [ KILLED ] [ 'myserver.website.com' ] [ SELECT *,
`TABLE_SCHEMA` AS `Db`,
`TABLE_NAME` ]
[ 1246814102 ] [ mycpid ] [ 3110971 ] [ 17 ] [ mycpid_site ] [ Query ] [ KILLED ] [ 'myserver.website.com' ] [ SELECT *,
`TABLE_SCHEMA` AS `Db`,
`TABLE_NAME` ]
[ 1246872061 ] [ mycpid ] [ 3444906 ] [ 88 ] [ mycpid_site ] [ Query ] [ KILLED ] [ 'myserver.website.com' ] [ SELECT *,
`TABLE_SCHEMA` AS `Db`,
`TABLE_NAME` ]
[ 1246895103 ] [ mycpid ] [ 3564170 ] [ 32 ] [ mycpid_site ] [ Query ] [ KILLED ] [ 'myserver.website.com' ] [ SELECT *,
`TABLE_SCHEMA` AS `Db`,

 
 

Drupal is a registered trademark of Dries Buytaert.