Optimize a SELECT * FROM query that is being killed in a shared hosting server due to execution time exceeds 15 seconds
| Project: | Mailout |
| Version: | 5.x-1.x-dev |
| Component: | Code |
| Category: | task |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | active |
Jump to:
The error happens only during cron runs.
The error suddenly happened when tables mailout_message_log & mailout_log reach 9,569 records & 307 records respectively. If those tables are truncated, there will be no errors.
What statement causes the error? Any suggestion to optimize it?
The information above is a summar of the following:
Annex 1: Hosting Provier Response
I see the mysql connections being killed for the script as it is running for too long.
[ 1246442407 ] [ mycpid ] [ 1756729 ] [ 19 ] [ mycpid_site] [ Query ] [ KILLED ] [ 'myserver.website.com' ] [ SELECT *,
[ 1246703581 ] [ mycpid ] [ 2703020 ] [ 25 ] [ ] [ Query ] [ KILLED ] [ ] [ SELECT *,
[ 1246703582 ] [ mycpid ] [ 2703021 ] [ 24 ] [ mycpid_site] [ Query ] [ KILLED ] [ 'myserver.website.com' ] [ SELECT *,
[ 1246706642 ] [ mycpid ] [ 2715851 ] [ 18 ] [ mycpid_site] [ Query ] [ KILLED ] [ 'myserver.website.com' ] [ SELECT *,
------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------
Annex 2: The error message thatwe get when the cron is run manually:
Warning: Lost connection to MySQL server during query query: SELECT * FROM system WHERE type = 'theme' in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173
Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'mysql_query() [<a href='function.mysql-query'>function.mysql-query</a>]: Unable to save result set in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 155.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173
Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'MySQL server has gone away\nquery: SELECT ml.mmid, l.mlid, ml.email_to, ml.name, ml.html AS user_html, l.html, l.stat_track, l.opt_out, l.email_headers, l.email_subject, l.email_body, l.email_plain, l.attachments, l.email_from FROM mailout_message_log AS ml JOIN mailout_log AS l ON ml.mlid = l.mlid WHERE sent = 0 AND l.scheduled < 1246895940 in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173
Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'MySQL server has gone away\nquery: LOCK TABLES variable WRITE in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173
Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'MySQL server has gone away\nquery: DELETE FROM variable WHERE name = 'cron_last' in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173
Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'MySQL server has gone away\nquery: INSERT INTO variable (name, value) VALUES ('cron_last', 'i:1246895940;') in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173
Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'MySQL server has gone away\nquery: UNLOCK TABLES in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173
Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'MySQL server has gone away\nquery: DELETE FROM cache WHERE cid = 'variables' in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173
Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'MySQL server has gone away\nquery: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'cron', 'Cron run completed.', 0, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173
Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'MySQL server has gone away\nquery: DELETE FROM variable WHERE name = 'cron_semaphore' in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173
Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'MySQL server has gone away\nquery: DELETE FROM cache WHERE cid = 'variables' in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173
Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'MySQL server has gone away\nquery: UPDATE sessions SET uid = 1, cache = 0, hostname = '193.188.105.20', session = 'current_locale|s:2:\\"en\\";watchdog_overview_filter|s:3:\\"all\\";form|a:6:{s:32:\\"eda39568fd2695910891bf50d571111a\\";a:2:{s:9:\\"timestamp\\";i:1246815151;s:4:\\"args\\";a:2:{i:0;s:17:\\"mailout_send_form\\";i:1;a:6:{s:5:\\"state\\";a:1:{s:4:\\"mtid\\";s:1:\\"0\\";}s:8:\\"template\\";a:1:{s:4:\\"mtid\\";s:1:\\"0\\";}s:2:\\"op\\";s:4:\\"N in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173
Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'MySQL server has gone away\nquery: UPDATE users SET access = 1246895940 WHERE uid = 1 in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173
Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', 'Cannot modify header information - headers already sent by (output started at /home/mycpid/public_html/mysite/includes/database.mysql.inc:173) in /home/mycpid/public_html/mysite/includes/common.inc on line 314.', 2, '', 'http://www.mydomain.net/mysite/admin/logs/status/run-cron?destination=node', 'http://www.mydomain.net/mysite/', '193.188.105.20', 1246895940) in /home/mycpid/public_html/mysite/includes/database.mysql.inc on line 173
--------------------------------------
List of SELECT * FROM:
In file mailout_send.in & mailout_admin.inc:
mailout_admin.inc:671: $not_subscribed = db_query("SELECT * FROM {mailout_list} ml WHERE ml.id NOT IN (". implode(', ', $sub_in) .')');
mailout_admin.inc:912: $all_emails = db_query('SELECT * FROM {mailout_message_log} WHERE mlid = %d ORDER BY email_to', $item->mlid);
mailout_admin.inc:920: $sql = 'SELECT * FROM {mailout_message_log} WHERE mlid = %d ';
mailout_admin.inc:1217: $template = db_fetch_object(db_query('SELECT * FROM {mailout_template} WHERE mtid = %d', $mtid));
mailout_send.inc:54: $res=db_query("SELECT * FROM {address_list} WHERE category='%s'",$catvalue);
mailout_send.inc:80: $res=db_query("SELECT * FROM {address_list} WHERE category='%s'",$catvalue);
mailout_send.inc:572: $res=db_query("SELECT * FROM {address_list} WHERE category='%s'",$catvalue);
- - - -
in file: mailout.module:
333: $result = db_query("SELECT * FROM {mailout_list_addresses} WHERE id = %d", $sub_id);
336: $result = db_query("SELECT * FROM {mailout_list_addresses} WHERE list_id <> %d AND email = '%s' AND list_id = %d", MAILOUT_GLOBAL_OPTOUT, $email, $list_id);
339: $result = db_query("SELECT * FROM {mailout_list_addresses} WHERE list_id <> %d AND email = '%s'", MAILOUT_GLOBAL_OPTOUT, $email);
342: $result = db_query("SELECT * FROM {mailout_list_addresses} WHERE list_id <> %d AND list_id = %d", MAILOUT_GLOBAL_OPTOUT, $list_id);
543: $result = db_query('SELECT * FROM {mailout_template} ORDER BY subject');

#1
Here are some additional info from the hosting company:
Here are the complete queries that were killed off. The fourth column is the time in seconds that they ran before being killed off.
[ 1246442407 ] [ mycpid ] [ 1756727 ] [ 19 ] [ ] [ Query ] [ KILLED ] [ 'myserver.website.com' ] [ SELECT *,
`TABLE_SCHEMA` AS `Db`,
`TABLE_NAME` ]
[ 1246442407 ] [ mycpid ] [ 1756729 ] [ 19 ] [ mycpid_site ] [ Query ] [ KILLED ] [ 'myserver.website.com' ] [ SELECT *,
`TABLE_SCHEMA` AS `Db`,
`TABLE_NAME` ]
[ 1246703582 ] [ mycpid ] [ 2703021 ] [ 24 ] [ mycpid_site ] [ Query ] [ KILLED ] [ 'myserver.website.com' ] [ SELECT *,
`TABLE_SCHEMA` AS `Db`,
`TABLE_NAME` ]
[ 1246706642 ] [ mycpid ] [ 2715851 ] [ 18 ] [ mycpid_site ] [ Query ] [ KILLED ] [ 'myserver.website.com' ] [ SELECT *,
`TABLE_SCHEMA` AS `Db`,
`TABLE_NAME` ]
[ 1246791605 ] [ mycpid ] [ 3023475 ] [ 18 ] [ mycpid_site ] [ Query ] [ KILLED ] [ 'myserver.website.com' ] [ SELECT *,
`TABLE_SCHEMA` AS `Db`,
`TABLE_NAME` ]
[ 1246814102 ] [ mycpid ] [ 3110971 ] [ 17 ] [ mycpid_site ] [ Query ] [ KILLED ] [ 'myserver.website.com' ] [ SELECT *,
`TABLE_SCHEMA` AS `Db`,
`TABLE_NAME` ]
[ 1246872061 ] [ mycpid ] [ 3444906 ] [ 88 ] [ mycpid_site ] [ Query ] [ KILLED ] [ 'myserver.website.com' ] [ SELECT *,
`TABLE_SCHEMA` AS `Db`,
`TABLE_NAME` ]
[ 1246895103 ] [ mycpid ] [ 3564170 ] [ 32 ] [ mycpid_site ] [ Query ] [ KILLED ] [ 'myserver.website.com' ] [ SELECT *,
`TABLE_SCHEMA` AS `Db`,