First let me say that I think the 1.6 version of this module was an awesome development.

I have a drupal site with 16k+ nodes. I upgraded to the 2.x-dev version as the 1.6 stable version was running out of memory when generating the sitemap and this was supposedly fixed in the latest version. I removed all the xmlsitemap tables then enabled the xmlsitemap and xmlsitemap_node modules.

The xmlsitemap_node table was completely empty.

By visiting my site's sitemap sitename/sitemap.xml I assumed then caused the system to start the process of trying to generate the necessary row entries for the the empty table. After about 30 seconds though I begin receiving a massive number of errors like those below. In fact these errors started showing up any time there was an insert or update to a page that would probably affect the sitemap. As a result I've had to turn off the xmlsitemap_node module.

I think the module needs to be optimized better. It looks like a query is taking way to long and a php query is timing out somewhere. I never had this problem with the old version. Any pointers appreciated.

Warning: Lost connection to MySQL server during query query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (0, 'php', '<em>Server shutdown in progress\nquery: SELECT n.nid, n.type, n.promote, n.changed, xn.previously_changed, xn.priority_override, s.comment_count, s.last_comment_timestamp, xn.previous_comment, COALESCE(ua.dst) AS alias FROM node n\r\n LEFT JOIN xmlsitemap_node xn ON n.nid = xn.nid\r\n LEFT JOIN node_comment_statistics s ON n.nid = s.nid\r\n LEFT JOIN url_alias ua ON ua.src = CONCAT(&amp;#039;node/&amp;#039;, n.nid)\r\n LEFT JOIN i18n_node i18n ON n.nid = i18n.nid WHERE (i18n.language =&amp;#039;en&amp;#039; OR i18n.language =&amp;#039;&amp;#039; OR i18n.language IS NULL) AND ( n.status &amp;gt; 0\r\n AND n.type NOT IN (&amp;#039;0&amp;#039;)\r\n AND (n.type NOT IN (&amp;#039;0&amp;#039;) AND xn.priority_override IS NULL OR xn.priority_override &amp;gt;= in /home/xxxx/public_html/includes/database.mysql.inc on line 174

Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', '<em>MySQL server has gone away\nquery: DELETE FROM cache_page</em> in <em>/home/bnewsome/public_html/includes/database.mysql.inc</em> on line <em>174</em>.', 2, '', 'http://site/node/20307/edit?destination=admin%2Fcontent%2Fnode', 'http://site/node/20307/edit?destination=admin%2Fcontent%2Fnode', '142.177.59.159', 1233519002) in /home/xxxx/public_html/includes/database.mysql.inc on line 174

Comments

avpaderno’s picture

Status: Active » Postponed (maintainer needs more info)

XML Sitemap doesn't execute the INSERT INTO watchdog query like shown in the error message; actually, it doesn't even call watchdog() passing such arguments.

It seems that the reported error is caused by another module, and by a coincidence the error happened when Drupal was executing a XML Sitemap query.

avpaderno’s picture

Title: xmlsitemap_node Causes Database timeouts? in Drupal 5.x-2.x-dev version » xmlsitemap_node causes database timeouts
yhzsailor’s picture

I'm still trying to diagnose this. I agree that it is likely that xmlsitemap per se doesn't actually cause this specific error message above, its Drupal's error handling routine trying to log to the watchdog database that an error has occured and it cannot do that because the database is tied up.

If you look closely at the message I've quoted, you'll notice that the query is trying to insert the an error message into the watchdog table related to an SQL query that, and it definitely refers to xmlsitemap_node. I believe that, with about 17k rows in the xmlsitemap_node table, this join is taking so long it is timing out my various PHP and MySQL timers. I'm trying to optimize those, but the reality is that I did not have this problem in the 1.6 version of the module with roughly the same number of rows. I've not looked to see if the underlying query/data structures have changed in the new version.

SELECT n.nid, n.type, n.promote, n.changed, xn.previously_changed, xn.priority_override, s.comment_count, s.last_comment_timestamp, xn.previous_comment, COALESCE(ua.dst) AS alias FROM node n\r\n LEFT JOIN xmlsitemap_node xn ON n.nid = xn.nid\r\n LEFT JOIN node_comment_statistics s ON n.nid = s.nid\r\n LEFT JOIN url_alias ua ON ua.src = CONCAT(&amp;#039;node/&amp;#039;, n.nid)\r\n LEFT JOIN i18n_node i18n ON n.nid = i18n.nid WHERE (i18n.language =&amp;#039;en&amp;#039; OR i18n.language =&amp;#039;&amp;#039; OR i18n.language IS NULL) AND ( n.status &amp;gt; 0\r\n AND n.type NOT IN (&amp;#039;0&amp;#039;)\r\n AND (n.type NOT IN (&amp;#039;0&amp;#039;) AND xn.priority_override IS NULL OR xn.priority_override &amp;gt;

avpaderno’s picture

I noticed the query that is being executed, but I also noticed that the second query (leaving out the part starting with INSERT INTO watchdog) it's not a query executed from XML Sitemap, and it's executed in a different time that the first query.

It could be another module causing the problem, and you see the last queries executed before the database engine shuts down.

yhzsailor’s picture

I understand your comment. But lets put it this way. When the xmlsitemap node module is turned on this error occurs. When its not turned on ... absolutely no error.

Anonymous’s picture

Is it possible that you've exceeded the threshold limits of your DB instance? Check your system logs and any logs for the DB.

yhzsailor’s picture

Earnie - this is exactly the problem, and I'm working to figure out what thresholds I can change within the limits of my hosting plan. I point out that another way to resolve this is to optimize the query which is my original complaint.

Anonymous’s picture

What do you have the chunk size set to? What are the database limits imposed? What are the PHP execution limits imposed?

sergmain’s picture

This bug related not to xmlsitemap but to weak design of url_alias table.
The workaround for this bug is change a query variable in function xmlsitemap_node_xmlsitemap_links() to:

  $query = 
    "select a.*, $coalesce(ua.dst) AS alias 
    from (
    SELECT $columns FROM {node} n
    LEFT JOIN {xmlsitemap_node} xn ON n.nid = xn.nid
    $left_join
    WHERE n.status > 0
      AND n.type NOT IN (". db_placeholders($excludes, 'varchar') .")
      AND (xn.priority_override = -2 OR xn.priority_override >= 0)
      AND n.nid <> %d
    GROUP BY $columns
    )a, {url_alias} ua
    WHERE ua.src = CONCAT( 'node/', a.nid )";

this fix "MySQL server has gone away query" on my site with 7k aliases

id 	select_type 	table 	type 	possible_keys 	key 	key_len 	ref 	rows 	Extra
1 	PRIMARY 	 	ALL 	NULL 	NULL 	NULL 	NULL 	6549 	 
1 	PRIMARY 	ua 	ALL 	src 	NULL 	NULL 	NULL 	6548 	Range checked for each record (index map: 0x4)
2 	DERIVED 	xn 	ALL 	PRIMARY 	NULL 	NULL 	NULL 	6549 	Using where; Using temporary; Using filesort
2 	DERIVED 	n 	eq_ref 	PRIMARY,node_status_type,node_type 	PRIMARY 	4 	xxxx.xn.nid 	1 	Using where
2 	DERIVED 	s 	eq_ref 	PRIMARY 	PRIMARY 	4 	xxxx.n.nid 	1 	 

old explain is:

1  	SIMPLE  	xn  	ALL  	PRIMARY  	NULL  	NULL  	NULL  	6549  	Using where; Using temporary; Using filesort
1 	SIMPLE 	n 	eq_ref 	PRIMARY,node_status_type,node_type 	PRIMARY 	4 	xxxx.xn.nid 	1 	Using where
1 	SIMPLE 	s 	eq_ref 	PRIMARY 	PRIMARY 	4 	xxxx.n.nid 	1 	 
1 	SIMPLE 	ua 	ALL 	src 	NULL 	NULL 	NULL 	6548 	 

origin sample sql:
EXPLAIN SELECT n.nid, n.type, n.promote, xn.changed, xn.previously_changed, xn.priority_override, s.comment_count, s.last_comment_timestamp, xn.previous_comment, COALESCE(ua.dst) AS alias

FROM node n
LEFT JOIN xmlsitemap_node xn ON n.nid = xn.nid
LEFT JOIN node_comment_statistics s ON n.nid = s.nid
LEFT JOIN url_alias ua ON ua.src = CONCAT('node/', n.nid)

WHERE n.status > 0 AND n.type NOT IN (". .") AND (xn.priority_override = -2 OR xn.priority_override >= 0) AND
n.nid <> 0
GROUP BY n.nid, n.type, n.promote, xn.changed, xn.previously_changed, xn.priority_override, s.comment_count, s.last_comment_timestamp, xn.previous_comment

patched sample sql:
EXPLAIN SELECT a . * , COALESCE( ua.dst ) AS alias
FROM (

SELECT n.nid, n.type, n.promote, xn.changed, xn.previously_changed, xn.priority_override, s.comment_count, s.last_comment_timestamp, xn.previous_comment
FROM node n
LEFT JOIN xmlsitemap_node xn ON n.nid = xn.nid
LEFT JOIN node_comment_statistics s ON n.nid = s.nid
WHERE n.status >0
AND n.type NOT
IN (
". ."
)
AND (
xn.priority_override = -2
OR xn.priority_override >=0
)
AND n.nid <>0
GROUP BY n.nid, n.type, n.promote, xn.changed, xn.previously_changed, xn.priority_override, s.comment_count, s.last_comment_timestamp, xn.previous_comment
)a, url_alias ua
WHERE ua.src = CONCAT( 'node/', a.nid )

avpaderno’s picture

Status: Postponed (maintainer needs more info) » Active
Anonymous’s picture

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.