Disable duplicate links during regeneration

You don't say if you uninstalled the 6.x-1 version before to install the other one. Did you?

I am changing the title to make it clear what is the cause of the problem.

Yep I've already been looking into how to solve the duplicate loc problem.

I have the same issue, subscribing

A few days ago I was getting a similar error in my logs:

Duplicate entry 'node/6439' for key 'loc' query: INSERT INTO xmlsitemap (type, id, status, loc, lastmod, priority, changefreq, changecount) VALUES ('node', 6439, 1, 'node/6439', 1240083382, 0, 0, 0) in ..../public_html/cms/sites/all/modules/xmlsitemap/xmlsitemap.module on line 790

I un-installed, then re-installed all the 6.x-2.x-dev modules and the error messages went away.

Please ignore #6

I realise I have been using XML Sitemap with the menu module disabled.

When I enable the menu module and carry out a rebuild I too get the error message:

Duplicate entry 'node/922' for key 'loc' query: INSERT INTO xmlsitemap (type, id, status, loc, lastmod, priority, changefreq, changecount) VALUES ('menu', 179, 0, 'node/922', 0, NULL, 0, 0) in ..../cms/sites/all/modules/xmlsitemap/xmlsitemap.module on line 804.

I get the same errors whenever XML sitemap menu and XML sitemap node are enabled. When I disable them, it seems fine. It looks like it is struggling to index pages that have URL Aliases. Perhaps I don't understand what the point of those modules is. Could someone enlighten me? I'll try reading a spot more, but it looks like the module does it's job just fine with those disabled.

@selwynpolit: The base xmlsitemap module provides all the common functions and displays all the sitemap link data, but all the sub modules are the ones that actually add things like nodes, menus, users, etc as sitemap links. If you disable them, no new items will be added into your sitemap.

Enabling just xmlsitemap.module you will not get a sitemap that is automatically updated with the links of new nodes, or users, or when the old content is updated.
I don't think that is what anybody wants. If somebody would want it, then he could create a static sitemap.xml file, and use it; there is not need to use XML sitemap if then the content it produces is static because the modules that effectively add links to the sitemap are disabled.

Subscribing...

Is this bug report still valid?

Technically yes, although I removed the unique index on {xmlsitemap}.loc to get rid of the glaring errors for now. Marking as postponed until I can figure out a better solution.

subscribe, cant rebuild the xmlsitemap_node.

Duplicate entry '0-node' for key 1 query: INSERT INTO xmlsitemap (type, loc, status, lastmod, priority, priority_override, changefreq, changecount, node_type) VALUES ('node', 'node/', 0, 1246146665, NULL, NULL, 0, -1, NULL) in

@mark_r: That's really odd. Are you using the absolute latest 6.x-2.x-dev build? If not, make sure to re-download and run update.php.

I've updated the module, updated the database. Still the same error. :(

print_r($values);

Array
(
    [0] => node
    [1] => node/
    [2] => 0
    [3] => 1246541885
    [4] => NULL
    [5] => NULL
    [6] => 0
    [7] => -1
    [8] => NULL
)
Array
(
    [0] => node
    [1] => node/
    [2] => 0
    [3] => 1246541885
    [4] => NULL
    [5] => NULL
    [6] => 0
    [7] => -1
    [8] => NULL
)
Array .... 

over 16 times

@mark_r: Can you add these lines of code to the beginning of the xmlsitemap_save_link() function in xmlsitemap.module:

  if (empty($link['id']) && $link['type'] == 'node') {
    drupal_set_message(var_export(debug_backtrace(), TRUE), 'error', TRUE);
  }

Let me know the new error messages that appear.

here are the results:

array ( 0 => array ( 'file' => '/var/www/vhosts/xxxxx/httpdocs/sites/all/modules/xmlsitemap/xmlsitemap_node/xmlsitemap_node.module', 'line' => 15, 'function' => 'xmlsitemap_save_link', 'args' => array ( 0 => array ( 'type' => 'node', 'id' => NULL, 'loc' => 'node/', 'priority_override' => NULL, 'lastmod' => 1246542120, 'changefreq' => 0, 'changecount' => -1, 'node_type' => NULL, 'status' => 0, 'priority' => 'default', 'language' => '', ), ), ), 1 => array ( 'function' => 'xmlsitemap_node_cron', 'args' => array ( ), ), 2 => array ( 'file' => '/var/www/vhosts/xxxxxx/httpdocs/includes/module.inc', 'line' => 471, 'function' => 'call_user_func_array', 'args' => array ( 0 => 'xmlsitemap_node_cron', 1 => array ( ), ), ), 3 => array ( 'file' => '/var/www/vhosts/xxxxx/httpdocs/includes/common.inc', 'line' => 2647, 'function' => 'module_invoke_all', 'args' => array ( 0 => 'cron', ), ), 4 => array ( 'file' => '/var/www/vhosts/xxxxx/httpdocs/modules/system/system.admin.inc', 'line' => 1715, 'function' => 'drupal_cron_run', 'args' => array ( ), ), 5 => array ( 'function' => 'system_run_cron', 'args' => array ( ), ), 6 => array ( 'file' => '/var/www/vhosts/xxxx/httpdocs/includes/menu.inc', 'line' => 348, 'function' => 'call_user_func_array', 'args' => array ( 0 => 'system_run_cron', 1 => array ( ), ), ), 7 => array ( 'file' => '/var/www/vhosts/xxxxx/httpdocs/index.php', 'line' => 19, 'function' => 'menu_execute_active_handler', 'args' => array ( ), ), )

again 15 times

@mark_r: Thanks for your patience in getting this debugged.

So that's really odd. The node_load function is possibly returning invalid data. Let's remove the change in xmlsitemap_save_link(), and modify xmlsitemap_node_cron() in xmlsitemap_node.module to be this:

function xmlsitemap_node_cron() {
  // Start with the most recently changed nodes first.
  $query = db_query_range("SELECT n.nid FROM {node} n LEFT JOIN {xmlsitemap} x ON x.type = 'node' AND n.nid = x.id WHERE (x.id IS NULL OR x.status IS NULL) ORDER BY n.changed DESC", 0, xmlsitemap_var('batch_limit'));
  while ($nid = db_result($query)) {
    drupal_set_message("Node nid: $nid");
    $node = node_load($nid, NULL, TRUE);
    drupal_set_message("Node: " . var_export($node, TRUE));
    $link = xmlsitemap_node_create_link($node);
    drupal_set_message("Node: " . var_export($node, TRUE));
    xmlsitemap_save_link($link);
  }
}

Run cron and show me the first couple results back here.

debug code removed...
no xmlsitemap problem ...

After some debugging chat with mark_r, we discovered that the referenced nodes had missing records in the node_revisions table, which causes the node_load() function to return FALSE since there is an INNER JOIN on node and node_revisions tables. The only solution was to delete the corrupted data from the node table.

Adding as a beta blocker and activating for beta release.

Moving to feature request as this is not a bug and the module is working as expected currently.

As mentioned above, am seeing duplicate links where content from multiple submodules gets output for each occurrence, e.g. if a node is added to a menu it'll show twice.

Until I figure out a better solution that reports back to the user if they have duplicate links, I've committed a small modification (with a test yay!) to the sitemap generation code to skip a link if it has the same path as the previous link. I'm still going to leave this active, but remove the beta blocker tag.

http://drupal.org/cvs?commit=304600

I haven't come up with any better solutions, especially now that we have contexts. Marking as fixed since it works good enough (tm) for now.