Duplicate entry in menu_router: Different symptoms, same issue

Sorry, I said "after uninstall VIEUWS module, everything is ok. It's false. I have always lot of errors when I clic on list modul or update available.

For example:

user warning: Duplicate entry 'admin/content/node-type/forum/groups/%/edit' for key 1 query: INSERT INTO menu_router (path, load_functions, to_arg_functions, access_callback, access_arguments, page_callback, page_arguments, fit, number_parts, tab_parent, tab_root, title, title_callback, title_arguments, type, block_callback, description, position, weight, file) VALUES ('admin/content/node-type/forum/groups/%/edit', 'a:1:{i:5;N;}', '', 'user_access', 'a:1:{i:0;s:24:\"administer content types\";}', 'drupal_get_form', 'a:4:{i:0;s:26:\"fieldgroup_edit_group_form\";i:1;a:19:{s:4:\"name\";s:19:\"Sujet de discussion\";s:6:\"module\";s:5:\"forum\";s:11:\"description\";s:114:\"Un sujet de forum est la contribution initiale d\'un nouveau fil de discussion à l\'intérieur d\'un forum.\";s:11:\"title_label\";s:5:\"Sujet\";s:4:\"type\";s:5:\"forum\";s:9:\"has_title\";b:1;s:8:\"has_body\";b:1;s:10:\"body_label\";s:5:\"Corps\";s:4:\"help\";s:0:\"\";s:14:\"min_word_count\";i:0;s:6:\"custom\";b:0;s:8:\"modified\";b:0;s:6:\"locked\";b:1;s:9:\"orig_type\";s:5:\"forum\";s:6:\"is_new\";b:1;s:7:\"url_str\";s:5:\"forum\";s:6:\"fields\";a:0:{}s:6:\"tables\";a:0:{}s:5:\"extra\";a:4:{s:5:\"title\";a:2:{s:5:\"label\";s:5:\"Sujet\";s:6:\"weight\";i:-5;}s:10:\"body_field\";a:3:{s:5:\"label\";s:5:\"Corps\";s:6:\"weight\";i:0;s:4:\"view\";s:4:\"body\";}s:8:\"taxonomy\";a:2:{s:5:\"label\";s:9:\"Taxonomie\";s:6:\"weight\";i:-3;}s:11:\"attachments\";a:3:{s:5:\"label\";s:18:\"Fichiers attachés\";s:6:\"weight\";i:30;s:4:\"view\";s:5:\"files\";}}}i:2;i:5;i:3;s:4:\"edit\";}', 125, 7, '', 'admin/content/node-type/forum/groups/%/edit', 'Edit group', 't', '', 4, '', '', '', 0, '') in /home/myweb/www/myweb/includes/menu.inc on line 2353.

In the menu.inc file (chmod: 644 - I tried 775), there's :

$item['type'], $item['block callback'], $item['description'], $item['position'], $item['weight'], $item['include file']);

Driss

Moving this to the Views issue queue. You're going to need to provide more information on what errors you're getting for this to be solved/helpful.

Hey, don't try to pass this one off on me. The error being shown isn't even remotely related to Views. And what's a .odt file anyway? Doesn't look like anything I can read.

Sorry Earl ;) ODT is some openoffice garbage.

Looks like a support request to me at this point.

I have the same errors (one by path entry !) :

Title changed ...

Duplicate : #248739: Duplicate entry errors - lots of them -> #238760: menu_router table truncated and site goes down

I am having this same issue with the site I am working on. Views is not being used on my site yet so it has nothing to do with it. The error occurs when I submit changes in content types, and sometimes when visiting random administration pages. This is the error below:

# user warning: Duplicate entry 'admin/reports/search' for key 1 query: INSERT INTO menu_router (path, load_functions, to_arg_functions, access_callback, access_arguments, page_callback, page_arguments, fit, number_parts, tab_parent, tab_root, title, title_callback, title_arguments, type, block_callback, description, position, weight, file) VALUES ('admin/reports/search', '', '', 'user_access', 'a:1:{i:0;s:19:\"access site reports\";}', 'dblog_top', 'a:1:{i:0;s:6:\"search\";}', 7, 3, '', 'admin/reports/search', 'Top search phrases', 't', '', 6, '', 'View most popular search phrases.', '', 0, 'modules/dblog/dblog.admin.inc') in C:\Documents and Settings\Aaron Hawkins\My Documents\websites\WaitingForTheStorm\includes\menu.inc on line 2353.
# user warning: Duplicate entry 'admin/settings/admin' for key 1 query: INSERT INTO menu_router (path, load_functions, to_arg_functions, access_callback, access_arguments, page_callback, page_arguments, fit, number_parts, tab_parent, tab_root, title, title_callback, title_arguments, type, block_callback, description, position, weight, file) VALUES ('admin/settings/admin', '', '', 'user_access', 'a:1:{i:0;s:29:\"administer site configuration\";}', 'drupal_get_form', 'a:1:{i:0;s:27:\"system_admin_theme_settings\";}', 7, 3, '', 'admin/settings/admin', 'Administration theme', 't', '', 6, 'system_admin_theme_settings', 'Settings for how your administrative pages should look.', 'left', 0, 'modules/system/system.admin.inc') in C:\Documents and Settings\Aaron Hawkins\My Documents\websites\WaitingForTheStorm\includes\menu.inc on line 2353.
# user warning: Duplicate entry 'search/node/%' for key 1 query: INSERT INTO menu_router (path, load_functions, to_arg_functions, access_callback, access_arguments, page_callback, page_arguments, fit, number_parts, tab_parent, tab_root, title, title_callback, title_arguments, type, block_callback, description, position, weight, file) VALUES ('search/node/%', 'a:1:{i:2;N;}', 'a:1:{i:2;s:16:\"menu_tail_to_arg\";}', '_search_menu', 'a:1:{i:0;s:4:\"node\";}', 'search_view', 'a:1:{i:0;s:4:\"node\";}', 6, 3, 'search', 'search', '', 'module_invoke', 'a:4:{i:0;s:4:\"node\";i:1;s:6:\"search\";i:2;s:4:\"name\";i:3;b:1;}', 128, '', '', '', 0, 'modules/search/search.pages.inc') in C:\Documents and Settings\Aaron Hawkins\My Documents\websites\WaitingForTheStorm\includes\menu.inc on line 2353.

Every few times I load my site, I get close to 301 error messages that state the following:

The only way I've found to fix the problem is to rebuild the menu's via the devel menu, but that is only temporary since that just clears the menu_router table.

The menu's load every time, I just end up with these huge error messages.

I'm running Drupal 6.x-dev, but the same thing happened with Drupal 6.2. Apache 2.0.61, PHP Version 5.2.6.

Attached is the complete error log the shows each time.

I don't think this is a duplicate as it seems to be different from #238760: menu_router table truncated and site goes down

I am a complete newbie to Drupal and I have experienced this exact problem. Upon loading the admin page (after what seems like several minutes) I get over 200 errors all having to do with user warning: duplicate entries -- every single one related to inserting values into the menu_router table.

After reading this thread, I went ahead and disabled and removed the Views module (I had installed Views 6.2.beta), truncated the menu_router table and then rebuilt it using a snippet of code I found, the problem went away.

Besides hoping to understand the error, my question is, why would loading the admin dashboard trigger all of these insertions in the menu_router table in the first place?? What is the trigger for that?

It does not seem to be locking the table when doing this. It seems to be a race condition. I did a little test inside of the menu.inc file

Line Number:2274

When the above code ran via me opening a whole bunch of tabs of the site, clicking various links the results were strange...

before delete 89 after delete 2
before delete 13 after delete 2

So to me it looks like while one load is creating the menu another load is tearing it down and rebuilding it.

This is just my observation, I could be wrong.

Until this bug is fixed you can use this small hack, that help me out. But you must use MySQL since other dbms doesn´t support "REPLACE INTO".

Drupal 6.3
Replace line 2356 of includes/menu.inc in the function _menu_router_build() {
    db_query("INSERT INTO {menu_router}
with
    db_query("REPLACE INTO {menu_router}

Dirty, but it works!

#15 works great! dirty indeed but it works...

so what's causing it?

Is it a race condition? I get it more when I insert menu_rebuild() into a module that I'm developing, to make updating the menu quicker.

Perhaps menu_rebuild() is getting called at roughly the same time for two different users, so two processes are trying to insert the new menu data into the table at about the same time, leading to duplicates?

I'm not using views either.

This is a duplicate of #238760: menu_router table truncated and site goes down. A fix was already released in 6.3 that should mitigate the issue.

This still can happen in 6.3 too, if menu_rebuild() is called by two site users at the same time.

I suspect it happens when two different users cause the menu_router table to be re-built at the same time, and it's a race condition caused by the lack of table locking. It appeared on my site much more often when I was calling menu_rebuild() on every page load (for lazy development reasons!).

I think what is happening is:

User 1: DELETE FROM {menu_router}
User 1: Do some INSERT INTO {menu_router} items (1,2,3)
User 2: DELETE FROM {menu_router}
User 1: Do some more INSERT INTO {menu_router} items (4,5,6)
User 2: Do some INSERT INTO {menu_router} items (1,2,3)
User 2: Do some more INSERT INTO {menu_router} items (4,5,6) => ERROR duplicate rows!!!

Replacing "INSERT" with "REPLACE" thus works around the problem, but if it is a race condition the menu routing might just end up slightly wrong depending on the order of calls to the query on line 2356.

To make this more likely, add menu_rebuild() into a module's .module file, so it gets called a lot. Then get several users to access the site at the same time.

Not sure about the duplicate link, as I _think_ these are different problems, albeit with the same section of code:

#238760: bug caused by code exiting while database state is invalid (between deleting old menu data and inserting new), so site's menu disappears.
#246653: bug caused by normal site use, when two processes try to rebuild the menu, site menu remains, but users see many error messages.

I think the fix for #238760 requires transactions, or equivalent, so menu table is always useable.
I think the fix for #246653 requires table locking, or equivalent, so only one process can update it at a time.

So not quite the same problem?

These are perfect duplicates, please comment on the other issue rather than on this one.

During the discussion in #238760 two things were committed: first, a stop-loss solution in case a module performed badly and the rebuild operation failed (hence the title), second a mitigation for the race condition you are describing.

The fix that was release in 6.3 is only a mitigation. The real issue cannot go away until Drupal has a locking framework, ie. a way to prevent two batch operations (like menu_rebuild) to happen at the same time. Anyway, in 6.3 you should only see that in extreme conditions (such as a broken module that calls menu_rebuild at every page load...).

Leaving this issue to D7 at #238760: menu_router table truncated and site goes down
but treating it as D6 in a separately issue #289618: menu_router table truncated and site goes down: Transaction or Lock required for D6
For the whole discussion, please refer to #238760: menu_router table truncated and site goes down
To know why this issue is in fact a duplicate of #238760 refer to #289618: menu_router table truncated and site goes down: Transaction or Lock required for D6

I don't think this is duplicate - this is a totally different (in fact opposite) problem from #238760

Yes it is the same issue.
Lets see
if you rebuild the menu with devel module and the problem goes off (if you ever get with this issue it will be appearing over and over almost for sure) then we are talking about the same issue.
Different symptoms, same issue
Once the menu fail to be rebuilt different symptoms might come up.
1- site goes down / administrative task are unavailable / some URLs are unavailable
2- site keeps complaining about duplicated keys in menu_router table

The cause is a timeout or whatever might interrupt a non-Transactional manipulation of the data (when I run manual test looking for robustness I suddenly shutdown the MySQL server), this is (IMO) a critical flaw of Drupal 6 (don't know what about 5 but I think it has some sort of table locking)
1- If the menu doesn't get rebuilt then the menu_router table might be almost empty causing the unavailability of the site
2- If the menu get partial rebuilt but isn't flagged as totally rebuilt the site might be usable and in further request it will attempt to rebuild it, but as it is partially built then it will complain about duplicated keys.

subscribing

I don't think this is duplicate - this is a totally different (in fact opposite) problem from #238760

Please, this issue was originally posted for 6.x
if this isn't a duplicated, then find, mark it as such, but don't change it to next generation of Drupal, the issue is present in 6.x and as such must remain until marked as fixed, won't fix, or whatever. Once the issue is marked maybe as "won't fix" then incoming issues might be marked targeting 7.x, but for now, this issue is present in 6.x, it's a major issue and as such deserves to be answered (in the worse case as "won't fix")
I won't mark it as duplicated again because you don't agree, but I'm returning it to the 6.x where it belongs to.

See what happened on #238760: menu_router table truncated and site goes down jumping to 6.x and to 7.x over and over, it's not traceable.
Please read #289618: menu_router table truncated and site goes down: Transaction or Lock required for D6 again and try to agree with me why this issue must be kept on both 6.x and 7.x but as different issues (for tracking purpose) and maybe both referencing a unique discussion until they split up (sooner or later one of them will be marked as fixed or won't fix while the other one will not)

@arhak - the process is that bugs are fixed in the newest version before backporting. The menu code in 7.x is very much the same as 6.x, so whatever bug is present should be addresed in 7.x first.

What I'm having a difficult time understanding is that there is a query to delete all entries from {menu_router} before any inserts are attempted. So, is that deletion query failing for some reason?

the process is that bugs are fixed in the newest version before backporting. The menu code in 7.x is very much the same as 6.x, so whatever bug is present should be addresed in 7.x first.

Well, I disagree with pointing every issue to D7 because there are proposed and tested patches for D6 (follow the references in my previous comment) and then the issue is changed back and forward every time someone think has the solution for D6 and later someone change it back to D7 because the patch is rather a improvement but not a solution.

What I'm having a difficult time understanding is that there is a query to delete all entries from {menu_router} before any inserts are attempted. So, is that deletion query failing for some reason?

The deletion always succeed. The insertion is what fails because of timeout. Now, if the menu is half built and some change (module enabling/disabling, accessing modules pages I think causes it also, and other situations) triggers the rebuild menu task, then there will be duplicate entries, of course. Now I don't remember when I looked into the code, but for some reason, rebuilding the menu with devel most of the time succeeds, deleting the whole menu_router table and recreating it again, but in other situations a rebuild is attempted without deleting the table first.
There are proposed patches that improve this awful situation in several ways without providing a definitive solution:
- the first one reorders the code so the computation is done first and the delete is near the insertion (originally the deletion is first, then computation and later an insertion loop, the time window for failing such task is huge)
- the second one remove the deletion attempt and uses ALTER instead of insert (valid only when menu is augmented, wrong otherwise)

please, follow the pointed issues on my previous comment

I have no idea which issue to post this on, but I have suddenly developed this issue. I have been on 6.3 for over a week (new site), using the site frequently. No errors. Today, I upgraded to the most recent beta4 of BOTH Date and Calendar modules. And now, getting LOTS of this error. Not sure if it is related.... Also commenting on the other thread....
Maria

@ahak - the already comitted patches (6.x and 7.x) move the deletion code to close to the insertion code. Your analysis does not make sense - the code should always delete all entries before attempting any insert. If only some subset was inserted, I'd expect you to 404 on the subset of pages corresponding to the missing items.

Hi guys, i had this same problems, i fixed commenting in my module the code>

/*
function artigo_init() {
// menu_rebuild();
}*/

then it work fine!

Doing a menu rebuild in a hook_init() function would (at the least) make the site very slow - why was that code there?

Let's close this one for good. This is a duplicate of #238760: menu_router table truncated and site goes down.

We *know* that a race condition can still happen. This will stay this way. menu_rebuild() is a costly non-transactional function should be called as few as possible, and clearly not on every page load.

If you use 6.6 the hack mentioned in #15 must change line 2370 in includes/menu.inc.

Number 15 also fixes it on D.10.

Sorry, I'm not a Core expert, but....will there be some change in 6.11 to account for this, or should I be managing my DBs to lock tables in some way, or...how do folks recommend avoiding this? It's a pretty ugly error, and lots of folks are experiencing it:

http://drupal.org/node/352978
http://drupal.org/node/333428
http://drupal.org/node/248739
http://drupal.org/node/375327 [ which indicates that just refreshing the page fixes the problem - not hacking core ]

subscribing - want to know when this is fixed (backported to D6)

I fail to see how this can be closed. #238760: menu_router table truncated and site goes down is marked as fixed and there are quite a few comments saying it is a different issue. The issue discussed in this thread is not fixed. I upgraded a production site to Drupal 6.10 yesterday and am seeing this in certain situations and it is ugly.

When I save a contact record in CiviCRM 2.2.0 I see this without fail. I think this is because the record can contain user information that might have changed, so the menus might need rebuilding to reflect any updates. There's a call to menu_rebuild() in some groups code that looks to depend on a user account changing but I can't read the code well enough to know for sure...

Saying that menu_rebuild() should not be called so often doesn't actually fix the problem! ;-)

So, can someone who understands what's happening please comment on the following?

• #15, above. Is it a safe workaround? If it is safe then why isn't it good enough to commit for Drupal 6.11 instead of leaving a known race condition in the code? ;-)
• #238760: menu_router table truncated and site goes down #83 seems to suggest that turning off error reporting to the screen is a good workaround. So, I just go to admin/settings/error-reporting and turn off errors to the screen? It's that easy? And recommended for a production site? I've read the comment on the error-reporting page but I'm wondering what people do on real systems... :-)

I just want to make an informed decision.

Thanks...

peace & happiness,
martin

* #15, above. Is it a safe workaround? If it is safe then why isn't it good enough to commit for Drupal 6.11 instead of leaving a known race condition in the code? ;-)

REPLACE is MySQL-specific. Plus, this basically only hide the error, it doesn't change the fact that two menu_rebuild() are running at the same time. The true solution is to implement a soft-locking framework, and that's being worked out in #251792: Implement a locking framework for long operations.

* #238760: menu_router table truncated and site goes down #83 seems to suggest that turning off error reporting to the screen is a good workaround. So, I just go to admin/settings/error-reporting and turn off errors to the screen? It's that easy? And recommended for a production site? I've read the comment on the error-reporting page but I'm wondering what people do on real systems... :-)

On real systems, like here on drupal.org, we don't have any problems. The main issue is badly written modules calling menu_rebuild() when they should not ;)

Thanks Damien. I've disabled errors to screen and I'm not seeing any problems.

subscribing...

subscribing

i've got what I believe to be the same problem in my 6.10 production site now. its causing me a great deal of slowdown for admin tasks - so much so that I can rarely load my admin/build/modules page without the script timing out.

I tried the fix in #15 on my site but it didn't work for me.

HELP!

In my case, a similar massive error message appeared when upgrading the views module to views-6.x-2.4.
The fact is that I was editing page content in parallel, in another browser tab, while upgrading the module.

Here's my workaround. Hope it shows some hint:
1) I deactivated completely the views modules
2) The nasty error message disappeared and the admin site load time returned to usual values.
3) I reactivate the views module, and again I was working in parallel editing some content -> OOPS! THE ERROR AGAIN!!!
4) Return to point 1) and 2)
5) This time I reactivate the views module making sure that I was not running any parallel operation in my drupal site.
6) The error message disappeared completely, and the site and views module worked as should once again.

I tried this. It seemed to work for a while, but after I logged back out and back in, the errors reappeared. I'm running 6.10 and the latest modules.

#238760: menu_router table truncated and site goes down is closed but doesn't fix this. I know some people *knows* about this race condition but there is no intention to fix it.

We have hit this bug sometimes and people are seen this on our sites when trying to delete 2 nodes with menu entries at concurrent page request (ie: 2 firefox tabs).

This patch seems to solve it for me and maybe for some of you too. Is there any downside in fixing it this way?

Code cleanup.

I applied the patch in #47 to a site that is seeing this issue.

It has not been there for a long enough time to be sure that it is gone forever, but will report back when I have something one way or the other.

Drupal 6 doesn't require the LOCK permission in MySQL anymore. Plus we have a db_lock_table() function that need to be used for that.

The true solution for this issue is #251792: Implement a locking framework for long operations, as I already stated above. Marking as a duplicate of this one. You will help everyone by reviewing the patch there.

The locking framework is a long term solution for Drupal 7.x.

It is the proper solution.

For the meantime, sites on Drupal 6.x that use modules that aggressively rebuild the menus will be around for a while, so it is best to keep this open, since it offers sites that see this issue solved via the above patch.

@kbahey actually the locking framework is intended for drupal 6

I understand why my patch is not the best solution but it can be used as a temporary workaround for people seen this problem often, because the locking framework issue is a year old and seems this bug is more easy to hit than you should expect.

I have a similar bug when updating or submitting new content. I'm testing the fixes from simplest to most complex (simplest being deactivate/uninstall views). I have uninstalled views per the instructions in #44, and things seem to work so far. I will post if anything else arises.

EDIT: #44 does not work. Will try a few of the other fixes for more clarification.

Thanks Drupal community for your help on this matter!

#47 worked for me. Thanks!

The patch in #47 eventually worked for me. It seemed to take a few days for the errors to stop occurring. This after clearing the cache.

same here, will try the patch now

I'm with kbahey in #50. ie. it would be great to have a solution based on db_lock_table(), in the meantime, until the locking framework is finished for D7, and then backported to D6.

subscribe

Just my 2 cents:
I am all in favour of a temporary solution, to fix this issue until the locking framework is finished.

Is there a better temporary solution than the one in #47? I'm not sure from reading #251792: Implement a locking framework for long operations. Also, I posted a script in that issue to rebuild the {menu_router} table in case yours gets trashed, which happens sometimes on my D6 site (presumably b/c the locking isn't good enough).

subscribe

subscribe

I'm having this same problem. I'm fairly new to Drupal, so please forgive my ignorance, but is there a specific method I need to use to apply a patch? I'm using Drupal 6.12. Is the patch in #47 appropriate for me to use?

@renee & tessmonsta, is the patch in #47 still working for you?

The patch works for me. It removes the error message.

As I can see, it does not reduce the queries needed for menu rebuild, or the time needed for each of these queries. (which can be quite long, see http://drupal.org/node/302638)

So far, I did not notice any evil side effects.

@aktary:
I am sure there are better ways, but I usually try to find the place in the code, and do it manually. Then I indicate where I applied the patch, with something like:

This helps to find these places again when I do a software update.

Subscribe

Subscribe

Subscribe

I seem to have beem able to resolve this problem for my site using the patch (as it exists so far for D6) from #251792: Implement a locking framework for long operations.

Note that the patch implements locking for menu_rebuild(), but I had to add an additional lock specifically for menu_router_build() because both Panels and Admin Menu call that function directly and can otherwise trigger the "duplicate entry" collisions.

I tried loading the Modules admin page and some Panels admin pages repeatedly, and couldn't get any errors to happen, whereas before I'd get pages and pages of errors without much effort using any of those modules.

PeteS: which patch from #251792: Implement a locking framework for long operations are you using? Are you using #65?

There's actually one more posted after that, #71, which I believe is the one I used

subscribing.

Proper duplicate link:
#333428: Duplicate entries in menu_router table (6.x branch)

Read more:
http://drupal.org/node/238760#comment-950563
http://drupal.org/node/238760#comment-1379898

subscribe

Any backport for 6.x?

subscribe

subscribe

Subscribing

I'm on Postgres and seeing this issue a lot, especially when editing two views in two different Firefox tabs at a time. It inserts inline into the views ajax interface and makes it really hard to navigate away.

I get this about every 2 or 3 minutes when I'm actively developing the site in admin.

I've solved it myself by putting the menu rebuild into a transaction (using raw BEGIN/COMMIT statements in _menu_router_build()). This works fine for me.

Like others, I see this as a major enough bug to need a fairly rapid fix in 6.x.

I turned off errors to screen just to get this out of the way, but I'm pretty sure this is a widespread and common issue in recent versions of 6.x.

hmm. This completely screws up mysql replication

ps. this can be worked around by adding slave-skip-errors=1062 to your mysl config file on the slave

Sorry for spamming everything..

here's a rewrite proposal for menu_router_build,
Rewrite the function menu_rebuild
(scroll to the end to find the latest patches)

The patch does not yet incorporate the locking framework, but it should be easy to add that later on. Plus, the number and chance for errors caused by concurrent requests will be much reduced even without the locking framework.

I haven't checked the code but looks to me like XML Sitemap Menu is rebuilding the menu every time cron runs. Could this be a problem too?

hotblack's solution works for me!

(http://drupal.org/node/246653#comment-922486)

Subscribing. Been having this problem since setting up multi-site. Slow speeds on both sites, and modules page displays many of these errors quite often

Shouldnt we have a solution if we replace the query of "insert into" by "insert ignore into"? I have not tried that yet but just putting forward my idea... the problem with me is that at time i see the error and at times not.

INSERT IGNORE
or INSERT IF NOT EXISTS.
or INSERT ... ON DUPLICATE KEY UPDATE
or REPLACE INTO
etc

Which of those work in all DB engines?
If we don't find a cross-db solution, maybe it should be abstracted away into drupal?

I used REPLACE INTO and I have not seen the issue arise since. I have been working with three new modules too and have been reloading the modules page a lot because of it. In the past I definitely would have seen it by now.

Nice catch.. Thanks.. I'll look into this further..

subscribing...

a nice drupal quirx, if it gets worse I'll try #15

Cheers!

using Drupal 6.13 on mysql - same problem here - will try #15, although - don't hack core...

subscribing... guess I'll end up using #15 too, as the site goes online tomorrow

I just noticed this duplicate entry problem on one of my sites running D6.13. At the time I was upgrading a few modules to their latest versions and I whenever I upgraded a module (I upgrade modules one at a time) I would see hundreds of these duplicate entry errors in the log.

I noticed that I had devel 6.x-1.16 installed, but it was disabled. I uninstalled the module and then installed devel 6.x-1.18. And since then I have not seen any more duplicate entry errors in the log after that.

I will update if I see them again, but for now it is looking good.

subscribe

If you want to reproduce this error, I found a way to get it consistently. If you have Views installed, edit one of your views. Press the Save button twice. Probably you'll get the error if you press the Save button twice on most content forms.

It's obviously a race condition, so apart from the solutions posted above, there are a couple of others. One is to reject any form that's been submitted more than once. Another would be to implement a module that disables the Save/Submit button on all forms the first time it's pressed, so it can't be pressed again. Unfortunately, the second solution could cause other problems. For instance, when I save views, I sometimes get an AJAX javascript error dialog and have to press Save again.

I was experiencing the same problem with many Duplicate Entry errors every time I would view a page on the site. I also had Devel module 6.x-1.16 enabled so I upgraded to 6.x-1.18 and the errors no longer appeared.

There has been a decent mention here of table locking for slow queries. Could it be beneficial in swapping this table to innoDB, as this supports row-level locking?

Actually in this case table-level locking is what you want. It's deleting all the rows from the table then reinserting them all. The error occurs when two cache-clears occur at the same time, and so there's two parallel processes doing the delete and reinsert. Usually what this means is that the second one will attempt to insert a row that the first has already put back. A full table lock would means that although the second cache-clear would have to wait until the first (making it take quite a bit longer), they wouldn't be working on the same data at the same time.

A row-level lock doesn't help, because all the rows are being purged.

We have this problem even on Postgres, but the solution is to simply take out an exclusive write lock on the menu_router table. It slows things down when you're working with multiple cache-clears (eg. the views UI on multiple tabs), but it stops all the errors, and the possibility of table corruption.

The main issue here is that Drupal 6 dropped the requirement (I believe) for Drupal to have the LOCK TABLE permission, therefore core doesn't rely on db_lock_table() working. Patching core to lock this table would require drupal to have this permission. Personally I think it should *try* to lock the table, but if it fails, just keep going anyway.

It's deleting all the rows from the table then reinserting them all.

Which is not necessary at all.

Well actually it is necessary as part of the menu rebuilding process, but maybe rebuilding the menu is not necessary in a lot of the circumstances where it's done.

Nope, it's not. We could just as well scan the old table contents, see what has changed, and write the changes to the database, as proposed in #512962: Optimize menu_router_build(). It would be even easier in D7 than in D6, as it already happens all in one function. Just I'm not gonna make it myself.

It would still be a good idea to have concurrency control, but it won't be that important as it is with the current bulldozer approach.

Ah yes, in an ideal world where Drupal went for efficiency, yeah that would be a nice option. It would require a fair bit of memory though, as it'd need to do an in-memory compare. I suppose it's not that big though.

I wish this could apply to the node_access rebuilding process too though, it's such a pain to change access control rules and have to rebuild the permissions and have most of your content offline for hours.

Ah yes, in an ideal world where Drupal went for efficiency, yeah that would be a nice option. It would require a fair bit of memory though, as it'd need to do an in-memory compare. I suppose it's not that big though.

That's what a lot of people said in the other issue. But it's not a valid argument: We already have all the information in memory!!
http://api.drupal.org/api/function/_menu_router_save/7
I mean, how do you want to refill a deleted table if you don't have the information in memory?

And, if you look at the other functions involved, you will see that more stuff of the same magnitude is in memory during that rebuild process. And all passed around (and thus copied) as a by-value array arguments.

A diff-based table contents replace is easily possible with no or little additional memory use (compared to the bulldozer way):
Do a SELECT loop on the menu (with one select query), and while doing that, remember what needs to be deleted, inserted, updated. Then execute.

To further reduce memory in special cases (if you are really serious about it):
- If the deletion queue grows too big, go back to bulldozer strategy.
- Delete immediately, don't queue deletions. I'm not sure if that would disrupt the SELECT loop, so maybe not.

If there's no guarantee of table locking, then it's Flag Time.

So, why not do this...?

Well, you may say, that variable might not ever get unset, and menu_router_build() might just keep spinning until the script times out.

To mitigate against this rare event, menu_cron() could look for the flag and if it was set before the previous call to menu_cron(), unset the variable and call menu_router_build() as a precaution.

Sounds weird, but I'm really looking for something reasonable here.

Explain to me why this might not be a good approach.

All of my modules are completely up-to-date and I continue to receive this error. Subscribing.

subscribe

I copied a site from one server to another. I made an exact copy and only changed settings.php.

On one server it is slow, showing errors and I can repeat the bug by changing a view, click save and while it is reloading browse around the site (since reloading takes over a minute). I use another browser to click around the site while the views page is reloading. This itself is not painstakingly slow, but it does generate the ' duplicate entry' errors.

On the other server I cannot repeat the bug. As a matter of fact, the page reloads so quickly I barely have time to browse around other pages.

Also the database seems to be slow 'randomly' while visiting the site it might be slow or even report not to be able to connect to the database server.

my solution seems simple enough: move to the another server. But I will never know why?

What is the difference between the servers?
Both are shared hosting. One is expensive and generates the bugs. One is dead cheap and works fine.
The expensive one where drupal shows this bugs is running FreeBSD 4.10, php 5.2.6, Apache/1.3.37, mysql 5.0.
The cheap one where drupal does not show this bugs is running a linux 2.6.9 probably fedora, php 5.2.3, apache 2.0.54 and mysql 5.0.

Could it be that the one database is just responding so much faster that there is no time for this problem to occur?

I found one thing that helped (after applying the patch) was to truncate the cache_menu and menu_router tables manually- it seems flush caches wasn't cleaning them properly.