Posted by anoopjohn on September 22, 2009 at 10:45pm
Jump to:
| Project: | XML sitemap |
| Version: | 6.x-2.0-unstable4 |
| Component: | xmlsitemap.module |
| Category: | bug report |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | closed (fixed) |
Issue Summary
When I try to rebuild the sitemap it starts with the process, goes through all the 14 thousand odd links and gets hung with 4/7 pending tasks. Even after waiting for a very long time it was still waiting at that position. I could see in firebug that there was a post to the url every second but there was no change in the progress bar. I finally went to the sitemap page and there was this warning about the sitemap not correctly built. I then went to the rebuild page again and rebuit the sitemap without xmlsitemap and xmlsitemap_node unselected. This time it started up with 3 tasks and finished creating the sitemap successfully.
Comments
#1
I set up the complete site on my local machine today and the rebuild went through completely on the local machine. I still have the same issue on the webserver though. Would this problem have to have some issues with file permissions?
#2
Not file permissions, you're already creating files. The issue maybe more related to the number of modules you have installed. Or what those modules are doing. Do you find an error in your php log file? Do you find an error in your mysql log file? Do you find an entry in the watchdog log (admin/reports/dblog)?
#3
Hmm, sounds like it caught on the xmlsitemap_node part. Anything in your watchdog or server error longs to help us out?
#4
Here are some of the errors that I found.
1)
Type access deniedDate Thursday, September 24, 2009 - 17:42
User Anonymous
Location http://www.benzinga.com/batch?op=start&id=24
Referrer
Message batch
Severity warning
Hostname 66.249.71.208
Operations
2) A tonne of following menu router errors.
Type phpDate Thursday, September 24, 2009 - 17:54
User Anonymous
Location http://www.benzinga.com/bzstocks/js/ig
Referrer http://www.benzinga.com/16355/ace/top-traded-stocks-f-crbc-intc-amr-xlnx... Duplicate entry 'newsletter/subscriptions' for key 1 query: INSERT INTO menu_router (path, load_functions, to_arg_functions, access_callback, access_arguments, page_callback, page_arguments, fit, number_parts, tab_parent, tab_root, title, title_callback, title_arguments, type, block_callback, description, position, weight, file) VALUES ('newsletter/subscriptions', '', '', 'user_access', 'a:1:{i:0;s:24:\"subscribe to newsletters\";}', 'drupal_get_form', 'a:1:{i:0;s:36:\"simplenews_subscription_manager_form\";}', 3, 2, '', 'newsletter/subscriptions', 'Manage newsletter subscriptions', 't', '', 4, '', '', '', 0, 'sites/all/modules/simplenews/simplenews.subscription.inc') in /home/benzinga/benzinga.com/includes/menu.inc on line 2423.
Severity error
Hostname 75.208.248.251
Operations
I had not taken the site offline while rebuilding the sitemap. I also have cron running every 2 minutes.
There were no php errors in the apache error log.
I then rebuild the sitemap with the xmlsitemap and xmlsitemap_node unselected in the select box in the rebuild page. The following error was recorded twice.
Type access deniedDate Thursday, September 24, 2009 - 18:12
User Anonymous
Location http://www.benzinga.com/batch?op=start&id=25
Referrer
Message batch
Severity warning
Hostname 66.249.71.208
Operations
#5
@Dave - Any suggested watchdog calls to debug this issue?
#6
Other people have reported a very similar issue with getting an access denied when running a batch process. It usually means that your database went down or away for some reason. You might want to give #434032: Batch update fails with access denied message a read and check if tweaking your database's configuration file helps.
#7
Marked #599710: Rebuild Sitemap stuck at 66% Remaining 4 of 9 as a duplicate of this issue.
#8
Can anyone confirm you've still had this problem with the latest code at all? If not, feel free to mark the issue as fixed.
#9
I have not rebuilt my sitemap on the live site since then. I had updated to version 6.x-2.0-unstable5. I have just tested the rebuilding on the development server with XML sitemap 6.x-2.0-unstable5 and with 50000 nodes and it completed without any errors. I however found that the watchdog call did not have the time output. Created a new issue for that.
So the issue does not exist for me anymore and this can be marked as fixed.
#10
Thank for reporting back anoopjohn. Much appreciated.
#11
Automatically closed -- issue fixed for 2 weeks with no activity.
#12
Ok I have tried to rebuild the sitemap and hit a similar issue again. Only that the processing gets stopped while still processing the nodes. However I have a feeling that the issue could be with mod_fcgid used on the server which is throwing internal server errors once in a while. I will post back if it was indeed an xmlsitemap issue or the server issue. BTW I was curious as to why the sitemap cannot be rebuilt partially over several crons?
Anoop
#13
My new issue is more like #720878: Problems running cron to build the sitemap on large sites. I have posted more information about the issue at that thread