Instead of trying to generate all sitemaps at once, we should add a {xmlsitemap_sitemap}.queued int column for when a sitemap is requested to be regenerated. Cron operations then pull in the specific sitemaps that have been queued, and update those.

CommentFileSizeAuthor
#3 xmlsitemap_queue_support-786486-3.patch4.44 KBjsobiecki
#1 786486-sitemap_update_queue.patch4.53 KBjsobiecki

Comments

jsobiecki’s picture

jsobiecki commented
Status: Needs work » Active
StatusFileSize
new 786486-sitemap_update_queue.patch4.53 KB

Hi Dave,

I had large problems with xmlsitemap cron execution on medium scale drupal site (200k nodes, split in 20 separate sitemaps). My problems were out of memory errors during sitemaps file execution.

As a solution, I replaced unprogressive batch solution used in xmlsitemap module with sitemap regenerations queue (by using drupal_queue module). Attached patch describes those changes (patch against 6.x-2.0-beta2). It's still bit dirty solution, but it works for me and can be considered as early proof of concept.

Anonymous’s picture

Anonymous (not verified) commented
Status: Active » Needs work

The patch needs to be generated for 7.x-2.x-dev and use the git repository instead of a file release.

jsobiecki’s picture

jsobiecki commented
Status: Active » Needs review
StatusFileSize
new xmlsitemap_queue_support-786486-3.patch4.44 KB

Attached patch is created against branch 7.x-2.x from git repository.

Status: Needs review » Needs work

The last submitted patch, xmlsitemap_queue_support-786486-3.patch, failed testing.

Anonymous’s picture

Anonymous (not verified) commented 
+function xmlsitemap_sitemap_queue_worker($smid) {
+  module_load_include('generate.inc', 'xmlsitemap');
+  xmlsitemap_regenerate_batch_generate($smid);
+  xmlsitemap_regenerate_batch_generate_index($smid);

Should be

+function xmlsitemap_sitemap_queue_worker($smid) {
+  module_load_include('generate.inc', 'xmlsitemap');
+  xmlsitemap_regenerate_batch_generate(array($smid));
+  xmlsitemap_regenerate_batch_generate_index(array($smid));