I thought I should open a new issue for this. As I mentioned in an issue with the 6.x.1.1, I am using xmlsitemap on a D6 instance with about 200,000 nodes. There's really only about 2,000 nodes I want in the sitemap.
Note: As I was assigning the version here, I noticed there's an unstable5 that was released today after I downloaded unstable4. I will try the newer version after I post this and report the results.
I updated to the 6.x-2.0 branch, configured my content types to exclude a majority of them, then I ran the rebuild.
In the status, it looked like it went through all 200,000 of my nodes (even though the content types not excluded only add up to about 2000 nodes), and then the rebuild hangs at about 66% with a message "Remaining 4 of 9".
I let it sit for about an hour and then stopped it.
I started the rebuild again and it did the exact same thing. This time I thought maybe I didn't give it enough time, but it's been sitting there at 66%, 4 of 9 for a few hours now.
I haven't had a debugging session in the code yet, but maybe I can modify it like I did the previous version to only worry about the nodes where the node type is the ones that are included in sitemap. That would cut a lot of time off the rebuild for me.
On a side note: good job with the 2.x branch. I like the batch rebuild and the overview of content types which are included/excluded. The one thing that would have made my life super easy today is if that list of include/excludes was a form that I could set all content types with one submit :)
I'll update the post as I find more info.
Comments
Comment #1
dave reidOdd. Yeah you're the second person to be able to duplicate this, but I haven't been able to yet. Any kind of debugging you can do would be greatly appreciated.
Glad to know it's working (mostly) great for you. Yeah the admin screen would be hell for me if everything was configurable on it, because when each of those kinds of options are updated, we have to run updates sql on the {xmlsitemap} table, which for sites with lots of data, is LARGE. Putting all those options on the admin/settings/xmlsitemap would put a big database strain on submission. This way you get a snappy response, but a little more setup.
Maybe down the road I can look into doing some AJAX-y type of option popups with those options. Not a huge priority right now. :/
Comment #2
dave reidI'm going to mark this as a duplicate of #585070: Rebuilding of sitemap stops at 4/7 tasks for now.