Execution time

samdeskin - June 27, 2008 - 21:35
Project:XML sitemap
Version:6.x-1.x-dev
Component:xmlsitemap.module
Category:support request
Priority:normal
Assigned:Unassigned
Status:closed
Description

I have a somewhat large website that I am trying to use the XML Sitemap module on. When I turn on XML Sitemap Terms, it works fine - in fact it works exceptionally fast - 1857 urls in about a second.

When I try to turn on Nodes, it takes a number of minutes for the server to populate the database. Then, I go to sitemap.xml and it gives me an error.

XML Parsing Error: no element found
Location: http://openjurist.org/sitemap.xml
Line Number 1, Column 1:
^

I have lowered the number of URLs to 1800 (since that worked before) and it takes a little while to load and then it still gives me this error.

My website currently has about 625,000 pages. We are working on expanding the website, but we need sitemaps to work!

One other thing that I have noticed... When I lower the number of URLs on the page to an amount that requires 2 or more pages it creates a list of sitemap pages, but those sitemap pages are links to Pages that do not exist.

Please let me know what we might do to get it work properly.

Sam

#1

nugz - July 4, 2008 - 00:12

I clicked your sitemap link and it is working for me. It shows 1857 links. As for the speed, it loaded in less than 2 seconds.

#2

samdeskin - July 4, 2008 - 16:42

Hi,

The website has 625,000 pages.

When you last looked, the Sitemap was showing 1857 pages - because it was showing only the "XML Sitemap: Term" module's links.

Now, I have turned on "XML Sitemap: Node" module for you to see what that does... I have the Chunk Size set to 1000.

http://openjurist.org/sitemap.xml

It bumps up the table size for xmlsitemap_node to over 30 megs:
xmlsitemap_node 623,121 MyISAM utf8_general_ci 30.8 MB
xmlsitemap_term 1,856 MyISAM utf8_general_ci 77.1 KB

#3

drupalnuts - July 28, 2008 - 23:57

I am having the exact same issue. I only have 40,000 nodes

#4

samdeskin - August 14, 2008 - 16:10

Hello,

I wonder if anyone might be able to help us with this problem. We have very large sites that need sitemaps and are having lots of trouble getting it to work.

Thanks!

Sam

#5

tobiasr - August 15, 2008 - 07:13

Same problem on a site with 130,000 nodes.

#6

sfrooz - August 19, 2008 - 10:50

Hello,

Same problem here, we have very large site with 1.000.505 nodes...

#7

samdeskin - September 1, 2008 - 00:27

Any Chance that anyone might help???

Please ...

#8

taqwa - September 4, 2008 - 22:24

same issue

#9

taqwa - September 8, 2008 - 17:59

Has anyone found a resolution for this?

#10

taqwa - September 9, 2008 - 15:30

i think it may be related to this error that comes up in my server log when I disable the xmlsitemap nodes module:
PHP Fatal error: Allowed memory size of 16777216 bytes exhausted (tried to allocate 58368 bytes) in /chroot/home/example/example.com/html/sites/all/modules/date/date.inc on line 1771, referer: http://www.example.com/admin/build/modules

#11

ephman - September 25, 2008 - 18:17

i'm having the same issue, HOWEVER, only have 10 nodes, and get this error. infact no actual sitemap.xml gets generated.

thanks,
ephman

#12

dfletcher - October 17, 2008 - 17:52

I had the same issue. For me it turned out to be part of the upgrade from gsitemap. XML sitemap still calls hook_gsitemap, but the API is not the same, so any calls to gsitemap_* functions inside my custom hook no longer worked. "Call to undefined function gsitemap_url" in my case.

#13

KiamLaLuno - November 16, 2008 - 00:23
Title:Large Website» Problems with a large site
Component:xmlsitemap» xmlsitemap_node

#14

ereq - November 19, 2008 - 01:54

We are having the same problem to with the XML Sitemapl. Ha anyone found a solution?

#15

KiamLaLuno - November 19, 2008 - 12:34
Status:active» postponed

#17

KiamLaLuno - November 19, 2008 - 23:27
Title:Problems with a large site» Execution time

#18

imoria - December 22, 2008 - 04:22

Have same issue need help, willing to pay for support!

#19

KiamLaLuno - December 22, 2008 - 11:11
Version:6.x-0.x-dev» 6.x-1.x-dev

I am changing the referring version, as 6.x-1.x-dev is the only Drupal 6 version actually developed.

#20

samdeskin - January 23, 2009 - 23:47
Component:xmlsitemap_node» xmlsitemap.module

I figured out a way around this ...

My website does not change very often and is about 640,000 pages, so it worked for me:

http://xml-sitemaps.com

Here is the result:

http://openjurist.org/sitemap.xml

There are another few pages that branch off of this one ... please don't call them ... they are each about 60 megs... it is bad enough when googlebot does!

Hope this helps.

sam

#21

parrottvision - March 11, 2009 - 10:38

I have about 25000 nodes. I have just figured out how to populate old nodes into xmlsitemap. I have set the chunk size from 2000 to 200 but it still times out? I wonder if this is similar?

Update: I think what confuses is me is that by creating smaller chunks xmpsitemap then creates additional sitemaps reducing the pressure on the server but this doesn't seem to be happening?

#22

parrottvision - March 13, 2009 - 05:41

with the latest update I think this is working great on all my sites except for the timeout issues on my biggest sites.

#23

KiamLaLuno - March 13, 2009 - 10:23
Status:postponed» active

I think that the timeout is now only caused by the database rows being copied from a table to another; that will be probably fixed in the 6.x-2 branch, even though there could be a timeout when creating the site map content from the table rows.

#24

parrottvision - March 13, 2009 - 19:47

I had timeouts on the forced updates but by re-running update.php it pick up from where it left off and all nodes made it through. I look forward to the 6.x-2 branch and not being impatient as I know you have been working hard on this but is that in the near future?

#25

parrottvision - March 29, 2009 - 23:10

I have bought this (xml-sitemaps.com) as a stop gap till 6.2 gets released but can't get it going. It runs for a while then stops, it re-runs a few more times and then it just won't resume...odd.

#26

samdeskin - April 5, 2009 - 09:06

I suggest you contact the owner:

http://whois.domaintools.com/xml-sitemaps.com

http://www.xml-sitemaps.com/contact.html

He was helpful for me - I have a website with 650,000 nodes and he needed to help me too...

#27

earnie - April 14, 2009 - 14:45

@Kiam: Should this have a status of postponed since it would be fixed for 6.x-1.x-dev?

#28

KiamLaLuno - April 14, 2009 - 20:07
Status:active» postponed

It should not be fixed, as the modules still move data from a database table to another. It will be probably fixed in the future Drupal 6.x-2 branch.

I am going to set it as postponed for the moment.

#29

Dave Reid - May 22, 2009 - 18:12

Marked #336200: Execution time as a duplicate of this issue.

#30

earnie - July 2, 2009 - 15:21
Status:postponed» fixed

Since this is being addressed by version 6.x-2.x-dev I'm marking this fixed.

#31

System Message - July 16, 2009 - 15:30
Status:fixed» closed

Automatically closed -- issue fixed for 2 weeks with no activity.

#32

KiamLaLuno - July 16, 2009 - 15:45

It would be better to set it as "won't fix". A issue must be fixed in all the supported branches of a module; if you are not going to fix it, there is the correct status to use.

 
 

Drupal is a registered trademark of Dries Buytaert.