I do hate those "sometimes" issues but I'm afraid this one only happens sometimes and I haven't found any logic yet, as and when it happens or what causes it.

We're hosting 20 domains at the moment and all of them use the sitemap module. It's usually working very well. Once or twice a week though, one of those sitemaps appears to be empty - reported by Google webmaster tools.

When I then go to the server, all the files in the xmlsitemap directory are available and if I open them in an editor, they appear to be OK. However, if I want to open them in the browser with http://www.domain.com/sitemap.xml I also get an empty page with a warning that no element was found.

At that point - I guess as a side effect - my Drupal session get's closed and I have to login again.

To repair this issue I have repaired databse tables, cleared cache, and what else. Doing this several times, at some point the sitemap then comes back and works fine. But I don't exactly know which of the actions is finally fixing my issue. Nor do I know what was causing it.

Even worse, there is nothing in the logfile. So I have no idea where to look next.

Any help is much appreciated.

Comments

jurgenhaas’s picture

Just went through some heavy debugging and I haven't found the issue yet but here is a sequence of functions that have been invoked before the thing failed:

hit xmlsitemap cron
_xmlsitemap_ping
_xmlsitemap_link_count
_xmlsitemap_links
hit xmlsitemap xmlsitemap_links
hit xmlsitemap xmlsitemap_links
hit xmlsitemap_node xmlsitemap_links
hit xmlsitemap_term xmlsitemap_links
hit xmlsitemap_node xmlsitemap_links
hit xmlsitemap xmlsitemap_links
hit xmlsitemap_node xmlsitemap_links
hit xmlsitemap_term xmlsitemap_links

After that it just fails and I haven't figured yet exactly where this happens.

jurgenhaas’s picture

OK, now I found it, it's this line

array_multisort($nid, $loc, $links);

in function xmlsitemap_node_xmlsitemap_links($type = NULL, $excludes = array()) in xmlsitemap_node.module

Looks like either something is wrong in the array to be sorted or the system runs out of resources because the array is too big? No idea, any suggestions?

jurgenhaas’s picture

...and if I disable this line array_multisort... then it fails at this line $count = isset($count) ? $count : count(_xmlsitemap_links()); in xmlsitemap.module where I can say that _xmlsitemap_links() finishes completely, so that we know that the function count() fails.

darren oh’s picture

Sounds like the array can be too big for your system. We may need to use a database table instead.

jurgenhaas’s picture

Yes, maybe. Although it's a pretty powerful server with 5GB RAM. The domain failing is having 150.000 nodes and we add a couple of thousands a day. Can we somehow help to get this done with a table quickly? Just because this is something in production and I don't want to upset Google with an empty sitemap.

darren oh’s picture

Version: 5.x-1.3 » 5.x-1.4
jurgenhaas’s picture

Version 1.4 doesn't make any difference. Unfortunately.

jurgenhaas’s picture

Title: Sitemap sometimes empty » Sitemap sometimes empty (Patched to work with db table)
StatusFileSize
new15.67 KB

The last couple of days I've worked on the attached patch to make the module work with a database table instead of an array. It seems to work pretty well. Looking forward to some feedback.

darren oh’s picture

Title: Sitemap sometimes empty (Patched to work with db table) » Sitemap sometimes empty
Status: Active » Needs work

The patch has too much unrelated code to review.

jurgenhaas’s picture

What do you mean, can you please a bit more specific? I think all the code is problem specific.

darren oh’s picture

Maybe some of the code is for problems specific to your site. The patch needs to be limited to replacing the array with a database table.

jurgenhaas’s picture

Well no. It's only 2 lines (gss location problem: first block) and (multi domain support: first line in third block). Everything else is related to the current problem.

darren oh’s picture

The patch needs to be cleaned up to be ready for review. If it's working for you, you can wait until I have time to do it.

druvision’s picture

Me too.

Got the error

XML Parsing Error: no element found
Location: http://emfpollution.com/sitemap.xml
Line Number 1, Column 1:

Subscribing.

Amnon
-
Professional: Drupal Israel | Drupal Development & Consulting | Eco-Healing | Effective Hosting Strategies | בניית אתרים
Personal: Hitech Dolphin: Regain Simple Joy :)

ashtronaut’s picture

ashtronaut’s picture

I have the same error. . . . .

XML Parsing Error: no element found
Location: http://mypetstreet.com/sitemap.xml
Line Number 1, Column 1:

Any ideas???

zombiee08’s picture

Same error : XML Parsing error. I tried the patch but still the same error.

The issue seems to be the way XML is rendered. There are 2 junk characters that are present at the start of the XML. Due to this the browser throws an exception in parsing the XML. Since the starting XML tags should not have any thing before that.

I tried :
- changing the encoding but that does not solve the purpose.
- tried extracting the XML from the gzip using gzread and then displaying it. did not solve the purpose
- manually extracted the XML and pointed the code to open the XML instead of the gzip.
- read another small XML from the same path.
- without any encoding it gives complete XML data without the nodes. On viewing the source one can see the XML but if I select all using cntrl + A and paste somewhere it shows 2 junk characters.

The following is the error i got on different browsers
Mozilla :XML Parse error
IE7 :Page cannot be displayed
IE6 :Blank Page
Opera :Some junk Characters

Please also look into this issue, it might be the reason why parsing problem occurs

ik2xnw’s picture

Title: Sitemap sometimes empty » Sitemap XML: non well-formed - Interpretation error

That happens to me when i'm not logged in.
When logged out, i exit and re-open the browser in order to meet the problem.

ashtronaut’s picture

The solution in #18 did not work for me at all. The post in #17 is consistent with my problem. The weird thing is, I can see a sitemap on my development machine (localhost), but once I post this to the live server, I get the xml parsing error and blank pages etc. I have read through other posts and tried to re-create the xmlsitemap folder, as well as change the permissions, but no dice yet. Anybody have any idea on whats going on here, or a possible fix?

ash

ashtronaut’s picture

I finally figured out my problem. . . . . .

http://drupal.org/node/230877#comment-774468

ash

wayland76’s picture

subscribe

wayland76’s picture

StatusFileSize
new16 KB

Hi all. The attached patch takes Juergen's patch above, and does the following:
1. Remove the site-specific stuff (mentioned in the above posts)
2. Changes the database alterations so they work with a new install as well as upgrade
3. Attempts to implement things for Postgres as well (but unfortunately I don't have a postgres setup to test with)
4. Makes the changes recommended by the Coder module, except one SQL thing that the Coder module was complaining about. The Coder module does automatic review of the code for conformance to the Drupal Coding Standards

Hope this is more applicable than the last patch. The patch is against the latest version of the DRUPAL-5 branch.

@Darren: is there anything else that you want to see in this patch before you apply it (other than testing)?

wayland76’s picture

Status: Needs work » Needs review

(Forgot to set status -- code needs review as per #22).

Freso’s picture

Status: Needs review » Needs work

I'm not sure it's a good idea to have the SQL in a function. What happens when you have to update the table structure? Sure, you'll have to update the SQL used in xmlsitemap_install() - but you also don't want to change the SQL of xmlsitemap_update_1()!

Also, since you've made the query for pgsql as well, why are you only calling the function when dbtype is mysql/i?

Freso’s picture

Version: 5.x-1.4 » 5.x-1.x-dev
Status: Needs work » Needs review
StatusFileSize
new17.14 KB

The previous patch also didn't apply cleanly to latest DRUPAL-5 branch code. Re-rolled with corrections for .install mentioned above, as well as various style corrections (capitalisations and single quotes instead of double ones) and adding t() to some introduced Watchdog calls.

Freso’s picture

Oh, and for the record: I haven't actually tested the above.

Freso’s picture

Hm. I just spotted a curiosity: Why is _xmlsitemap_xml_links() in xmlsitemap.module making a return $links already on its second line now? Won't that effectively stop the rest of the code from running? If so, either the rest of the code should be removed (cruft) or the unintentional behaviour should be fixed.

I also noticed I missed uppercasing a "true" in _xmlsitemap_array_slice()...

Freso’s picture

StatusFileSize
new17.15 KB

Style (capitalisation) corrections and a break. Question about return $links from previous comment still stands unanswered.

wayland76’s picture

Well, I'm afraid I don't know the answer to that question; that code was added by juergen. If I get time, I'll have a look at it.

wayland76’s picture

StatusFileSize
new18.19 KB

Ok, I've had a look, and gotten rid of that function altogether, and things still seem to work. But it should still be reviewed :).

darren oh’s picture

Version: 5.x-1.x-dev » 5.x-2.x-dev
Status: Needs review » Fixed

Fixed in CVS commit 116912. This approach requires modules that implement hook_xmlsitemap_links() to be updated, so the fix will only be applied to the 2.x branch.

Anonymous’s picture

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.

tracy_pilcher’s picture

Hi,

Make sure you run cron before you try accessing the sitemap.xml file for the first time.

That fixed this problem for me.

ansari.wajid’s picture

Version: 5.x-2.x-dev » 6.x-1.1
Component: xmlsitemap » xmlsitemap.module
Priority: Critical » Normal
Status: Closed (fixed) » Active
Issue tags: +error, +XML parsing error, +xmlsitemap, +sitemap, +xmlerror, +system error

Hi,

I am getting below errors on different browsers when navigating to the sitemap file: http://chiprv.org/sitemap.xml. Please let me know what might be the problem. Thanks

Error details on Fire Fox:
==================

XML Parsing Error: not well-formed
Location: http://chiprv.org/sitemap.xml
Line Number 58, Column 9:
<ul ="links primary-links"><li class="menu-166 first"><a href="/about-us" title="About Us">About Us</a></li>
--------^

Error details for IE:
===============

The XML page cannot be displayed
Cannot view XML input using style sheet. Please correct the error and then click the Refresh button, or try again later.
--------------------------------------------------------------------------------
System error: -2146697204. Error processing resource 'http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd'.

jurgenhaas’s picture

The part <ul ="links primary-links"> should be <ul class="links primary-links">, so the attribute-name class is missing in the XML-code. But I'm not sure who is getting this wrong here. Is the sitemap.xml file created by the XMLsitemap module in your case?

avpaderno’s picture

@ansari.wajid: You posted the same comment in three different issues; please stop cross posting, and adding the comment to reports that have nothing to do with your issue.

As I already reported, that is not the output of XML sitemap. Are you sure you are not using Sitemap?

avpaderno’s picture

Version: 6.x-1.1 » 5.x-2.x-dev
Priority: Normal » Critical
Status: Active » Closed (fixed)

I am restoring the report metadata as they were before.

ansari.wajid’s picture

Thanks for your reply.

I am using xmlsitemap module to generate sitemap.xml file. I have 2 domains pointing to my site chipr.org and chiprv.dev.triocircle.com. I have enabled xmlsitemap module through chiprv.org. The sitemap is generated on chiprv.dev.triocircle.com/sitemap.xml, but giving error on chiprv.org/sitemap.xml. I unable to figure out what might be going wrong. Please help me resolve this issue.

thanks

avpaderno’s picture

I am using xmlsitemap module to generate sitemap.xml file.

Which version?

dave reid’s picture

@ansari.wajid: Please open a new issue instead of continuing a closed issue.