By superjacent on

Hope someone can help. I've just started on a new site but I'm noticing in my recent log entries quite a few page not found errors relating to 500.shtml.

page not found 3 Feb 2009 - 17:00 500.shtml Guest

The detail of the above error is as follows:

Type	page not found
Date	Tuesday, 3. February 2009 - 17:00
User	Guest
Location	http://satmemory.com/cron.php 
Referrer	
Message	500.shtml
Severity	warning
Hostname	xxx.228.230.xxx

Can somebody shed some light on this (the cron jobs are working).

Thanks,

Steve
satMemory (birth of a site)

⋅ Categories: Drupal 6.x

Comments

superjacent’s picture

superjacent commented

Bump...... anyone......
___________________________

Steven Taylor
Prime357

superjacent’s picture

superjacent commented

I'm not sure what is going on, I've now noticed that the page not found error for 500.shtml is preceded by this:-

Type	cron
Date	Thursday, 5. February 2009 - 11:00
User	Guest
Location	http: //satmemory.com/cron.php 
Referrer	
Message	Cron has been running for more than an hour and is most likely stuck.
Severity	error
Hostname	xxx.228.230.xxx
Operations

When I manually run cron all appears to be working and no error messages are generated. Does anybody have any ideas about this? Is Cron the culprit for the 500.shtml error? If so (if not) how do I resolve it.

___________________________

Steven Taylor
My new site, the beginnings of

ps. I purposely broke the cron link above, a space between http: and the rest of the link.

c4clicks’s picture

c4clicks commented

Hi Steven,

I'm having the same problem on two sites - live and development. I first noticed the problem on the live site as cron was only being run once a day on development, but since I have increased the interval on dev to every hour, it is more noticeable. I estimate that it fails about 50% of the time - but I can't see any pattern - the failure times don't even match between live and dev sites!

For reference I am running Drupal V6.9 at DrupalValueHosting [I took the liberty to check that these are common factors to both of us]. Also I have the following contrib modules installed and enabled:

cck 6.x.2.1
google analytics 6.x.2.0
Meta tags (nodewords) 6.x.1.0
Page Title 6.x.2.0
Pathauto 6.x.1.1
Views 6.x.2.3
Webform 6.x.2.5

There is no output to either error.log or from the cron run itself.

I now plan to install a test site with just Drupal 6.9, no contrib modules and run cron every 15 minutes to monitor for failures. The aim is to narrow the problem down to drupal core or DVH - I'll update with my findings...

Gary

superjacent’s picture

superjacent commented

Thanks, I'll await your updates.

___________________________

Steven Taylor
Prime357

sime’s picture

sime
any/any
Location Melbourne
commented

This is (sometimes?) a server error thrown by (eg) apache after it times-out the server request. This is different to a php error, you can set php's execution time to 0 (ie. no limit) but apache might still time out.

Ideally it shouldn't run that long (on my server that would be 5 minutes). The trick is to find out what's hanging in cron, ie. which module's cron hook, because the drupal error message won't tell you. A non-coder could start by disabling each module until the problem goes away, then follow up depending on what the module is. A coder could of course go to module_invoke_all and debug on the 'cron' hook'
http://api.drupal.org/api/function/drupal_cron_run/6
http://api.drupal.org/api/function/module_invoke_all/6

Anyway, that's about as much as I know, I might have some of the description wrong, but in my case the problem was solved by more efficient code.

c4clicks’s picture

c4clicks commented

Sime - I agree, the more I look into this the more it looks like a server error.

Worryingly I've just noticed we are seeing the 500 errors appear for index.php as well. Both the cron.php and index.php errors seems to point back at the fast cgi module (mod_fcgid) so I suspect some sort of config problem.

Anyway, I have submitted a ticket so I'll just have to wait and see what happens ...

c4clicks’s picture

c4clicks commented

I added debug output to the module_invoke_all module as suggested above by sime.

A successful cron run gives the following watchdog entries:

cron 24 Feb 2009 - 11:00 Cron run completed.
debug 24 Feb 2009 - 11:00 module_invoke_all: views_flush_caches
debug 24 Feb 2009 - 11:00 module_invoke_all: content_flush_caches
debug 24 Feb 2009 - 11:00 module_invoke_all: update_flush_caches
debug 24 Feb 2009 - 11:00 module_invoke_all: hook=flush_caches
debug 24 Feb 2009 - 11:00 module_invoke_all: hook=cron
debug 24 Feb 2009 - 11:00 module_invoke_all: hook=init

The next run failed with the 500 error, and appears (as per the message detail) that it did not find cron.php. I assume all the calls following the error are related to the 'Page Not Found' error?

debug 24 Feb 2009 - 12:00 module_invoke_all: hook=exit
debug 24 Feb 2009 - 12:00 module_invoke_all: token_token_values
debug 24 Feb 2009 - 12:00 module_invoke_all: page_title_token_values
debug 24 Feb 2009 - 12:00 module_invoke_all: token_token_values
debug 24 Feb 2009 - 12:00 module_invoke_all: page_title_token_values
debug 24 Feb 2009 - 12:00 module_invoke_all: content_token_values
debug 24 Feb 2009 - 12:00 module_invoke_all: user_token_values
debug 24 Feb 2009 - 12:00 module_invoke_all: taxonomy_token_values
debug 24 Feb 2009 - 12:00 module_invoke_all: node_token_values
debug 24 Feb 2009 - 12:00 module_invoke_all: comment_token_values
debug 24 Feb 2009 - 12:00 module_invoke_all: hook=token_values
debug 24 Feb 2009 - 12:00 module_invoke_all: hook=footer
debug 24 Feb 2009 - 12:00 module_invoke_all: forum_node_info
debug 24 Feb 2009 - 12:00 module_invoke_all: webform_node_info
debug 24 Feb 2009 - 12:00 module_invoke_all: hook=node_info
warning page not found 24 Feb 2009 - 12:00 500.shtml
debug 24 Feb 2009 - 12:00 module_invoke_all: hook=init

On the next run it appears to have worked (except for the all important missing 'Cron run completed' message):

debug 24 Feb 2009 - 13:00 module_invoke_all: views_flush_caches
debug 24 Feb 2009 - 13:00 module_invoke_all: content_flush_caches
debug 24 Feb 2009 - 13:00 module_invoke_all: update_flush_caches
debug 24 Feb 2009 - 13:00 module_invoke_all: hook=flush_caches
debug 24 Feb 2009 - 13:00 module_invoke_all: hook=cron
debug 24 Feb 2009 - 13:00 module_invoke_all: hook=init

So an hour later I'm informed:

"Cron has been running for more than an hour and is most likely stuck."

Does anyone have any idea what may be happening or how I might debug this further please?

sime’s picture

sime
any/any
Location Melbourne
commented

I notice it sometimes happens to us inside a minute so my guess about server timeouts is probably wrong. Interestingly we get the same behaviour with the same site upgraded to d6 and on nginx not apache. weird

jenlampton’s picture

jenlampton
she / her
commented

I'm having the same problem on my server, but its caused by the XMLsitemap node module (I am using CCK 6.x-2.5).

With XMLsitemap node disabled sitemap.xml displays fine, when I enable XMLsitemap node I get a 404 error on sitemap.xml and my watchdog has an entry for:

page not found 2010-01-11 11:40 500.shtml

I tried increasing the php memory limit, and adding the code above to my .htaccess (just for kicks) but neither prevents the 404 / 500 error.

What's going on on my server that needs to be adjusted? (I'm on a shared server and the hosting company is unable to identify any problems).

Jen

bassplaya’s picture

bassplaya commented

In the Apache log it says: Access denied with code 500.
there is a solution for this problem but I'm not sure whether or not this would create a security hole.
in the .htaccess you can add the following:

<IfModule mod_security.c> 
SecFilterEngine Off 
</IfModule>

Guys, I'm not the expert here but it found that it works.
In my case my cck content types all work fine but the "page" and "story" gave an error message.

Authentically,
BassPlaya

suydam’s picture

suydam commented

I checked my apache error logs and found the culprit.
I suspect others may run into the same thing.

Using CCK 6.x-2.6 I created a text field with a select box as the widget.
If I try to "Change basic informatioN" about the field I get a page not found error and my logs show a 500.html error as described on this page.

Here is what my apache instance thinks:
[Wed Nov 18 06:12:38 2009] [error] [client 71.205.109.168] ModSecurity: Access denied with code 500 (phase 2). Pattern match "(insert[[:space:]]+into.+values|select.*from.+[a-z|A-Z|0-9]|select.+from|bulk[[:space:]]+insert|union.+select|convert.+\\(.*from)" at ARGS:previous_field. [file "/usr/local/apache/conf/modsec2.user.conf"] [line "355"] [id "300016"] [rev "2"] [msg "Generic SQL injection protection"] [severity "CRITICAL"] [hostname "i.changed.this.to.post.on.drupal.org"] [uri "/admin/content/node-type/profile/fields/field_oh_counties"] [unique_id "UhKm@UPhj1QAAHpdl24AAAAE"]

I've not taken it any further since I think either CCK is doing something naughty and opening itself up to injections, or my host is being overly strict. Either way, I'm going to work around it.

I'd suggest others get their hands on their apache error logs. The clue is probably in there.

Just for the sake of making this thread useful to others, there is a CCK issue open on this matter as well: http://drupal.org/node/265255

superjacent’s picture

superjacent commented

My hosting company (drupalvaluehosting) have now changed servers and everything seems to be working fine. Not getting as many of these error messages and checking the details behind the error messages they seem to be correct.

3rdvalve’s picture

3rdvalve commented

I discovered this same issue by running a site crawl testing tool (SeoMoz). Whenever I ran it a bunch of pages randomly generated server error 500s and 404s. No pattern. Sometimes a page would error out sometimes not. Drupal generated this 500.shtml errors when this happened.

I checked the server error logs and could see a series of the following errors around the time the crawl test ran:

SystemException in API_Linux.cpp:172: setuid() failed: Resource temporarily unavailable

and

Premature end of script headers: index.php

I contacted my hosting company - Hostgator - they said my account is hitting its process limit (which is usually set to 25 by default).

They resolved the issue by increasing the limit to 50 by editing the /usr/local/apache/conf/includes/pre_main_global.conf file. The setting is "RLimitNPROC 25 30". Change the first number to the new limit, then add 5 to get the second number, and finally restart Apache.

This seems to have resolved the problem.

jameswoods’s picture

jameswoods commented

I hope this post helps someone.

I was getting an 'page not found' error when trying to view files that were uploaded to a node (both with cck filefield and the core upload module).

My apache logs had the following error:
[Sat Apr 24 23:07:36 2010] [error] [client xxx.xxx.xxx.xxx] Options FollowSymLinks or SymLinksIfOwnerMatch is off which implies that RewriteRule directive is forbidden: /home/account/public_html/subfolder/sites/default/files/coin_toss.gif

in my /sites/default/files/.htaccess file, I commented out the following lines:
Options None
Options +FollowSymLinks
-changed to-
#Options None
#Options +FollowSymLinks

and all was well in the world.

-James

m.tavousi’s picture

m.tavousi commented

it help me.

freddym’s picture

freddym commented

As suydam mentioned above http://drupal.org/node/368422#comment-2278966
mod_security can cause this problem, in our case we could reproduce the error when we entered the following text in a new content node or in a webform: "select a shoe from the store where you are"
This should trigger an SQL injection rule and then trigger a 500.shtml page not found error.

If mod_security is your problem this post may help you: http://drupal.org/node/144582

Regards,
Freddy

vinoth.3v’s picture

vinoth.3v commented

Hi had similar issue with cron.php & update.php

permissions was set to 777. access denied and 500.shtml :(

but it works after I have changed it to 644

Vinoth - வினோத்

sydneyshan’s picture

sydneyshan commented

I've found a number of 'SoftException' entries in my log that were seemingly unrelated to the error 500.shtml issue. Upon resolving each one (setting the /sites directory to 755 and /sites/default/modules to 755) the error 500.shtml issue disappeared. It was only appearing when the ad module tried accessing its serve.php file via javascript reference.