bizarre cron.php 404/406 - Page Not Found/Not acceptable

Posted by jsm174 on October 8, 2008 at 6:34pm

Hello.

I'm not sure where to start. I'm having an extremely bizarre problem running cron:

1) I turned on cron jobs for a site that was moved into production. The job is set to run every 45 minutes.

2) After an hour, realized the cron job didn't run.

3) With a browser, tried to hit cron.php and received a Page not Found

4) Logged into the site and ran update.php. Update worked as expected.

5) Tried to hit cron.php again and the cron job worked as expected!

6) A few minutes later, the scheduled cron job tries to kick off. It did't execute.

7) Tried to hit cron.php again and it returns a Page not Found.

In the severs access logs,

xxx.xxx.xxx.xxx - - [08/Oct/2008:12:40:45 -0500] "GET /cron.php HTTP/1.1" 200 - "-" "Mozilla/5.0 (iPhone; U; CPU iPhone OS 2_1 like Mac OS X; en-us) AppleWebKit/525.18.1 (KHTML, like Gecko) Version/3.1.1 Mobile/5F136 Safari/525.20"

xx.xx.xx.xx - - [08/Oct/2008:12:45:01 -0500] "GET /cron.php HTTP/1.1" 404 8325 "-" "lwp-request/2.06"

xxx.xxx.xxx.xxx - - [08/Oct/2008:12:47:01 -0500] "GET /cron.php HTTP/1.1" 404 2933 "-" "Mozilla/5.0 (iPhone; U; CPU iPhone OS 2_1 like Mac OS X; en-us) AppleWebKit/525.18.1 (KHTML, like Gecko) Version/3.1.1 Mobile/5F136 Safari/525.20"

In Drupal's Recent Hits,

10/08/2008 - 22:47   catalog               Anonymous   details
10/08/2008 - 22:45   Page not found /   Anonymous   details

In Drupal's Recent Log Entries,

warning  page not found    10/08/2008 - 22:45    406.shtml                  Anonymous	
cron                              10/08/2008 - 22:40    Cron run completed.    Anonymous

Now to further complicate things, I have the home page changed to go to ubercart's catalog page - catalog

I pinged cron.php several times consecutively and get page not found. Check the recent hits, it always points to catalog:

10/08/2008 - 22:51   catalog               Anonymous   details
10/08/2008 - 22:50   catalog               Anonymous   details
10/08/2008 - 22:50   catalog               Anonymous   details
10/08/2008 - 22:50   catalog               Anonymous   details
10/08/2008 - 22:50   catalog               Anonymous   details
10/08/2008 - 22:50   catalog               Anonymous   details
10/08/2008 - 22:50   catalog               Anonymous   details
10/08/2008 - 22:50   catalog               Anonymous   details

Has anyone experience this? I'm wondering if it's an .htaccess or a webserver issue?

Thanks
-- Jason

Comments

Did you check your cron.php

Posted by cog.rusty on October 8, 2008 at 8:01pm

Did you check your cron.php file in Drupal's directory, to see if it exists, if its permissions change in any way or if it gets corrupted?

Yup..

Posted by jsm174 on October 8, 2008 at 8:14pm

Yes. It's all good.

If I clear the cache (using the admin menu module), I can execute cron.php with no problem.

The minute cron.php is accessed from the server using lwp-request, then cron.php starts 404ing.

I know, extremely bizarre...

-- Jason

That sounds really weird...

Posted by jacobmn on October 8, 2008 at 9:13pm

That sounds really weird... Are you sure the cron job has been set up correctly? I ran into really weird problems like this once because something was not properly set in my crontab.

Maybe look here for suggestions? Otherwise if all else fails there is always Poor Mans' Cron (I know it's not preferred).

I just figured it out...

Posted by jsm174 on October 9, 2008 at 2:03am

Wow. I use Site5 for hosting. Check this out:

When I call

GET -x www.example.com/cron.php

I get back

LWP::UserAgent::new: ()
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: GET http://www.example.com/cron.php
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::Protocol::collect: read 400 bytes
LWP::Protocol::collect: read 4096 bytes
LWP::Protocol::collect: read 821 bytes
LWP::UserAgent::request: Simple response: Not Found
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
  <head>
    <title>Page not found | Example.com</title>
    <link rel="apple-touch-icon" href="/icon.png"/>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
.
.
.

I decided to try adding my own user-agent:

GET -H 'User-Agent: Site5 cron' -x www.example.com/cron.php

I get back

LWP::UserAgent::new: ()
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: GET http://www.example.com/cron.php
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::UserAgent::request: Simple response: OK

I guess they need a User-agent header in there. Very messed up..

-- Jason

Heh, they gave me wget

Posted by cog.rusty on October 9, 2008 at 3:01am

Heh, they gave me wget access after asking, and I had no problem with cron. And they are cheap! I mean it! Their drives are working permanently at 99% full and I am allowed 25K inodes in my infinite storage space!

Cool.

Posted by jsm174 on October 9, 2008 at 2:46pm

Oh, I know! I have no problem with Site5. I am more than happy with them!
(When we signed up, we got 6 years with unlimited everything for ~$300.)

I will contact their support and see if we can get wget access.

Still, funny thing is I had the same issue with cURL as well.

Thanks,
-- Jason

Ditto ditto

Posted by Reinette on December 17, 2008 at 7:54pm

Ditto on both problems (wget and curl), using another CPanel host. Thanks for the fix :)

Headers are missing

Posted by farzan on March 9, 2009 at 9:24pm

If this helps anyone...

I had the same problem with both lynx and curl. I fixed the curl with adding a fake user agent with switch -A:

/usr/bin/curl -A 'Konqueror/1.0' http://www.example.com/cron.php

for lynx or other programs, check their manuals.

Thanks

Posted by toomanypets on October 4, 2009 at 1:24pm

Thanks farzan. Adding the user agent solved my problem:
/usr/bin/curl -A 'Konqueror/1.0' http://www.foo.com/drupal/cron.php

Thanks!

Posted by njathan on April 10, 2010 at 8:52pm

Thanks a lot folks! this thread just rescued me :-)

curl -A 'Mozilla/5.0' http://example.com/cron.php

mod_security

Posted by CrookedNumber on May 3, 2009 at 11:01pm

There could be an issue with apache's mod_security as well. If mod_security grabs the request and tries to send it to, say, 406.shtml, drupal will -- in all likelihood -- return a 404 (since 406.shtml likely doesn't exist in your drupal install).

More info here: http://drupal.org/node/110219

http://CrookedNumber.com

404/406 Plus +bad interpreter+ +access denied+ +etc+

Posted by romanse on May 20, 2009 at 9:56pm

Please have a look at my rather long comment related to the above at Multisite cron-curl.sh.
It required a dozen emails to/from the webhost and lots of scratching together clues to get around the above errors and others.

Thank you farzan for a critical piece of the puzzle which was the -A option to get cron-curl.sh to ask
for a browser to borrow!

the Portal

If you're experiencing 404's

Posted by dale42 on May 28, 2009 at 5:24am

If you're experiencing 404's with cron.php, it might be this issue: http://drupal.org/node/177790