It appears, after MUCH searching and exploration as to why i could not get cron to run my drupalCron.php file, that Bad behavior is the culprit Here's a log file copied from the Watchdog:

IP Addr 208.64.136.179
Hostname 208.64.136.179 (whois)
Date 2007-01-19 22:20:02
Request type GET
Host tourlinkconference.com
URI /drupalCron.php
Protocol HTTP/1.0
Referrer
User Agent lwp-trivial/1.41
Headers GET /drupalCron.php HTTP/1.0 User-Agent: lwp-trivial/1.41 Host: tourlinkconference.com
Request Entity
Denied Reason User-Agent beginning with 'LWP' prohibited
Response 403

So, I have a crontab entry that calls drupalCron.pl from a directory one step higher than the root website dir, then the drupalCron.pl file runs the drupalCron.php file that is living inside the root Drupal 4.7.4 website dir.

What should we do about this? Is the whitelist the best way to go? Or should this User Agent string be coded into the module/third-party files?

Comments

senpai’s picture

senpai commented

I found another tidbit of info today while trying to get my own cron job running. Apparently, a user-agent string of LWP/trivial is a well known perl scraper-bot and can also be a worm script. Here's a link to some posts on this subject

In my case, I'm using a perl wrapper to run a cron.pl file that, in turn, calls the cron.php file inside my Drupal directory. Bad-Behavior prohibits this perl script from connecting via http because it has a user-agent string of LWP/trivial. I also found out that unless the perl script's owner has set a specific user-agent string, perl is gonna default to LWP/trivial. This is bad for anyone who's using Bad-Behavior and is also limited to cron-via-a-perl-script.

I will post more after exploring the setting of a unique user-agent string. My web hosting company is helping me out with this troubleshooting. We'll see how it goes.

funana’s picture

funana commented

Blocking my server cron jobs, too. I have to deactivate it...

senpai’s picture

senpai commented

I received this link to the cpan PERL User Agent guide from my web hosting company. Perhaps I'll be able to make sense of it someday, and actually re-write the default perl script with a user-agent that Bad Behavior likes. BTW, if anybody knows how to write perl, have a crack at this:

#!/usr/bin/perl
use LWP::Simple;
$doc = get('http://example.com/drupalCron.php');
print "Content-type: text/html\n\n";
print $doc;

If I receive a followup to this issue that solves the user-agent problem, I'll make a Haodbook entry for it, and also notify the module maintainer and the third party bad-behavior developer as well.

senpai’s picture

senpai commented
Status: Active » Fixed

I'm closing this because it's been over one year with no movement.

funana’s picture

funana commented

Closing is okay, but I know how to resolve the "blocking cron" issue. BB has now an option to whitelist by IP or user agent. While whitelisting User Agents is not a very good idea, the IP whitelisting works very well. To enable this, just edit the whitelist file in the BB direktory and add the IP Address you would like to unblock. Take a look in the BB Logs inside Drupal to find out the IP Address. You can do the same for other IP Addresses you would like to unblock (i.e. Amazon Crawlers, etc.).

senpai’s picture

senpai commented

Awesome! Thanks for the followup info, Funana.

Anonymous’s picture

Anonymous (not verified) commented
Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.