cron doesn't work, so search didn't work, now trip_search doesn't work
I'm still having problems getting my site to be searchable. My cron.php hasn't worked since Feb 24th (as detailed here: http://drupal.org/node/53394), so my site hasn't been indexed since then. That means my users can't search all of the new content.
My understanding is that trip_search (which looks like a great module) is supposed to search in the actual database itself, which sounds great. Except that, even with the "full_text_indices.mysql" table installed, it doesn't do any better searching than the default "search.module" did.
What's stranger is that I used Webmin to create a schedule to run the /scripts/cron-lynx.sh script every 15 minutes...and it appears to be running cron.php! But, it's odd in the way it's doing it:
1. Every 15 minutes my newsfeeds are getting updated, as I would expect (and I can see the cron runs in watchdog)...but not all of them, and not every time. So, a random couple of them get updated one run, another couple another run.
2. Even though the cron run appears to be taking place, indexing of my site still reads "0% of 100 items have been indexed". I've tried changing the minumum number of characters, I've tried changing the number of items to index...nothing makes a difference. cron.php still does nothing to index my site.
I'm telling ya, I'm about ready to climb to the top of a tower with a high-powered rifle if I can't figure this out.
Vince
Cron ideas
I too was having lots of trouble getting cron to run on a new site. I was not able to get it to run even from the web browser (http://www.mysite.com/cron.php) -- I got a system 500 error and a notice that it did not complete. I finally figured out my problems (there were a couple) and have it working. Don't know if they have any relation to your problems, but here's what got it working for me:
I tried to figure out what specific module was creating the problem by unselecting all contributed modules, then adding them back in one at a time and trying a cron run. By that process I found out it was getting stuck by the search module. I could have all other modules activated and there would be no problem with cron, but with search activated cron quit working. This made sense -- I had added a ton of new material during the process of porting the site over and search would obviously have a lot of work to do.
Once I knew where the problem lay, I googled around for "search" and "cron" and found there is a setting in search that is specifically intended to alleviate time-out problems with search and cron. There is an option to limit the number of items that are searched at one time. By trial and error, I found I had to take it all the way down to 10 items (because I had some extremely large nodes that had been added in). Once I made this change, I could have search enabled and manually run cron from http://www.mysite.com/cron.php without errors and I would (finally) get a message that cron had completed successfully.
I then ran it a few times manually and it started to index the site. But then I ran into my other problem. I got impatient when it didn't seem to be moving forward and took the option to "reindex the whole site." I tried that a couple times when nothing seemed to be moving. By then, no matter how many times I ran cron, it never got past "0%" indexed. More googling around convinced me that every time I took the option to reindex the whole site, it started over again on the pages already indexed, which is why I never got any further. I quit selecting that option, set up poor anscron to run every 1 minutes and went on my way doing other things on the site, letting poormanscron do my dirty work in the background.
I went back later and found it had finally gotten to 100%. By that time I could raise the limit of the number of items to search and cron now seems to be working fine.
It's fixed!!
I tried your suggestion of disabling modules and, sure enough, the dba module was interrupting the cron run. I left it disabled and my cron is working again....woohoo! Site is now 100% indexed.
You've made me extremely happy, KarenS (climbs down from the tower, puts down the rifle, and produces a bouque of flowers and a ring)...will you marry me?
Good grief, I forgot how many modules I had installed to "try out"...I think I need to do some house cleaning.
I don't suppose you now have an answer for my email woes?:
http://drupal.org/node/53394
KarenS's new fiancee,
Vince
---------
The hottest Linux distribution? Ubuntu
Talk about Ubuntu Web Servers at:
http://www.ubuntuwebservers.com
Glad to hear it
Glad my long-winded comments were helpful. Not sure about your email problem, I'm still getting Drupal email figured out myself. Enjoyed the flowers, gonna have to give the ring back :-)
I got my email fixed
I don't know if this would help you out in any way, KarenS, but here's the thread:
http://drupal.org/node/54228
Vince
---------
The hottest Linux distribution? Ubuntu
Talk about Ubuntu Web Servers at:
http://www.ubuntuwebservers.com
i have this reindex issue in search too
i am with siteground hosting Drupal 5.1
now my cron runs once a few hours..say 10-12 hrs..
even though i configured it for every hr..
so how do i make it work...
i can do what u said..about the search item indexing to 10 per run...
will this help...
shld i disable some or all modules except search..run the cron..then enable them again ??
pls help..
SecurePages module conflict
After reading the above posts, I decided to try disabling some modules to see which one could be the problem in my case.
A little background, however.
Cron was not running, nor was I getting ANY messages in the logs about cron completing or not completing. When I tried running cron directly, I got redirected to the home page and nothing would happen.
It turned out to be a conflict with the SecurePages module. With that module deactivated, cron ran fine, but when it was activated again, cron wouldn't run and I'd get the home page.
After further testing, I found that it was only a conflict when I tried to access cron with the https: protocol. When I manually entered the cron URL as http://www.site.com/drupal/cron.php, then it worked even with SecurePages active.
Hope this helps someone else find their problem.
- Alan Tutt
http://www.PowerKeysPub.com
More thoughts
After re-reading your original message and my comments, I think it may look like I completely missed your point since I mentioned some things that you had already tried. I guess my thought is that this is some sort of time-out issue -- newsfeeds can take a long time if there a connection problem, for instance -- and it looks like the failure of one thing will keep later things from happening at all. I am not sure, but guessing that you are trying to index both nodes and newsfeeds and the nodes are not getting indexed? If so, and if the search of the nodes is timing out (which would explain the erratic results you are seeing) it may be that cron is never even getting to your nodes. What happens if you turn off the newsfeeds and get your nodes indexed, then turn the newsfeeds back on? If that solves the problem, maybe you can reduce the number of items indexed and increase the frequency. Or maybe you need to see if there are specific newsfeeds that are getting in the way of processing anything else and turn them off (or update them in some way that won't interfere with the search index process).
Don't know if that helps or is even relevant :-)
Determining which newsfeed is the problem
Karen,
The tips you mentioned are great. As of now, we have 71 news feeds that are trying to update on each cron run. Our cron jobs have not completed for quite some time (about 2 weeks now). I'm thinking this is related to specific feeds that cause the timeout. Is there a way to figure out which feeds are causing the problem, other than removing them all, then one by one add them back in.
For example, within aggregator2 I can 'refresh items' for each feed. I need to understand what to look out for when refreshing feeds so i can determine which oneis causing the issue. Any ideas?
Thanks,
-=Vince
Technology White Papers
That's a lot of newsfeeds
My guess is that you are just trying to do too much on one cron run rather than that any one of them is a particular culprit. I would play around with staggering them there aren't so many updating at once. Start with just a few on each cron run until you get to the point where cron will complete, then try increasing it until you figure out the magic number you can safely run at once (then probably back down a bit from that number to allow for intermittant connection problems that might come up).
Yes, it sounds like a time-out issue
(Are you seeing "cron did not complete" in your logs?)
You might try staggering your cron-driven tasks. You also might look at extending your maximum time settings in php..
Hope this helps.
Laura
_____ ____ ___ __ _ _
pingVision, LLC • BlogHer • rare pattern • scattered sunshine
Thanks for the thoughtful replies
The good news is that I got poormanscron to successfully run so that I now have a "cron run completed (via poormans cron)" in my logs.
The bad news is that it didn't updated anything when it ran. I then changed the search settings to run only 10 items and poormanscron to run every 5 minutes...still no luck.
I occasionally do searches, using both search.module and trip_search for terms that should be accessible, but so far nada.
I'm about to try the disabling of modules suggestion
Vince
---------
The hottest Linux distribution? Ubuntu
Talk about Ubuntu Web Servers at:
http://www.ubuntuwebservers.com
Remember, poor mans cron runs only from page loads
If you or someone is not there, clicking away, the cron won't get triggered.
Laura
_____ ____ ___ __ _ _
pingVision, LLC • BlogHer • rare pattern • scattered sunshine
Cron is still not functioning after disabling most modules...
I've tried to implement a lot of the suggestions here, such as lowering the number of items indexed on each cron run within Search. I've also deleted every single aggregator2 feed within the site. After running cron.php from the browser, I still get the message "The last cron run did not complete"
Any other suggestions? Someone mentioned increasing the PHP timeout...does anyone have tips for doing this?
THanks,
-=Vince
Technology White Papers
Go to your settings.php file
Go to your settings.php file (under sites/default or wherever it is). At the bottom of it you will see a list of ini_set commands. Add the following to the list:
ini_set("max_execution_time", 60);
The number at the end is the number of seconds a program can run before timing out. Setting it to 0 means "never timeout" which you should use very carefully, like on a development site where you have control over the server. Try increasing the number at the end until your cron completes.
There is also a way to do this in a .htaccess file (the command is worded slightly differently), but some isps, like mine, have the system set up in a way that will cause a fatal error if you try to control php ini_set from the .htaccess file. As far as I know, you will never go wrong by putting it in the settings.php file.
Hi, can you tell me what the
Hi, can you tell me what the default_socket_timeout means?
My cron script stops with a server error after about 60 seconds. the max_execution_time on my system is set to 30 seconds. So its probably the socket timeout causing problems on my machine because this is set to 60?
I tried the way you described with both settings in the settings.php but no change. Maybe its denied from the provider?
May I also ask you Karen, what the name of the module is you use in your side for the navigation? I like this +- style better than the arrows from the standard settings. You have a very nice webpage.
---
Join the virtual march to stop global warming.
http://www.stopglobalwarming.org/marchers/?142067
Don't know what else to
Don't know what else to suggest on cron, I'm not really a systems expert, I've just figured out a few things over time :-)
As for my site navigation, my theme is basically the chamelon theme with some modifications. I borrowed the +- buttons from the civicspace theme images because I like them, too, and they seem to fit the style of the site.
bad installation of aggregator2 caused cron headaches...
So after disabling just about every module, cron started working again properly...so one by one i started re-enabling modules again. then i got to the aggregator 2 module..once i enabled that, cron no longer completed. this is with 0 feeds in the site.
so i figured i would drop the aggregator2 tables and remove the module...i downloaded a fresh copy of aggregator2, reinstalled, a voila...cron is running fine with many aggre2 feeds...thanks for the help
Technology White Papers
None of these solutions work for me.
I have read through all the descriptions of problems with "Last cron run did not complete."
None describes why any of these things should happen:
1. From my watchdog:
- 1:00 a.m. "Cron run completed."
- 2:00 a.m. "Last cron run did not complete."
2. Run http://www.myexample.com/cron.php in IE and get "The page cannot be displayed" and "Cannot find server."
3. Run the same thing in firefox and get a dialog: "You have chosen to open cron.php. What should Firefox do with this file." If I run the phpinfo.php script, which resides in the same directory, it runs fine. cron.php has execution enabled for all users.
This isn't a crontab problem, because I'm getting bizarre results from manual execution, and at other times everything works just fine, from crontab and manually. It works fine until it doesn't.
I have these directives in setup.php:
ini_set('max_execution_time', 60);
ini_set('memory_limit', '16M');
I don't run the aggregator or dba modules.
I have not gone through the process of manually disabling modules, because I don't believe one can get any definitive information from that. That is, if you turn off a module and cron works, does that mean that module is to blame? Does it mean you just happened to undercut some resource limitation when you turned off that module? Does it mean that, coincidentally, cron.php started working again, but it would have anyway? (This seems to eventually happen, for me.) It seems that different people succeed with different solutions, here. What does that mean?
Since this is an intermittent failure, I wouldn't worry so much except when I see that my users are experiencing failed searches because recent material wasn't indexed, I just don't think that's right.
Any thoughts?
D Icenogle
Different solutions work for
Different solutions work for different people because they have different things that are causing the problems. Your case is something completely different than anything described here. Messages like 'cannot find server' and Firefox's inability to know what to do with a php file point to either a corrupt cron.php file or a serious problem with your server setup.
I am no server expert, so I can provide very little help, but if everything but cron.php is working, I'd assume there is something wrong with that file. Grab a clean copy from the site (I'm not sure where to get it, it will depend on what version you are running).
If that doesn't work, you *do* need to try disabling other modules. It is not unheard of to find out that when one module quits working the problem is really caused by some other module. No, you will not immediately know what caused the problem, but at least you can narrow things down to something like 'cron quits working when I enable the xxx module' which will give you (and others) something to go on for further debugging.
If you have a messed up server setup, you will be seeing problems with other files and modules, too. I can't help you with debugging that, you may need to try a clean install. In fact, if nothing else works, that may be what you need to do anyway.
Different solutions work for
Perhaps, although I don't think you can know that. These problems have cropped up periodically over the last couple of years. I suspect a root cause or a basic fragility in the code, which it appears I'm going to have to track down.
One that crops up periodically, and then goes away? Intermittents are tricky things to diagnose. Just in case, though, I got a fresh copy of cron.php from the 4.7 distribution. "Serious problem with your server setup" is just way too vague to use. I'm not experiencing any other difficulties, so how serious can it be?
Well, you're right, I may just have to do that. However, I installed 4.7 last week, have installed every baselevel of the the beta code, and used 4.6 before that. This problem has cropped up intermittently through all of them. It has also cropped up inexplicably for others. As I said, I'm not experiencing any other difficulties.1 My server update script is three pages long. I do it very carefully. Nor can I arbitrarily just go in and start turning off modules, and disabling functionality for my ~700 users. I'm going to hope that someone has a better idea, for at least a day or two. I'm going to hope that we can spelunk the problems with collective experience, and actually solve them, so that they go away.
1Now I'm going to take that back. There is one other persistent, and possibly related, difficulty which, again, I have seen others have, without a clear explanation or solution. That is, cron-based indexing has never worked. The system never seems to recognize when new material has been added. My site is substantially updated every week, so on a weekly basis I set search to re-index and manually run the cron script. I saw some talk about how PHP in blocks could be a problem, but I never saw a solution to the problem. I have written about it, and I have seen others write about it. I'll fix it if I can, but it would help if others could shed some light on where to begin. I have over 20 years of software engineering experience. Unfortunately, none of it is with PHP or this kind of client/server environment. I'll do what I can do.
Here's another clue that
Here's another clue that pretty much kills the corrupt file/messed up server theory. I have a multisite setup, both of which run the same cron.php. For one site it works, in IE and firefox, and for the other it doesn't work in either.
This doesn't kill the bad module theory, but why would it go south a 2:00 a.m.? I was long past counting sheep, let alone installing modules at 2:00 a.m.
D Icenogle
As I said, I'm not a
As I said, I'm not a technician nor a cron expert, I was just trying to be helpful by giving you some suggestions. I don't have any other ideas, except that you might have more luck posting something on the cron issues page which is where the cron experts will be. The issues page is at http://drupal.org/project/issues. Cron is part of Drupal core, so you would select that as the project.
Thank you
I do appreciate your help...
D Icenogle
search not happening
for new content..search is not happening..
i hit reindex search once by mistake..
since then none of the new content is indexed..and is not searchable..
i am with Siteground Drupal 5.1
and lost..and frustrated...need help..
sitegrnd guys are not helpful...they have a standard reply..upgrade to VPS...for any query :(
I've recently installed DBA
I've recently installed DBA to automate my database backups. Unfortunately Poormanscron no longer runs. That means not only no backups but none of the other functions which it's responsible for. I receive no error messages or the usual notification in the watchdog module. When I manually run cron.php I get a 500 server error. If i disable dba it works fine again. I've tried the solutions mentioned here: decreasing search items, changing php settings, but can't seem to get the two to work together. However, if I disable the automatic updates I can have DBA installed and cron will work. But it's not doing me any good. Should I focus on the ini_set('max_execution_time') and ini_set('memory_limit') settings. What are safe values to use in these settings? The database is 15 mb. I'm running 4.6.4.
Any thoughts?
DBA module the problem I think
I’ve been struggling with a cron problem. It started with my host locking up my site because it was using too much memory. They called it CPU Abuse. When I found it in the morning they restored it for me. I had over 2000 unique visitors that night. I have 3 feeds that seem to get errors but didn’t pay attention to that. I was using poormanscron. After reading the help here I decided to empty the watchdog table, which didn’t help. Calling cron from the command line www.psrunner.com/cron.php didn’t help either. Disabling the DBA module is what solved the problem for me.