Difficulties w/ Boost 6.x1.0-alpha4 - Multi-site, Folder Permissions - _boost_rmdir_rf()
| Project: | Boost |
| Version: | 6.x-1.x-dev |
| Component: | Code |
| Category: | support request |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | closed |
Just installed Boost 6.x1.0-alpha4 and faced some critical difficulties. Everything was working fine until cron run. Every time the cron runs, it deletes all the files (and also internal folders within the cache directory) even though the set minimum cache expiration time was one day.
The difficulty is that I need to run cron by the hour. Another difficulty is that on my VDS server I have to manually create the cache directory and the sub directories within the cache directory and grant them the required permission. Therefore, since the internal folders are getting deleted every time (every hour) the cron runs, I have to again recreate the directories which is causing a lot of headache. Another difficulty is connected with the ownership of the cache folders. By default the ownership is granted to "apache".
I would be grateful to know:
(1). If Cron can be prevented from deleting the internal directory folders within the cache directory every time it runs.
(2). If cron can be prevented from deleting all the pages in the cache every time it runs and instead be forced to delete only the expired pages.
(3)If the directory ownership can be altered in favour of the owner/admin of the server.
Many thanks
#1
We are actively working on boost, there is a reason it's still in alpha. Luckily the latest dev doesn't kill it's self on cron runs anymore. If you are upgrading to the latest dev, be sure to change the htaccess rewrite rules; as boost will not work correctly without that. This #453426: Merge Cache Static into boost - Create GUI for database operations (database table) is also in the works, which would allow for a more refined approach to clearing the cache.
#2
Thank you very much for responding and for all your good efforts to give us all a great module. It is extremely exciting to know that you are in the process of merging Boost with Cache Static. I shall certainly try the latest DEV module.My best wishes to you...
Thanks again
#3
do you know how to apply patches? if you do run the latest dev with these patches
#326683: Increasing cache lifetime beyond presets
#443736: Smarter boost_cache_expire(). Path Changes & Wildcard Support
#478144: files purged from cache way before min cache lifetime, ignoring disabled purging
Should fix all the cache issues.
Also Please let me know if
(3)If the directory ownership can be altered in favour of the owner/admin of the serveris still an issue with this latest dev.#4
I will insert the patches and report back in the next 24 hours.
Thanks
#5
I inserted the following patches :
#326683: Increasing cache lifetime beyond presets
#478144: files purged from cache way before min cache lifetime, ignoring disabled purging, into the latest DEV version.
I did not insert the patch: #443736: Smarter boost_cache_expire(). Path Changes & Wildcard Support, as I do not use "pathauto" or "pathredirect" modules.
The module is working well (pages are getting cached and served as desired and cron run is not nuking the cache folder contents abruptly) except for a few issues which I am stating below:
(1). The Admin Status Report continues to display the message "cache file paths not set to default......" even after setting the default path and even when the module is working.
(2). In case of multi sites, hitting the configure button on the performance page of any one site is deleting the entire cache contents of all the sites.
(3). In case of multi sites, hitting the clear cache button in any one site also clears the entire contents of the cache folder including cache files of other sites.
(4). The default ownership rights of the cache directory and its sub directories is still an issue, particularly in cases like mine (and of other similar cases), as it prevents me from granting or altering permission to the cache directory and its sub directories with a visual interface and instead use shell commands every time the cache is flushed.
The ideal situation for me would be to have cron (or the clear cache button) delete only the files and not the entire directory and its sub directories.
I hope this feedback will be of some help.
Many thanks again to you Mikey for all the hard work.
best regards
#6
Issues 2-4 are all related, all dealing with _boost_rmdir_rf(). It returns by doing this
return ($empty && @rmdir($dirname));. There needs to be an if statement in there that makes sure it doesn't del the top directory.The directory structure inside your cache dir is it something close to this, correct?
cache/www.example.com/...cache/www.example.net/...
cache/new.example.com/...
cache/new.example.net/...
If it doesn't look like that, please tell me what it looks like.
Issue 1 should be fixed with attached patch; please report back if it works.
#7
The directory structure is exactly similar to your example.
As regards issue (1) I shall use your patch (boost-477848) and report back thereafter.
thanks
#8
For issues 2-4 here is a patch that removes the rmdir() call. It's a temp fix for your situation; need to code a better one.
#9
Mikey your patch (boost-477848.patch) solves issue (1) above.
Thanks for the new patch (boost-477848-2-4).
I shall insert it soon and report back.
Edit: On testing this patch on my production server I am getting a blank page.
#10
patch from #6 committed.
#11
Mikey, I attempted to insert patch (boost-477848-2-4) but could not locate the stated function _boost_rmdir_rf($dirname, $call in the boost module file.
Instead found the function _boost_rmdir_rf($dirname, $flush = TRUE) {
This is for your information
#12
Mikey, I just noticed some strange behaviour. On testing this patch on my production server I am getting a blank page while on my local server this does not happen.
#13
this just showed up in the issue queue, give it a shot[#478644]; also i screwed up that stupid install file again... #478674: missing semicolon. Guess I'm pushing patches out a little too hard trying to get a beta ready.
#14
Sorry Mikey, for bothering you once again. Just noticed that the sitemap.xml files are not getting cached. This could be an important issue for large sites. Just thought, I should bring it to your notice.
#15
That was kinda a bug, not a feature, that is now fixed until I can send out the proper mime that goes along with xml #182686: Static Feed Caching. xmlsitemap has its own file cache so the server doesn't kill it's self on every request. If you must have this feature back, here is the commit that changed it around http://drupal.org/cvs?commit=219120.
#16
Mikey, I inserted patch (#478306) but still no luck with caching sitemap.xml files.
#17
if you want sitemap cached by boost, here is the online diff
http://cvs.drupal.org/viewvc.py/drupal/contributions/modules/boost/boost...
Change This
if (preg_match('!/feed$!', $normal_path) || preg_match('!\.xml$!', $normal_path))To This
if ($normal_path == 'rss.xml' || preg_match('!/feed$!', $normal_path))You do understand that by applying this, it will most likely work; but it doesn't follow standards.
#18
BTW does boost work as you would expect with the patches you applied (#4)?
#19
Mikey, thanks again.
I shall apply the latest patch for the sitemap (stated in #17) and provide you a feed back tommorrow (3rd June), once it is done.
With the patches that you have already provided, boost does work Ok, with some minor issues here and there.
However, I was just wondering whether hitting the "clear cache" button in the performance page of any one site or hitting the "Save Configuration" button (in the performance page) for reconfiguring the module (in a multi site environment) would still delete the cache files of the other site?
If it does not and if the patch for the sitemap files work Ok (after I insert the patches), in that case I can confirm that Boost Dev version is working exactly as I need it to.
However, I take this opportunity to express my gratefulness to you for all your help.
Best wishes & best regards
#20
It depends on how drupal is setup. If its using 1 database in a multi-site environment then you need to apply this patch and change the settings so boost doesn't clear the wrong cache #478644: Cache file path shouldn't be stored in the DB by default (odds are that you do not need to do this). Also as stated in #5, #6; the above issues is related to problems with _boost_rmdir_rf(). #8 should solve the symlink issue, but it might not solve the cache being cleared for all sites in a multi-site setup. Please let me know, because if this is the case then I need to make a smarter _boost_rmdir_rf() function (my guess is I need to make that function smarter).
Also please report any as this is what makes boost better. Thanks for all your testing!
#21
If one of your sites contains the name of another site in your multi-site setup then this could be a hard to slove issue.
www.example.com => site awww.example.com/drupal => site b
Clearing 'site a' will also clear 'site b' because it appears to be a normal subdir of 'site a'. I don't even know if this multi-site layout is possible; so hopefully the above situation will never happen.
#22
After carefully reading Setup of /sites directory for multi-site it appears that I need to take into account subdirs; like the above situation. Set a file called
.boostin the root of the site's cache dir so that _boost_rmdir_rf() knows it's in a different site and shouldn't clear it.<?phpif (file_exists($dirname . '/.boost')) {
//figure it out
}
?>
Need to pass along something so it knows if it was the original call or a recursive one, which will fix the symlink issue correctly; and make the above easier to do.
#23
Caching Sitemap:
Mikey, I did apply patch (477848_0) but sitemap files still does not get cached. I am using 6x.1 Dev version of the sitemap module.
Multisite:
As regards, multisite - my multi site setup is with multiple databases, one for each site (path is similar to your example stated in #6).
I would love to see the "clear cache" or "configure" buttons (both on the performance page), when hit, not delete the cache files of the other existing sites.
Minor Issues:
I have noticed that every hour when cron runs it does not delete all the files in the cache directory but destroys the sub directories within the cache directory where there are no files and attempts to rebuild them, whenever the next visitor visits a page type. In my server situation, the sub directories get created by boost automatically but the files are not getting written/cached as the directories do not have the required write permission and since the ownership of the sub directories are given to "apache" by default, granting them the required write permission has to be done by me manually using shell commands, in my server environment.
I would prefer to have boost delete the stale and expired files on cron run but not the cache directory (folder) itself or its sub directories (folders), which would also ensure that the permissions once granted to the folders can continue permanently while the files within them keep getting deleted on expiry.
Thanks
-------------------------------------------------------------------------------------------------------------------------------------
#20: You are right, at present, in a multi site situation, hitting the "clear cache" or the "configure" button in any one site clears the cache files of all the sites.
#24
I'm going to put the patches that are queued up in, and then work on the multi-site issue as well as an option to not delete the directories; otherwise I'm writing patches for the patches, which is something I do not like to do.
#25
#26
Is there anything we can do to help on the xmlsitemap end of this?
#27
@ Mikey: Thanks once again
@ Dave: Thanks for your offer to help
#28
@Dave Reid
If you want to, come up with some rewrite rules for #182686: Static Feed Caching. Boost can handle xml if you allow it to, and don't mind the wrong mime type being sent out. Main issue is the mime type, I don't want it in boost unless it meets standards. I've been busy with other issues, but it is something that wouldn't be that hard to do.
#29
#30
This patch allows the admin to disable folder deletion. It also prevents the root folder of the cache from being cleared. Won't delete other multi-sites caches. It's patched against the latest dev (Jun-06).
#31
Thank you very much!
Best Regards
#32
@froggie
Does the patch work as expected? Are there any other issues?
#33
Yep, it sure does! Thanks again!
Sorry for the delay in my reply. It's just early morning now here where I live.
The only issue that remains for me now is caching the XML sitemap.
To the best of my knowledge, XML Sitemap Module now has a feature to cache the sitemap pages. However, it uses the default drupal core caching system.
I was just wondering if it is possible for Boost to not disable the Drupal core caching entirely and let the drupal core handle the caching of xml sitemap pages while Boost takes care of the caching of all other pages.
Thanks
#34
#204174: Work with page cache, not instead of it
#182686: Static Feed Caching
Both of these are issues because boost is set to explicitly not work with them. Next step here is to re-enable it and see what happens. You should watch the above issues for any patches in regards to your requests. Thanks for your testing and input!
#35
committed.
check out the above 2 issues, they have patches now.
#36
@Froggle/33: XML sitemap does *not* use Drupal's built-in caching system. It generates files onto sites/example.com/files/xmlsitemap and then uses file_transfer() on the 'sitemap' Drupal paths to transfer the file content to output.
#37
Automatically closed -- issue fixed for 2 weeks with no activity.