By unleash on

hi there good evening dear drupalers,

i am running drupal-commons version 6.25
i have installed it local and now i am trying to go on a server with all the local stuff. Note; on my local - lampp aill went nicer, i have had no problems at all. Well all went very beautiful!!

and now i have to face another -- a brandnew image.
i tried to configure my imagecache..installed n enabled the module..now i created a cck for images. and set its teaer mode display as thumbnail preset linked to the node..

now wen m tryn 2 add content to it, its not displaying the content on the node page/..

btw if i look at the settings - i find very interesting things... there are no previews visible

http://schulcenter.net/?q=admin/build/imagecache/Status_picture

http://schulcenter.net/?q=sites/default/files/imagecache/Status_picture/...

see the page-heading: Edit preset: user_picture_meta

btw- does it have any IMPACT on image uplaoding and finding the images...what kind of configuration exists regarding the URL-aliasing and rewrite

i heard that this thing is pretty important to many many drupal-features and functions.
Drupal provides complete control over URLs through aliasing, which is often used to make URLs more readable
or easy to remember. For example, the alias 'about' may be mapped onto the post at the system path 'node/1',
creating a more meaningful URL. Each system path can have multiple aliases.

love to hear from you

⋅ Categories: Drupal 6.x

Comments

dman’s picture

dman commented

It's hard to tell what your actual question is here.

Most likely - if your imagecache problems started when you transferred to a new server - is that the *permissions* on your /files/ directory are not writable by the web user. This means that images you have generated already will be readable, but new ones cannot be written. Check that.

Also consider checking your site status under the 'reports' tab and see the Checklist for migrating to a new server. It's possible that some requirements may be different there. It's even possible that your server doesn't support (or has not enabled) image generation at all.

Secondly, if you are concerned about URLs, you should turn on 'clean URLs' , http://drupal.org/getting-started/clean-urls
You haven't done this - which is why you see the ?q= links.
Clean URLs etc is not considered extremely important for generated image thumbnails, and is not related to your difficulties. Although once your image generation is working properly again, the image paths will be stable 'normal' URLs like http://schulcenter.net/sites/default/files/imagecache/Status_picture/ima...
... and this has some advantages for performance.

.dan. is the New Zealand Drupal Developer working on Government Web Standards

unleash’s picture

unleash commented

hello dear Dman - many thanks for the quick answer - i will follow your advices 

Most likely - if your imagecache problems started when you transferred to a new server - is that the *permissions* on your /files/ directory are not writable by the web user. This means that images you have generated already will be readable, but new ones cannot be written. Check that.

Also consider checking your site status under the 'reports' tab and see the Checklist for migrating to a new server. It's possible that some requirements may be different there. It's even possible that your server doesn't support (or has not enabled) image generation at all.

weill it sounds very very convincing what you say about the permissions and the user- on the webserver.
i think that these things a re the most important things - i have to keep in mind: what do i have to take care for - especially if i have troubles with

- file ownership
- permissinos

if i set up a drupal-commons version 6.25 locally on my machine at home - which is a openSuse 12.1 - everything runs nice. All goes smoothley. especially if i want to upload files - no problem here-

but if i wantto do the same - /uploading pictures - on the version that i either installed directly on the webserver or portet over to the webserver i have terrible issues.
i it is not possivble to upload any file - i have trouvbles and get error messages....

regarding the temp-directory (wich is complained to b emissing)
regarding the permissions on site/default/files

well some words regarding the server and the circumstanceds and conditions it runs with:
my serveradmin works with the setgit bit

I run a linux sever that is administered by a friend of me. He prepares the chown and permissins - with the setgit-bit to on.

the tremendous troubles i run into i have described here: http://superuser.com/questions/382792/setgid-bit-always-unset-when-chang...

on a sitenote: Well the trouble is that i get lost the setgitbit every time when i touch the permissions with FileZilla. That is the terrible issue! Note - at the moment i do not have a access to the server wit WinSCP - winscp can set the Setgit. Filezilla cannot do this at all!

well the question is: what do i have to take care for if porging over the drupal. both versions

.a a drupal commons version 6.25 and
.b a drupal version 7.12

do i have to set up a tmp directory again note: if i create the whole installation here on my local server then my assumption is, that all things have been done propperly

dman’s picture

dman commented

If the site-status report says you have problems with the temp directory, that MUST get fixed, because it's a serious requirement.
Settings for this can be found in the configurations admin section.

If you need to have different settings for this between your local development and the remote copy, you can set those settings differently in your settings.php files. But it's usually easier to just make sure that both systems are mostly similar. - eg, both use /tmp as the temp dir.

setgid may be helpful when setting up shared development environments, but it's not necessary unless you and your sysadmin also know how to administer it. I would advise not using setgid if you cannot manipulate it correctly. Me, I often put g+ws on the files/ directory - by setting the group permissions "sticky" it means that files created in that directory also remember that they should be group-writable. But I can't explain that better than the online guides for file permissions can - so you'd be best to look for some of those.

If you use filezilla and the permissions go across to the destination site when you upload, then that means that filezilla CAN set the permissions for you - you just need to find the right way to do it.
A short work-around may just be to set the permissions correctly on your local development filesystem (eg, set *just* the files directory world-writable) so that when you upload, that folder will be writable for the webserver. Assuming that permissions are being transferred identically.

I'm surprised that you are running winscp on openSuse... But any client that can talk either scp or sftp has the ability to work with permissions. even plain FTP clients usually can too.

.dan. is the New Zealand Drupal Developer working on Government Web Standards

unleash’s picture

unleash commented

hello DMan

- many many thanks for the hints.

All your advices are valuable assets of drupal-knowledge. i will follow your advices! BTW: on OpenSuse 12.1 i do not have WinSCP - but i had that great tool on a Windows-installation some months ago! Guess that FileZilla and WinSCP are a bit different in the handling of SETGiT-Bit. At least this discussion makes me assuming this:
http://superuser.com/questions/382792/setgid-bit-always-unset-when-chang...

Another - perhaps the last step towards a first solution might be the correct setting of a tmp-directory.

Dman, i really do not know where to put the tmp-directory

i am relly a bit clueless

have created (with FileZilla) one tmp-directory here:

/home/vhost/WWW/schulcenter.net/

after that i have done it here:
/home/vhost/WWW/schulcenter.net/sites/default/files

nothing helps - even if i put the permissions to 2777

i still get complains in the admin-setting:

http://schulcenter.net/?q=admin/settings/file-system

what the directory /tmp does not exist.

what should i do now!?

Does it have to do something to do that i have created the directories with FileZilla?

look forward to hear from you

greetings
unleash

BTW if you want to have a closer look at the backend i can provide you with the login-data:

login-name: xx
passwd: xy

sent to you a pm... - i would appreciate any and all help!!!

unleash’s picture

unleash commented

first positive results...

after digging in the deep of the site . i guess that i have first results... see an image - (note a default-image that is not (!!!) uploaded by me but anyway.. i can see this one
http://schulcenter.net/?q=sites/default/files/imagecache/Status_picture/...

well i do not know what else is needed - but afaik i come a very very long way to this first results:

i run the drupal commons version 6.25 fresh installation at http://www.schulcenter.net

i got various errors - since weeks now - and have no glue what goes on here.

on my local server on opensuse 12.1 all runs nice without any issue. after porting over and migrating the site to the webserver i face the error that i cannot upload any image.

well i see various little issues & errors that come along ´with this behaviour:

several folders and files were created by the system.

a css-folder with the owner www-run
a js-folder with the owner www-run

several files called

imagecache_sample.png png-image from fallback

After having set up the paths so that nothing else have been complained

http://schulcenter.net/?q=admin/settings/file-system

i have no glue why i am not able to upload any images?!

do you have any idea why i continue to get errors - and cannot upload any image...?

well now i can see this one here

http://schulcenter.net/?q=sites/default/files/imagecache/Status_picture/...

how to continue: what should i test / configure next!?

dman’s picture

dman commented

I used the login you sent me to have a look around.
There were still some complaints at http://schulcenter.net/?q=admin/reports/status - I suggested you start by looking there.
- cron was not set up and had not run
- the database has some updates pending.
Also some warnings about the memory that image GD processing will take.

Any of those could be causing problems for you and the warnings should be taken care of. All admin pages were showing the warning and linking to the report page, so it doesn't help to ignore them.

When I looked at the normal imagecache previews at /admin/build/imagecache/Status_picture I'm seeing a preview.

Looking at your pages so far...
only content types with *images* attached will dispaly the image and use the imagecache preset.
Your page http://schulcenter.net/?q=document/new-image-new-document had a jpeg that was attached as a *file* and that means you can download the file again, but it's not displayed as an image.

Instead, I found that the 'Notice' content type does support having a proper 'image' attached to it.
But when trying to do that, I got a message "The selected file puzzle-piece.jpg could not be uploaded. The file is not a known image format."
... which was odd.

Instead, I looked at your reports (I told you to "Also consider checking your site status under the 'reports' tab ")
http://schulcenter.net/?q=admin/reports/dblog
There is a lot of noise there. red means bad, ok?
If you are having problems, and there are red warning messages, those two things may be related.
You need to fix the errors before expecting things to work right.

The big bad error I'm seeing is

opendir(*) [function.opendir]: failed to open dir: Too many open files in /home/vhost/WWW/schulcenter.net/includes/file.inc on line 963.

Now, I've not encountered that error before with Drupal on any system, but it does seem like it could make *anything* on your site break - especially things that work with files - but also anything that reads files. So you've got to fix this.

Or rather, your sysadmin has to fix this. It's probably not something you can do from your end - talk to your host and see what they can tell you.

.dan. is the New Zealand Drupal Developer working on Government Web Standards

dman’s picture

dman commented

http://www.karakas-online.de/forum/viewtopic.php?t=9834

.dan. is the New Zealand Drupal Developer working on Government Web Standards

unleash’s picture

unleash commented

hello dear dman ;-)

just came back from a little walk - after spending all day long in front of the computer.

Dan - you deserve a monster congratulation. many many thanks for having a closer look.

the issues regarding opendir - too many files open is also apperaring when you try to install drupal-commons 6.2x on the server. Note: the server is a bit aged .. It is planned to substitute this with a new one - with much much better hardware. So that these kind of issues do not appear any more.

i know that the opendir - issues and the total amount of filehandle (of Apache) is related to the hardware - especially to the Ammount of RAM we have. This max-file-handle-amount might show some mission critical factor.

btw: i guessed that this issue - with the filehandles - was only related with the installation-process. Therefore i decided to install the system locally and migrate it afterwards.

well - i will have some talks with the server-admin and ask him to increase the amount of max-file-handles...

again - many many thanks to you for having a look at the backend.

i come back and report all my findings

greetings

unleash’s picture

unleash commented

hello dear dman - again me _ with a short note...

i mused a bit about the findings - well it is really a big thing. There is a LOT OF NOISE in the repots...

i have more than 1200 reports of such a message opendir(profiles/drupal_commons/modules/acquia/acquia_search/apachesolr/drush)

BTW - seems to be some thing drush here involved, or am i total wrong here. Note;: i do not have installed any drush...but i am willing to do so!!
Besides this thing i have no clue why we get such a big amount of open filehandles the Apache is faced with!?

it is difficult and hard to understand - but anyway i will contact the serveradmin - and we have to look for a solution - ie. to increase the number of filehandles and then (afterwards
have a closer look at the options that we have with this fix)

i come back and report all my findings.
greetings.
unleash

see one of the reported issues - note i have more than 1200 such reports collected only in 10 hours. It seems that here something is very very brocken...

http://schulcenter.net/?q=admin/reports

The dblog module monitors your website, capturing system events in a log to be reviewed by an authorized individual at a later time. The dblog log is simply a list of recorded events containing usage data, performance data, errors, warnings and operational information. It is vital to check the dblog report on a regular basis as it is often the only way to tell what is going on.


Details
Type php
Date Sunday, 18 March, 2012 - 21:48
User martin
Location http://schulcenter.net/?q=admin/reports/dblog
Referrer
Message opendir(profiles/drupal_commons/modules/acquia/acquia_search/apachesolr/drush) [function.opendir]: failed to open dir: Too many open files in /home/vhost/WWW/schulcenter.net/includes/file.inc on line 963.
Severity error
Hostname 188.99.184.201
Operations

dman’s picture

dman commented

Don't worry about the actual number, but it is a thing that is happening all the time - any time a file is read. As Drupal core reads a LOT of files each time it bootstraps, you'll get a lot of these notices.
The difference between 50 errors and 10,000 is not the point. The difference between 0 and 1 is what needs action.

It's nothing to do with drush. It's just about the resources that are available to the server slice that you are working on.
I've not worked with this problem before and can only direct you to the helpful explanation I found when searching. Linked above.

Option A - increase the limit to make things work again. Easy
Option B - find why that limit is being hit at all - it's not normal. Instructions are in that link. It may be a side-effect from other things running on that server also, any process there shares the pool of file handles.

.dan. is the New Zealand Drupal Developer working on Government Web Standards

unleash’s picture

unleash commented

hello dear dman

many many thanks for the answer - and your ideas.

i mused a bit about the findings - well it is really a big thing. There is a LOT OF NOISE in the repots...
i have more than 1200 reports of such a message opendir(profiles/drupal_commons/modules/acquia/acquia_search/apachesolr/drush)
during 12 hours of Sunday.

Well what can be done here? - besides the mail and the talks that i will have with my Serveradmin - today and
in the next days i wonder why i have these errors. I want to see if there is a relationship between the amount of allowed filehandles and the occurance of the error.

What about some investigations and a little test:

Note: on my local webserver (that runs Apache 2.2) all goes nice: no problem with installation no problem with all actions i take during the running of the site.
All jobs and works are proecessed without any issues.

Well - is it possible that the webserver is tooo weak - it has not too much memory. and it is 6 year old.

My question is - what causes the open filehandles!?

does the openFile-handles that are allowed by Apache are the root of my lack of processing image-upload.

And subsequently: At which rate of allowed Apache FileHandles i can get (reproduce and verify) the behaviour that the webserver show.

Beside the fact that the webserver runs Apache 1.3.8 and my local Apache is a Version 2.2 (on OpenSuse 12.1) i want to do some tests.

I want to lower the rate of allowed Apache FileHandles to see at which rate the system (the image upload) breaks?

if we can see that there is a relationship between allowed Apache FileHandles and errors in Image uplaod.

What do you think about this idea!? Now i have to have a closer look where i can configure the amount of allowed ApachefileHandles

dman’s picture

dman commented

Don't worry about the actual number, you'll get a lot of these notices.

I've not worked with this problem before and can only direct you to the helpful explanation I found when searching. Linked above.

As there are many differences between your test setup and the live setup, and this is an OS-level problem, the only debugging that can be done is on the real site. You will not be able to replicate usefully unless you have a full clone of the live site.

.dan. is the New Zealand Drupal Developer working on Government Web Standards

unleash’s picture

unleash commented

hello Dman

many thanks for the answer - and all your help!

well i guess that you are right - i have to debug the live-site and the real-webserver -that runs Apache 1.3.8
btw: if i can find the reason that causes so many open filehandles... that opens so many files at one time!?

i will contact my serveradmin and ask him to do a first time workaround with decreasing the limits of ApacheFileHandles...

unleash’s picture

unleash commented

new Update one a sitenote...

i want to mention that i - for the purpose of some tests have put two **sites onto the server**

http://schulcenter.net - (for sure Version Drupal-Commons 6.25)
http://schulcenter.org - (probably Version Drupal-Commons 6.23)

and they run on the same database ;-) that means the connect to the exactly same tables!!!

guess that that can cause the open File handles
what do you think !? Love to hear from you

well i guess that i have to switch off one site. ;-)

greetings (end of update)

unleash’s picture

unleash commented

hello dear all -after doing a search in the forums i stumbled upon this threat

http://drupal.org/node/1065864

here a guy is happy with a given solution:;

Thanks so much with your help parsing this error, mjolk. After reading your post I did:

chmod a+r includes/

and the error disappeared.

-Beth

see the General Solution Posted by mjolk on June 14, 2011 at 4:23pm
in the same thread http://drupal.org/node/1065864

i'm responding due to the very high rank on Google of this common question (and to help those that have misinterpreted community support or misplaced their manners (read, digest http://drupal.org/forum-posting)).
Programmer-by-day or not, make a reasonable attempt to parse the given warning or message. It may turn out that the solution is not even Drupal-specific in scope. When in doubt, respond politely and don't abuse grammar out of frustration.
That aside, the error/warning messages usually provide better starting information for troubleshooting than a search engine.

For instance:* Warning: opendir(/tmp/update-cache) failed to open dir: Permission denied in DrupalLocalStreamWrapper->dir_opendir()...blah blah blah....

Suggests an error with /tmp on the local linux filesystem. What does "permission denied" mean in any context? Warnings such as these can be easy traced. To elaborate:
Warning: opendir(sites/all/modules/captcha): failed to open dir: Permission denied in file_scan_directory() (line 1975 of /var/www/example.org/blog/includes/file.inc).
Warning: opendir(sites/all/modules/admin_menu): failed to open dir: Permission denied in file_scan_directory() (line 1975 of /var/www/example.org/blog/includes/file.inc).

Cancel out some of the common factors on the lines:
Warning: opendir(sites/all/modules/captcha)
Warning: opendir(sites/all/modules/admin_menu)

Looks like a warning for "opendir" on sites/all/modules, with captcha and admin_menu being data points.
Let's not look at the line of code from the file that was being reported - variables can be annoying to chase and who says you're sold enough on this CMS-du-jour to spend the time tracking errors in code?

Anyway, Drupal seems to complain about various modules that we assume should work. Oh, and Drupal is telling you that it doesn't have permissions to do something to the file.

Start with the easiest and least insecure solution, telling your computer to let your webserver read a file.
In other words, our solution:
-Check that Apache (or whatever server) can read the files in the directory sites/all/modules. This is as simple as setting the file owner as your webserver user or putting your webserver in a group with permissions to read those files.

I hit this problem when I changed user groups on my seldom-used hosting box. I solved this by giving the Drupal directories to the group "drupalweb" with the group members of myself and www-data.

well this may not help in my solution - i guess that my way is a harder one.
i have to debug the whole server... and all the configurations have to be overhauled -..

one single one might cause all the damned issues...

unleash’s picture

unleash commented

after taking off the second site i have no openDir-errors any more..

now i have only a few of the following ones: 

http://schulcenter.net/?q=admin/reports/event/21045

Details
Type	php
Date	Wednesday, 21 March, 2012 - 19:35
User	martin
Location	http://schulcenter.net/?q=filefield/ahah/notice/field_content_images/0
Referrer	http://schulcenter.net/?q=node/add/notice
Message	is_file() [<a href='function.is-file'>function.is-file</a>]: open_basedir restriction in effect.
 File(/usr/local/httpd/phptmp/phpmEkeBj) is not within the allowed path(s): (/home/vhost/WWW/schulcenter.net/:/usr/local/httpd/icons/) 
in /home/vhost/WWW/schulcenter.net/includes/image.inc on line 118.
Severity	error
Hostname	92.74.22.238

and of course the issue with the image upload...