I'm still getting the internal server error when I click on a link to files stored in the private directory.

Here's what I've tried:
1. restoring from a backup to a time before I tried switching to drupal's private file system.
2. upgrading to private_download 6.x-1.2
3. uninstalling, deleting the private_download module and private directory, and reinstalling
4. disabling the content_access module to make sure it wasn't conflicting

And I'm still getting the same 500 Internal Server Error. I'm getting the same error in both of the ways (below) I'm trying to link to the uploaded file. I'm trying both ways because I'm not sure which is correct. But the result is the same either way.
/drupal/system/files/private/filename.pdf
/drupal/system/files/filename.pdf

Interestingly, when I tested the links before I re-uploaded the files, on the first link, I got a normal drupal "Page Not Found" error. But with the second link format, even without the file being uploaded, I was getting that Internal Server Error.

The only other variable I can figure may be affecting my results is the fact that I had tried the private_upload module first. But I have deleted that module now. (it didn't provide an uninstall option)

Is there anything else I should be checking?

Comments

johnhanley’s picture

Here's a bunch of random questions.

1) Are clean URLs working correctly otherwise?
2) Is your file system set for public download?
3) Can you access a file from /sites/default/files?
4) Does /drupal/sites/default/files/private/filename.pdf correctly redirect to /drupal/system/files/private/filename.pdf (before the error)?
5) With Private Download disabled is /drupal/sites/default/files/private/filename.pdf correctly inaccessible (protected by the .htaccess)?
6) What's your server setup?

Bottom line, we need to determine if the error is Private Download related or if it's something lower level and more fundamentally wrong with your web server configuration. I suspect it's the latter, but I (or perhaps another user) will attempt to at least point you in the right direction.

John

BTW, thanks for creating a separate issue. :-)

pltmdude’s picture

Thanks John. In answer to your questions:

1. yes, clean urls are working beautifully
2. file system is set to public, but before I found private_download, I had briefly tried setting it to private, and then back to public. I saw the warning that it could produce unexpected results, so, when I started see these server errors, I reverted to a backup that was before I switched to private. So I think I've undone those problems. ???
3. yes, I can access files from sites/default/files
4. yes, it does redirect to the system address if I put in the direct address.
5. with private_download disabled, I CAN access the file using both the /system address and the /files/private address. Here's the contents of the .htaccess file:

<IfModule mod_rewrite.c>
  RewriteEngine on
  RewriteBase /drupal/system/files/private
  RewriteRule ^(.*)$ $1 [L,R=301]
</IfModule>

6. Server setup:
Shared server running Apache
MySQL database 5.1.39
PHP 5.2.14
Drupal 6.19
Status report confirms the file system is writable (public download method)
[what else do you need to know?]

Thank you for these questions. I hope the answers help.

johnhanley’s picture

Hmm, all your responses are as expected. Your web server configuration is also pretty standard although you left out the OS (assumption is some flavor of Linux). I see you're using the default .htaccess content so there's no information to glean there either.

Since you're on a shared server you probably don't have access to the error logs. However even if you did they might not reveal anything because 500 Internal Server errors happen at a lower level before the web server has a chance to write to the log.

Switching from public to private and then back to public can be problematic, but mostly because it messes with pre-existing file entry paths in the database. It shouldn't matter with new files.

I suppose you could try creating a separate clean install and only enabled Private Download. This will either succeed (and therefore suggest there's a problem with your primary application configuration) or fail (and perhaps help to isolate the problem.)

Other than that I'm honestly stumped. The module should work as described without any fuss (especially when judging it by the limited number of bug reports and support issues.) Perhaps someone else can jump in with a comment or suggestion. Anyone?

pltmdude’s picture

Thank you so much for your time/help, John.

I found the OS: Debian Linux.

I think I found the error log. Here are the 2 lines that refer to the error I'm getting (time stamp and client IP address removed):

malformed header from script. Bad header=<p>: php-wrapper.fcgi, referer: http://domain/drupal/node
File does not exist: /home/username/domain/internal_error.html, referer: http://domain/drupal/node

Does that help? I'm new to drupal, so I don't have much experience to base this on, but I agree that if this was an issue with this module, then this issue would have come up before.

But I'd love to use this module, so I'd appreciate any help people can provide to point me in the right direction of what to investigate next.

Oh, and I'm not ignoring your suggestion to do a separate clean install. But I'm not sure what it would tell me if it worked. I've put too much time into configuring this install to rebuild based on that clean install, and don't have enough experience to know what short-cuts I could take to reconfigure a new install.

johnhanley’s picture

That's actually very helpful, but unfortunately still not conclusive.

The error message suggests there's a problem with the "file headers" attributes. This could be due to case-sensitivity or possibly the incorrect number of line breaks after each entry.

Sometimes the only way to resolve this kind of problem is to get some sort of sniffer utility application that examines the page header data contents and formatting. This can be tedious and time-consuming. I'm admittedly not an expert on this sort of thing and I try to avoid it like the plague (for obvious reasons!)

Before going any further you might try opening a trouble ticket with your web host. Tell them you're getting an 500 Internal Server error and include the error message from the log. I still think it's a server configuration issue. They might not have an answer, but could provide a clue.

Another suggestion is to Google the name of your web host along with "malformed header from script" "Bad header". You might get some hits of people talking about the problem and possible resolutions.

There's gotta be a solution out there and you're likely not the first person to have encountered this problem. I have certainly dealt with my fair share of mysteries like this and persistence usually wins out. :-)

pltmdude’s picture

Here's what I've found.
The documentation on my web host (Dreamhost) indicates that a 500 Internal Server Error probably has something to do with permissions. So I did some checking, and everything seems to be ok related to private_download.

The file "php-wrapper.fcgi" is a file I created, which is outside of drupal, and allows me to have a customized php.ini so that I can edit the memory limits and file upload size limits. So it might be that there is something unique about my host setup that doesn't work with this module.

Anyway, unless somebody has any other ideas, I think I'll have to abandon this and try other solutions to making some files private.

John, thanks for all of your help.

johnhanley’s picture

php-wrapper.fcgi sounds like it could be the culprit. Just for giggles why not try temporarily disabling it...?

Druid’s picture

See http://www.catskilltech.com/freeSW/SMF/faqs/index.html > "500 (Internal Server) Errors" and "Proper Permissions" to see if any of those things are getting you on your particular server.

Druid’s picture

(remove dupe). Stupid comment system.

pltmdude’s picture

Thank you both.

I tried removing the php-wrapper.fcgi file and then EVERYTHING gets the internal server error.

And I didn't find anything new on the catskilltech site. My problem with checking permissions on file and directories is to figure out which files/directories might be connected to this module and process. As far as I can tell, the permissions are fine. I tried changing some of the permissions on files and directories that I know are related, but no change. And I don't want to go any further into changing permissions on other files because I'm afraid I could make a mistake and not get it set back to what it was before. (Perhaps a lame excuse, but as a beginner, I'm not confident enough that I can fix the mistakes I make.) Are there specific files or directories you want me to list the permission settings for?

Druid’s picture

You'll just have to slog through all your files and permissions. I'd concentrate on the "private directory" and its files, if your Drupal site works fine everywhere else. Are you sure there's nothing in the way of an error message specifying exactly what the server doesn't like? It might appear on screen (do a View > Page source) or be in a file such as error_log. Have you checked if your host bans "world writable" directories or files (and of course, do you have any)? Do you have any .htaccess entries restricting access to this directory? A badly formed .htaccess command could cause a 500 error, and if the .htaccess file is in the private directory, it would only be triggered when you actually go into that directory. If these are script files (e.g., PHP) you're linking to, are they valid (no blank lines at the beginning or end)?

pltmdude’s picture

Thanks, Druid.

The private directory is is set to 755, and the files contained inside are pdf files that are set to 644.

I posted the error message above at #4, but you might not have noticed it because I just now figured out how to show it as code. And the contents of my htaccess is shown in #2, which is now showing its entirety since I specified it as code. That's the .htaccess file and contents that were generated by the module.

Druid’s picture

Well, it looks like you have two problems. First, it's trying to send an HTTP header starting with, apparently, a "paragraph" tag. It should be something like "Content-type: text/html" (plus a blank line), I would think. This fcgi is "Fast CGI PHP"? Perhaps you need to explicitly send the header, as you would with a CGI Perl program? You might want to talk with your host about how to properly use this (is there a reason you can't use your server's ordinary PHP?).

Second, it appears that an "internal_error.html" file is supposed to be invoked, but isn't found. I can't tell you if Drupal is trying to call it, or if PHP or the server is doing it. A "500" error is also known as an "Internal Server" error, so I would guess that the server has tripped a 500/IS error, and is complaining that it can't find a custom error handler for that error. That's common to see Apache servers complain about not being able to find a custom error handler, even though a default one exists. Anyway, if that's all it is, you can probably ignore this second error, as it will go away once you fix the first error (the one actually triggering the 500 error). Think about providing some custom error handlers (error documents), per instructions from your host: 400, 401, 403, 404, and 500 errors at a minimum.

pltmdude’s picture

Thanks Druid.
Yes it is Fast CGI PHP. I had to set-up my own php.ini in order to increase the memory limit and upload limit. I followed the instructions here:
http://wiki.dreamhost.com/PHP.ini
Unfortunately, Dreamhost doesn't support any attempts to do this. What I don't understand, related to the private_download module, is why it's been working for me all along until now. And I only get this problem when I have private_download enabled.

So, what do you think is trying to send a paragraph tag as the start of a header? I've checked .htaccess, php-wrapper.fcgi, and php.ini, and none of them start with a paragraph marker or blank line or anything.

The error message that I see on the screen is sufficient for now. It must be coming from drupal, because it gives my webmaster email account as the contact person. Anyway, I can trouble shoot on that later. Thanks, though.

johnhanley’s picture

And I only get this problem when I have private_download enabled.

It's certainly possible the header structure sent by Private Download is incompatible with your web host and/or your particular server configuration.

Have you considered switching web hosts? One that allows you to adjust the memory and upload limits without all the workarounds and hoop-jumping.

pltmdude’s picture

Status: Active » Fixed

I found it! In the private_download settings page, the 3rd field is for file headers. That's where the paragraph tag was coming from that was causing the malformed header!

I'm using CKEditor as a wysiwyg editor, and had added the htaccess field on that page to the exclusion list, but not the file headers field. So, when I switched it over to plain text, I saw the paragraph tag coming before "Content-Transfer-Encoding".

Wow, that turned out to be so obvious! I'm sorry that I didn't think to check that when we saw that the error had something to do with headers. But since I haven't had to edit anything in that field, I didn't pay any attention to it.

So, a note to all who are dense enough like me to not think of it, make sure your wysiwyg editor isn't messing with the contents of either of those fields! I was right that it had something to do with the module, but it wasn't a problem with the code, it was an operator error in USING the module.

John (bacteriaman) and Druid, thanks for sticking with me. And John, thanks for a wonderful module!

johnhanley’s picture

pltmdude,

Funny, I didn't even think to ask if you had a WYSIWYG enabled. It was just too obvious, as you said. Sidebar rant: I seriously hate WYSIWYG editors and avoid them like the plague for this reason and more. Unfortunately many clients like them (ugh).

Anyway, that's good news.

John

pltmdude’s picture

I'm feel the same way! I hate the problems they cause, but they help me to delegate more of the content production. And I even find them to be handy at times.

Thanks again. I'm looking forward to finally putting this excellent module to use. It certainly fills a great need.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.