Firstly, I know someone is going to say that "this isn't critical" and other people will say "it is not a bug, it's a problem with your PHP installation", so please, before you do, read the entire post and listen to my reasoning. If you still think this issue needs to be changed from 'critical' to 'normal' or 'bug report' to 'support request' or anything else after that, then please just change it and accept my apologies.

Server
FreeBSD running PHP 4.3.11.
PCRE support is enabled and running PCRE Library Version 4.5

SITE 1
Running 4.7.0-beta2
On the admin/settings page, I have this error:

The PCRE library in your PHP installation is outdated. This will cause problems when handling Unicode text. If you are running PHP 4.3.3 or higher, make sure you are using the PCRE library supplied by PHP. Please refer to the PHP PCRE documentation for more information.

SITE 2
On the same server, also running 4.7.0-beta2 but without the error.

This is a list of core system files installed that I think are relevant and the different version numbers for each site at the point where SITE 1 was producing the error and SITE 2 wasn't.

theme.inc - SITE 1 - 1.272, SITE 2 - 1.267
bootstrap.inc - SITE 1 - 1.80, SITE 2 - 1.78
unicode.inc - SITE 1 - 1.12, SITE 2 - 1.11
common.inc - SITE 1 - 1.504, SITE 2 - 1.499
form.inc - SITE 1 - 1.42, SITE 2 - 1.34
locale.inc - SITE 1 - 1.62, SITE 2 - 1.60
menu.inc - SITE 1 - 1.96, SITE 2 - 1.94
locale.module - SITE 1 - 1.130, SITE 2 - 1.129
system.module - SITE 1 - 1.277, SITE 2 - 1.268

I recently started to upgrade the files in SITE 2 to see if I could spot where the problem was. Here's the path I took.

1) Upgraded theme.inc from 1.267 to 1.272
2) Tested settings page, error message did NOT appear
3) Upgraded unicode.inc from 1.11 to 1.12
4) Tested settings page, error message DID appear
5) Downgraded unicode.inc from 1.12 to 1.11
6) Tested settings page, error message STILL appeared

So now I'm stuck and any help would be appreciated.

Why am I posting this as a bug? Because if the error message doesn't appear for unicode.inc 1.11 and does appear for unicode.inc 1.12, I am assuming that it is not expected to be there and, despite it's content, does not accurately describe the problem. I'm willing to accept that I could be wrong about this, but I am relatively new to this, so please forgive me.

Why am I posting this as critical? Because, although I have a dedicated server with root access, I host with Donhost who were recently purchased by Pipex and the server is one of their standard builds hosted in one of the largest data-centres in the UK. What I'm trying to say is that it is pretty much a vanilla-flavour server and my thinking is that, if I get this problem, then potentially, so will lots of other people. Again, willing to be wrong.

Regards

Patrick

CommentFileSizeAuthor
#3 testpcre.php.txt101 byteskilles@www.drop.org

Comments

killes@www.drop.org’s picture

killes@www.drop.org commented
Priority: Critical » Normal

That's very strange, he difference between the two unicode.inc versions is only some whitespace. The error message could also have been cached in your session somehow.

You could maybe look at the compatibilty test in unicode.inc, factor it out, and see what happens when you run it stand-alone?

The pcre version you have should be sufficient.

I disagree that this is critical, it hasn't been reported very often...

Patrick Nelson’s picture

Patrick Nelson commented

Killes,

Thanks for your reply. I'll go with the 'normal' rating if that's the case of it having not been reported very often.

You could maybe look at the compatibilty test in unicode.inc, factor it out, and see what happens when you run it stand-alone?

Could you give me some pointers as to how to do this? I have root access to the server and know a bit of CLI stuff but I'm not up-to-date on the whole PHP installation stuff.
Regards

Patrick

killes@www.drop.org’s picture

killes@www.drop.org commented
StatusFileSize
new testpcre.php.txt101 bytes

I've attached a testscript, upload it to the server and access it through the browser, what do you see?

Patrick Nelson’s picture

Patrick Nelson commented

Killes,

Thanks again for your help. The result given was:

Warning: preg_match(): Compilation failed: invalid UTF-8 string at offset 2 in /home/v/c/vcommunityaab/public_html/testpcre.php on line 2
PCRE works

The file is at http://www.v-community.co.uk/testpcre.php

Regards

Patrick

killes@www.drop.org’s picture

killes@www.drop.org commented

Well, then there seems to be an actual problem. Are you sure your PCRE library is 4.5? You can look at the version in the output of phpinfo().

Patrick Nelson’s picture

Patrick Nelson commented

Yeah, that's where I got the info: http://www.vcommunity.org.uk/phpwhat.php (both sites are on the same server, one's a test site, the other is live)

killes@www.drop.org’s picture

killes@www.drop.org commented
Status: Active » Closed (won't fix)

Then the only remaining option is that there is some bug in the php/pcre for freeBSD. I've googled a bit and found for example this one: http://bugs.php.net/bug.php?id=32659

Settign to "won't fix" as it is system specific and not Drupal's fault. You can simply not use search.module and use trip_search instead. search.module is the only module requiring pcre for unicode.

Patrick Nelson’s picture

Patrick Nelson commented

Killes,

I appreciate your help and fully accept your answer - I'm going to check this out with the hosting centre - but, assuming they cannot (or will not) help me, I will still have a problem (even if I disable the search module) inasmuch as I cannot change any of the settings on the /settings screen

Basically, if I do, I get this tagged onto the end of the PCRE error message:

The settings have not been saved because of the errors.

Is there any way I can just turn off the unicode requirements altogether? I accept that it is not a Drupal problem and, so I appreciate any help you can offer.

Regards

Patrick

killes@www.drop.org’s picture

killes@www.drop.org commented

Yeah, I was wrong about pcre only needed by search.module, this was true for 4.6 only. In 4.7 it will be also needed in user.module. So the only solution seems to be to revert to 4.6.5 and figure out the PCRE issue before upgrading.

Patrick Nelson’s picture

Patrick Nelson commented

Killes,

Yeah, thanks for this.

I've raised a support issue with my host and I'll wait to see what they say - I have some other servers that I can use if necessary but I'd rather not move the site if I can help it because I'm most familiar with this particular set-up.

Let me know if you want to know what the host says and I'll keep you posted. Other than that, thanks for all your help - I wouldn't even be halfway toward a solution without it.

Regards

Patrick

killes@www.drop.org’s picture

killes@www.drop.org commented

Please add any resolution here for the benefot of people having similar problems.

killes@www.drop.org’s picture

killes@www.drop.org commented

Here's a solution for removing the problem if you have root:
http://drupal.org/node/40961

Patrick Nelson’s picture

Patrick Nelson commented

Thanks Killes, but even that didn't work. My host tells me that the problem is the string given in preg_match contains characters not supported by utf-8. I don't know enough to disagree with them but my instinct tells me that that is not true.

If there is anyone else suffering similar problems, I have temporarily got around it by changing the following in /includes/unicode.inc from:

Line 32 if (preg_match('/[à -á]/u', 'â')) {

to:

Line 32 if (preg_match('/[^\.\/]+\.[^\.\/]+$/', 'â')) {

I stress that this is a temporary solution. If you use this, don't make the mistake of thinking your problems are over. I don't know what else this will affect or break, although search, etc., does seem to work. However, I have little content in the test site I'm using for this and so, cannot be 100% sure.

Regards

Patrick

Steven’s picture

Steven commented

The test as it is coded in Drupal's unicode.inc and in Killes' script is valid UTF-8. Most likely you have accidentally converted or corrupted it by copy/pasting it into or out of a bad editor.

It is not:
if (preg_match('/[à -á]/u', 'â')) {
but:
if (preg_match('/[à-á]/u', 'â')) {
(encoded as UTF-8)

Perhaps the following ASCII-safe equivalent will not be mangled:
if (preg_match('/[\xC3\xA0-\xC3\xA1]/u', '\xC3\xA2')) {

Patrick Nelson’s picture

Patrick Nelson commented

Steven,

That's it! That worked brilliantly. Thank you very much - as you can probably see, this problem has been a pain in the ****, so I'm very grateful for your help.

Regards

Patrick

Patrick Nelson’s picture

Patrick Nelson commented
Status: Closed (won't fix) » Closed (fixed)

Just a note to anyone else having a similar issue.

Make sure whatever you're using to upload files uploads the relevant Unicode files in BINARY and not ASCII (ahem!)

Dublin Drupaller’s picture

Dublin Drupaller commented

Hiya Patrick,

I've just used that snippet to get Drupal 4.7 RC1 working on a VPS account that was returning the error you mentioned. I've edited unicode.inc as instructed and the error message dissapears under the AMINISTER -> SETTINGS page now. Thanks.

Just wondering if there are any remifications for fixing it this way. You mentioned that it is just a "temporary solution". Assume that means that it is not an "ideal solution".

The hosts I'm using for the VPS account are slow in getting back to me about upgrading their php from version 4.3.2 to 4.3.3 (they mentioned that will break their webmail system) so I'm trying to find a workaround that *doesn't* involve relying on the hosts upgrading or having to move hosts, just to use Drupal 4.7.

Cheers for any guidance/tips provided

Dub

gerhard killesreiter’s picture

gerhard killesreiter commented

Patrick's problem was caused by inaproriate settins in his ftp client:
http://drupal.org/node/43567#comment-68510

Dublin Drupaller’s picture

Dublin Drupaller commented

am confused Gerhard,

If you look at the unicode.inc file here:

http://cvs.drupal.org/viewcvs/*checkout*/drupal/drupal/includes/unicode....

the offending line 32 that was causing problems looks like this: if (preg_match('/[à -á]/u', 'â')) {

Are you saying that when you download Drupal 4.7 RC1, you have to open the unicode.inc in a text editor, change the encoding from ASCII to UTF-8 and then upload it to the server?

Apologies if that is a stupid question..but, I assume that line 32 was the one that was causing Patrick the grief.

Dub

killes@www.drop.org’s picture

killes@www.drop.org commented
Category: bug » support

no, the problem was that he untarred the tarball locally and then uploaded the file through ftp. ftp can transfer files in two modes: ascii and binary. While ascii is appropriate for text files most of the time, that is no longer true if they contain unicode.

the full octave’s picture

the full octave commented

thanks killes. So when I download Drupal 4.7 RC1 I should upload everything else as normal (I always used to use the AUTO function when FTPing files which detects whether it is an ASCII/BINARY transfer) and *just* upload the unicode.inc file seperately as a binary transfer?

Is that correct?

Or am I confusing myself?

Dub

killes@www.drop.org’s picture

killes@www.drop.org commented

You can just upload everything as binary, this shouldn't cause trouble.

Dublin Drupaller’s picture

Dublin Drupaller commented

Thanks Killes. (Dub here - I was logged in earlier as another user to see if other problems I was having with drupal.org was because of my Dublin Drupaller user account)

Just tried that...uploading everything as Binary..still getting the same error message:

"The PCRE library in your PHP installation is outdated. This will cause problems when handling Unicode text. If you are running PHP 4.3.3 or
higher, make sure you are using the PCRE library supplied by PHP. Please refer to the PHP PCRE documentation for more information."

When I change line 32 in unicode.inc to this: if (preg_match('/[\xC3\xA0-\xC3\xA1]/u', '\xC3\xA2')) { the error message dissapears.

So my question was does this workaround have any ramifications with Drupal 4.7 rc1?

In other words, is it a short-term elastoplast or a usable solution for using Drupal on a site that has PHP 4.3.2 running?

Apologies in advance if that is a dumb question.

Dub

cog.rusty’s picture

cog.rusty commented

Modern FTP clients do send the required commands for a correct utf-8 ascii transfer, but the server must also understand these commands. If auto has been working for you so far, you don't need to change anything.

Dublin Drupaller’s picture

Dublin Drupaller commented

thanks cogrusty..appreciate the heads up on that.

the root to the problem I'm having is that my server is using php 4.3.2 instead of php 4.3.3. so to play around with Drupal 4.7 RC1 I needed to make the change to line 32 in unicode.inc, just to get past the "YOUR SETTINGS HAVE NOT BEEN SAVED" error message on the ADMINISTER -> SETTINGS page that is triggered by "The PCRE library in the PHP installation is outdated" thing.

So the question is does changing line 32 in the unicode.inc file get around that problem completely for the Drupal 4.7 RC1 site or is it just an elastoplast that does not get around the problem completely?

I hope that makes more sense...

Dub

killes@www.drop.org’s picture

killes@www.drop.org commented

The only real resolution is to update your PHP install. 4.3.2. is _ancient_, Pharaoh Tut used it.

Steven’s picture

Steven commented

Actually the snippet I posted might be bad... I think it needs double quotes rather than single quotes to work well.

Still, if you get the error, something is wrong. As of Drupal 4.7, Unicode functions are used in more places than just search (e.g. username validation). You'll run into issues if it does not work. Get your host to upgrade, or switch hosts.

Dublin Drupaller’s picture

Dublin Drupaller commented

thanks for the heads up Steven.

I've asked the host to upgrade, but they are reluctant because the upgrade will "break other things, like horde email".

After spending a while getting used to a new host, I'd prefer not to move again just to get Drupal 4.7 working, so I thought I would ask if it was possible to patch Drupal to work with php 4.3.2. I don't mind patching a bunch of files if needed.

If you think it's an impossible task and there's no option but to move hosts...I'll start looking now.

Dub

killes@www.drop.org’s picture

killes@www.drop.org commented

Moving from php 4.3.2 to say 4.3.11 should not break anything. I'd dump this hoster for cluelessness...

Dublin Drupaller’s picture

Dublin Drupaller commented

I tend to agree Killes. I get the impression they are stonewalling me on the upgrade php thing.

Will start looking for a new host!

Dub

Patrick Nelson’s picture

Patrick Nelson commented

Dublin,

I'm only suggesting this because of the "Dublin" thing, but if you like, I can host your site for you. I have servers based in the UK and, as a result of what I do, know that they are Drupal-4.7 compliant.

Let me know if this is of interest to you and we can talk further.

Regards

Patrick

Dublin Drupaller’s picture

Dublin Drupaller commented

thanks for the offer Patrick. Just sent you an email.

Not too sure what you meant by "just because of the 'Dublin' thing". but thanks anyway.

It will be useful to have some space so I can test and play around with Drupal 4.7 properly.

I'm also going to have a chat with a few php guys to see if they have an alternate approach to the PCRE library thing.

Cheers

Dub

Patrick Nelson’s picture

Patrick Nelson commented

Dublin Drupaller,

Thanks - had your email and replied - which you should have had by now.

The "Dublin" thing - I'm in the UK, you're in Dublin. My UK-based hosting won't be much use to most Drupallers who are US-based, but might be to you... :)

Dublin Drupaller’s picture

Dublin Drupaller commented

aha. I see. well thanks for that.

you should promote that more in the Drupal hosting page on this site. Because Drupal is essentially a community building tool and can gather personal details it's sometimes important to have that information (for some projects) stored on servers physically located in the UK. theoretically it could be stored anywhere within the EU, but the privacy laws are tightening up and individals/companies are becoming more concious of where their information is stored.

Anyway. I'm digressing from the main point of this thread. I'll post back up here if the php help I have can come up with an alternative approach to the PRCE stuff without having to bring the mountain to mohammed.

cheers and thanks again.

Dub

munyiva’s picture

munyiva
Location Nairobi
commented

Thank you this solved my issue