Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.
Constant CTOOLS_PREG_CLASS_ALNUM in includes/cleanstring.inc contains \x{d800}- which is ill-formed code point. PHP issues this warning:
Warning: preg_match(): Compilation failed: disallowed Unicode code point (>= 0xd800 && <= 0xdfff) at offset 1811 in ctools_cleanstring() (line 157 of /srv/http/XXXX/www/sites/all/modules/contrib/ctools/includes/cleanstring.inc).
Easily reproducable by running this:
define('CTOOLS_PREG_CLASS_ALNUM',
'\x{0}-\x{2f}\x{3a}-\x{40}\x{5b}-\x{60}\x{7b}-\x{bf}\x{d7}\x{f7}\x{2b0}-' .
'\x{385}\x{387}\x{3f6}\x{482}-\x{489}\x{559}-\x{55f}\x{589}-\x{5c7}\x{5f3}-' .
'\x{61f}\x{640}\x{64b}-\x{65e}\x{66a}-\x{66d}\x{670}\x{6d4}\x{6d6}-\x{6ed}' .
'\x{6fd}\x{6fe}\x{700}-\x{70f}\x{711}\x{730}-\x{74a}\x{7a6}-\x{7b0}\x{901}-' .
'\x{903}\x{93c}\x{93e}-\x{94d}\x{951}-\x{954}\x{962}-\x{965}\x{970}\x{981}-' .
'\x{983}\x{9bc}\x{9be}-\x{9cd}\x{9d7}\x{9e2}\x{9e3}\x{9f2}-\x{a03}\x{a3c}-' .
'\x{a4d}\x{a70}\x{a71}\x{a81}-\x{a83}\x{abc}\x{abe}-\x{acd}\x{ae2}\x{ae3}' .
'\x{af1}-\x{b03}\x{b3c}\x{b3e}-\x{b57}\x{b70}\x{b82}\x{bbe}-\x{bd7}\x{bf0}-' .
'\x{c03}\x{c3e}-\x{c56}\x{c82}\x{c83}\x{cbc}\x{cbe}-\x{cd6}\x{d02}\x{d03}' .
'\x{d3e}-\x{d57}\x{d82}\x{d83}\x{dca}-\x{df4}\x{e31}\x{e34}-\x{e3f}\x{e46}-' .
'\x{e4f}\x{e5a}\x{e5b}\x{eb1}\x{eb4}-\x{ebc}\x{ec6}-\x{ecd}\x{f01}-\x{f1f}' .
'\x{f2a}-\x{f3f}\x{f71}-\x{f87}\x{f90}-\x{fd1}\x{102c}-\x{1039}\x{104a}-' .
'\x{104f}\x{1056}-\x{1059}\x{10fb}\x{10fc}\x{135f}-\x{137c}\x{1390}-\x{1399}' .
'\x{166d}\x{166e}\x{1680}\x{169b}\x{169c}\x{16eb}-\x{16f0}\x{1712}-\x{1714}' .
'\x{1732}-\x{1736}\x{1752}\x{1753}\x{1772}\x{1773}\x{17b4}-\x{17db}\x{17dd}' .
'\x{17f0}-\x{180e}\x{1843}\x{18a9}\x{1920}-\x{1945}\x{19b0}-\x{19c0}\x{19c8}' .
'\x{19c9}\x{19de}-\x{19ff}\x{1a17}-\x{1a1f}\x{1d2c}-\x{1d61}\x{1d78}\x{1d9b}-' .
'\x{1dc3}\x{1fbd}\x{1fbf}-\x{1fc1}\x{1fcd}-\x{1fcf}\x{1fdd}-\x{1fdf}\x{1fed}-' .
'\x{1fef}\x{1ffd}-\x{2070}\x{2074}-\x{207e}\x{2080}-\x{2101}\x{2103}-\x{2106}' .
'\x{2108}\x{2109}\x{2114}\x{2116}-\x{2118}\x{211e}-\x{2123}\x{2125}\x{2127}' .
'\x{2129}\x{212e}\x{2132}\x{213a}\x{213b}\x{2140}-\x{2144}\x{214a}-\x{2b13}' .
'\x{2ce5}-\x{2cff}\x{2d6f}\x{2e00}-\x{3005}\x{3007}-\x{303b}\x{303d}-\x{303f}' .
'\x{3099}-\x{309e}\x{30a0}\x{30fb}-\x{30fe}\x{3190}-\x{319f}\x{31c0}-\x{31cf}' .
'\x{3200}-\x{33ff}\x{4dc0}-\x{4dff}\x{a015}\x{a490}-\x{a716}\x{a802}\x{a806}' .
'\x{a80b}\x{a823}-\x{a82b}\x{d800}-\x{f8ff}\x{fb1e}\x{fb29}\x{fd3e}\x{fd3f}' .
'\x{fdfc}-\x{fe6b}\x{feff}-\x{ff0f}\x{ff1a}-\x{ff20}\x{ff3b}-\x{ff40}\x{ff5b}-' .
'\x{ff65}\x{ff70}\x{ff9e}\x{ff9f}\x{ffe0}-\x{fffd}');
if (preg_match('/^[^' . CTOOLS_PREG_CLASS_ALNUM . ']+$/uD', '-')) {
echo 'Warning issued';
}
Patch to be posted in comment.
Comments
Comment #1
kotnik CreditAttribution: kotnik commentedPatch with fix attached.
Comment #2
merlinofchaos CreditAttribution: merlinofchaos commentedNice! And a properly formatted commit message which is rare! Thank you.
Comment #3
activelink CreditAttribution: activelink commentedThis bug is likely caused by pcre 8.30+, most Linuxes still use older versions, but FreeBSD has 8.30_1 currently. But this version of pcre will be on Linux soon - it's developed on FreeBSD.
It breaks core search too in D6/7.
Ctools has it because it has code in cleanstring.inc that is copied from includes/unicode.inc (D7 core) or modules/search/search.module in D6.
It's examined in some depth here:
Invalid Unicode code range in PREG_CLASS_UNICODE_WORD_BOUNDARY fails with PCRE 8.30
There are patches for D6 and D7, and these work also if adapted for ctools' unicode.inc
These patches are slightly different to the one above. They replace the D800 parameter with E800 and this works. The patch above removes it entirely. I don't understand this code or what it does, but it seems like the patches that I'm referencing are more evolved than the one given on this page (which removes this value rather than replacing it). Maybe someone who understands its function might want to look at it?
Comment #4
kotnik CreditAttribution: kotnik commentedYes, this is just the same issue as one described in #1446372: Invalid Unicode code range in PREG_CLASS_UNICODE_WORD_BOUNDARY fails with PCRE 8.30 (pcre > 8.30, observable in FreeBSD and Arch, probably Gentoo too).
As a matter of fact, CTOOLS_PREG_CLASS_ALNUM (changed in patch from #1) is copied from search.module's PREG_CLASS_SEARCH_EXCLUDE.
Removing d800 is quite safe, since CTOOLS_PREG_CLASS_ALNUM is used to escape separator if it's not alphanumeric.
Comment #6
Marc Angles CreditAttribution: Marc Angles commentedHi, I really don't know if this patch is right, but for the moment there is no more messages on my websites (This is D6.25)
Comment #7
kotnik CreditAttribution: kotnik commentedUpdated to be in sync with #1446372: Invalid Unicode code range in PREG_CLASS_UNICODE_WORD_BOUNDARY fails with PCRE 8.30.
As noted in 1446372's comment #16 it's better to scan Unicode private use instead of surrogate area, then to ignore it.
This code has been merged in Drupal core 8.x and 7.x (patch waiting to be ported to 6.x), and I think ctools should do the same.
PS. @Marc Angles: yes, that's the same fix that'll be included in next 6.x core.
Comment #8
lucascaro CreditAttribution: lucascaro commentedI've just manually replaced
with
'\x{a80b}\x{a823}-\x{a82b}\x{e000}-\x{f8ff}\x{fb1e}\x{fb29}\x{fd3e}\x{fd3f}'.
as in #7 and it fixed the error messages in 6.x-1.8+50-dev
Since the first half of the patch seems to be already committed, that last change is all that is needed to remove the message (at least using my configuration).
RTBC, anyone?
Comment #9
kotnik CreditAttribution: kotnik commentedThe exact same fix as in #7 has already been commited in D6, D7, and D8. Marking it as RTBC.
Comment #10
lucascaro CreditAttribution: lucascaro commentedare they committed to dev? awesome! also, maybe this should be marked as fixed?
thanks!
Comment #11
kotnik CreditAttribution: kotnik commented@lucascaro:
No, patch from #7 needs to be commited to ctools, since the exact same fix was applied to Drupal 6, 7 and 8.
Comment #12
lucascaro CreditAttribution: lucascaro commentedooh, nice. This is starting to trigger error messages in all of my projects, so I'd love it to be committed. How can I help making it happen?
Comment #13
tim.plunkettFor tonight.
Comment #14
lucascaro CreditAttribution: lucascaro commentedawesome!
Comment #15
tim.plunkettOh wow the format-patch threw me off there. This is much more straightforward then I thought.
Also, why'd you unassign me? I almost couldn't find the issue.
http://drupalcode.org/project/ctools.git/commit/a1c5c9b
Comment #16
lucascaro CreditAttribution: lucascaro commentedog god, sorry about that. I don't know why that happened, I didn't touch the assigned box. :S
Comment #18
gdl CreditAttribution: gdl commentedSorry to reopen, but it looks to me like this didn't make it into 6.x-1.x-dev.
Attached is a patch for 6.x-1.x-dev made by copying PREG_CLASS_SEARCH_EXCLUDE from search.module from Drupal 6.26. Works for me, but not thoroughly tested.
Best,
-G
Comment #19
hansfn CreditAttribution: hansfn commentedI agree that this seems to be unfixed still.
@gdl: Why the difference between your patch and the one in #1697538: Unicode compilation fails because of invalid code range in CTOOLS_PREG_CLASS_ALNUM?
Comment #20
gdl CreditAttribution: gdl commentedIt looks like the patch in #1697538 changes just a single code point, while mine changes the list of code points in CTools to match the code points from the core search module. I guess it's up the the developers to determine which approach to take. I don't have an informed opinion about which is better - I just want to see the issue resolved!
Cheers,
-G
Comment #21
jhedstromFix in #18 gets rid of the warnings in 6.x-1.x.
Comment #22
carnau CreditAttribution: carnau commentedThe #18 patch works here, with pcre package version 7.8 and ctools 6.x-1.10+0-dev.
Comment #23
EugenMayer CreditAttribution: EugenMayer commentedconfirmin, patch from #18 fixes the issues you get when using PHP > 5.3.18 . Applying did not succeed though. Thanks!
*removed since it up there already*
Comment #24
trotskyicepick CreditAttribution: trotskyicepick commentedJust upgraded to 6.x-1.10, and found the error from the OP was again occurring, #8 again solves the problem.
Why is this not changed in the module core?
Comment #25
neclimdulThanks guys. Tested and applied #1697538-2: Unicode compilation fails because of invalid code range in CTOOLS_PREG_CLASS_ALNUM since it brings the regex into a state matching the 7.x-1.x branch it seemed more reasonable. If we need the extra change here can you reroll with an explanation?
Comment #26
beto_beto CreditAttribution: beto_beto commentedhello
i am not professional in Drupal core settings
and this error appear for me too
warning: preg_match(): Compilation failed: disallowed Unicode code point (>= 0xd800 && <= 0xdfff) at offset 1811 in C:\inetpub\vhosts\xxxxx\httpdocs\sites\all\modules\ctools\includes\cleanstring.inc on line 157.
how can i fix this problem pleas ????
Comment #27
beto_beto CreditAttribution: beto_beto commented#8 Work for me :D
I've just manually replaced
'\x{a80b}\x{a823}-\x{a82b}\x{d800}-\x{f8ff}\x{fb1e}\x{fb29}\x{fd3e}\x{fd3f}'.
with
'\x{a80b}\x{a823}-\x{a82b}\x{e000}-\x{f8ff}\x{fb1e}\x{fb29}\x{fd3e}\x{fd3f}'.
Comment #28
crutch CreditAttribution: crutch commentedthis is happening more often after I search and search, then post, I find the issue when searching google on drupal. I could not find this issue in Drupal search.
possible duplicate: http://drupal.org/node/1878284
This seems still unfixed in the module.
Comment #29
aitona CreditAttribution: aitona commentedI solved it doing this. I TRUNCATE the following tables: cache_page, acceslog, cache_menu and cache_filter.
I think that commen #27 don't work for me, but i set the line like beto_beto say:
'\x{a80b}\x{a823}-\x{a82b}\x{e000}-\x{f8ff}\x{fb1e}\x{fb29}\x{fd3e}\x{fd3f}'.
Comment #30
gnugetI've the same bug and also the patch from #18 fixed it.
Comment #31
mkindred CreditAttribution: mkindred commentedI just ran into this problem for this first time today (6.x-1.10).
#8 solved the problem.
Comment #32
bmcginnis CreditAttribution: bmcginnis commentedI'm on 6.x-1.10, running PHP 5.3.21, and running Drupal 6.28. Anyone know if #18 or #8 is the better route to take?
Comment #33
bmcginnis CreditAttribution: bmcginnis commentedI've used the approach from #8 and that's solved the problem for me.
Comment #34
Grabby CreditAttribution: Grabby commentedI was getting
warning: preg_match() [function.preg-match]: Compilation failed: disallowed Unicode code point (>= 0xd800 && <= 0xdfff) at offset 1811 in /var/www/mysite/public_html/sites/all/modules/ctools/includes/cleanstring.inc on line 157
a number of times after upgrading my server from
PHP 5.2.17-0.dotdeb.0
MySQL 5.1.56
to
PHP 5.3.21-1~dotdeb.0
MySQL 5.1.66
on a Drupal 6.28 site running cTools 6.x-1.10, but #8 solved the problem. Not sure why this hasn’t been committed!
Comment #35
kmare CreditAttribution: kmare commentedSame as Grabby for me, #8 fixed the problem on my drupal6 installations. I really believe it should be commited.
Comment #36
jacquelynfisher CreditAttribution: jacquelynfisher commentedThanks! I had the same issue as #34: the error appeared after our servers were upgraded to PHP5.3.x. The simple fix in #8 resolved the problem.
Comment #37
generalconsensus CreditAttribution: generalconsensus commentedThe fix from #8 works but doesn't affect search.module. This patch incorporates #8 with a fix for the search module using the same code.
Works for me on:
FREEBSD
Drupal 6.16
PHP 5.3.19
APACHE 2.2.23
Comment #38
dadderley CreditAttribution: dadderley commentedAnother vote for solution in #8
Comment #39
laken CreditAttribution: laken commentedConfirm that patch in #18 worked - CTools 6.x-1.10, Drupal 6.28, PHP 5.3.19, Debian 6.0.6
Comment #40
jonathanhuot CreditAttribution: jonathanhuot commented#18 worked very well for my installation :
CTools 6.x-1.10
PHP 5.3.23
Drupal 6.28
I updated to "tested by the community" because it's enough trivial to include it into git.
Comment #42
merlinofchaos CreditAttribution: merlinofchaos commentedAs per #25, this was marked fixed; neclimdul committed the patch. Why are there 17 comments after that asking for this to be committed?
The reason it failed testing is the patch, which is already committed, doesn't apply.
Do people actually read issues before they start in? Because it seems like a giant waste of time to have so many people asking why a fix, which is committed, hasn't been committed. Because it's been committed.
Comment #43
hansfn CreditAttribution: hansfn commentedThere are 17 comments because, as I said already in comment 19, the fix was only committed to the 7.x-1.x branch, not the 6.x-1.x branch. Hence, the bug is still present on 6.x-1.10... This is clearly frustrating for people using Drupal 6. (The fix is almost one year old.)
OK, merlin, you are right. The fix was applied to the 6.x-1.x branch - ref this commit. So the only problem is that there hasn't been a new release of Ctools since 6.x-1.10.
Comment #44
hansfn CreditAttribution: hansfn commentedUpdating version info since this isn't fixed for 6.x-1.10, only in 6.x-1.x-dev.
Comment #45
hansfn CreditAttribution: hansfn commentedSorry about the noise. The bug, which is fixed, was in version 6.x-1.10. Some times I'm just that stupid.
Comment #46
AnybodyThis is a quite important issue. Wouldn't it make sense to release a new stable version 2 month after that? We all have to switch to .dev else.
Comment #47
KarlSheaI have 6.x-1.10 installed and the commit you reference that fixes the bug has not been applied.
cleanstring.inc line 61 is:
'\x{a80b}\x{a823}-\x{a82b}\x{d800}-\x{f8ff}\x{fb1e}\x{fb29}\x{fd3e}\x{fd3f}'.
d800 should be e000
Comment #48
GiorgosKHad same problem with 6.x-1.10
#18 patch solves this problem
#37 includes a patch for core search module that should be posted to core queue
Comment #49
merlinofchaos CreditAttribution: merlinofchaos commentedAs per #42, this was committed. It was committed after 6.x-1.10 was released. Please pay attention when changing issue statuses.
And I'm sorry there hasn't been a new release, but releases for Drupal 6 are not high on my priority list.
Comment #50
KarlSheaUh, I was paying attention. #45 indicates that it was in 6.x-1.10. Which it isn't.
Comment #51
ifrikpatch #18 worked for me
Comment #52
darshanchoudhary CreditAttribution: darshanchoudhary commentedpatch #18 worked for me too.. :)
Comment #52.0
darshanchoudhary CreditAttribution: darshanchoudhary commented:)
Comment #53
beto_beto CreditAttribution: beto_beto commented#8 Work for ME
this error was appear to me
Warning: preg_match(): Compilation failed: disallowed Unicode code point (>= 0xd800 && <= 0xdfff) at offset 1811 in ctools_cleanstring() (line 157 of /srv/http/XXXX/www/sites/all/modules/contrib/ctools/includes/cleanstring.inc).
Solution
on
sites/all/modules/contrib/ctools/includes/cleanstring.inc
I've just manually replaced
'\x{a80b}\x{a823}-\x{a82b}\x{d800}-\x{f8ff}\x{fb1e}\x{fb29}\x{fd3e}\x{fd3f}'.
with
'\x{a80b}\x{a823}-\x{a82b}\x{e000}-\x{f8ff}\x{fb1e}\x{fb29}\x{fd3e}\x{fd3f}'.
Thanks
Comment #54
Sylvain Lasnier CreditAttribution: Sylvain Lasnier commentedme too
Comment #55
jasonabc CreditAttribution: jasonabc commentedthanks @gdl (comment #18) for the D6 patch. Works great.
Comment #56
gpk CreditAttribution: gpk commented@50 and for general clarification : #45 indicates that the bug was in 6.x-1.10 (which was released Nov 14, 2012) and since the fix was committed on Nov 29, 2012 it only appears in subsequent releases, i.e. 6.x-1.11 released Jan 12, 2014 and any later releases that there may be.
Comment #57
davidwbarratt CreditAttribution: davidwbarratt commentedI'm receiving the same/similar error on 7.x-1.0-rc1:
Any ideas?
Comment #59
DamienMcKenna@davidwbarratt: Please open a new issue for the D7 problem if it persists in the latest D7 release.