Add support for sites lacking full unicode multibyte support

Can you please specify your full PHP version and compile options? It would be the first part of the output from the PHP phpinfo() function...

Looked into this, and while you really really should have PCRE + Unicode support (Millennium and/or Drupal speak Unicode) you can PROBABLY use this replacement function to bypass your error.

Just replace the current function with this one... tell me if it works!

function millennium_trim_marc_value($value) {
  global $multibyte;

  $newvalue = $value;
  if ($multibyte == UNICODE_MULTIBYTE) {
    $newvalue = preg_replace('/[^\p{L}0-9")\?!-]+ *$/u', "", $newvalue); // \p{L} => handle extended charset
    $newvalue = preg_replace('/^ *[^\p{L}0-9"(-]+/u', "", $newvalue);
  } else {
    // Fallback to non-unicode handling
    $newvalue = preg_replace('/[^a-z0-9")\?!-]+ *$/i', "", $newvalue); 
    $newvalue = preg_replace('/^ *[^a-z0-9"(-]+/i', "", $newvalue);
  }
  $newvalue = preg_replace('/\((.*)\)$/', "\\1", $newvalue);
  return $newvalue;
}

I tested it a bit and it seems to work, at least on my full unicode and "new" PCRE installation. Since it seems code-safe (although perhaps not very wise to bypass this requirement) I went ahead and commited to 6.x-2.x-DEV. You can fetch that new version starting tonight (wait till Drupal.org gets a chance to repackage from CVS).

Thanks for checking this. I asked around in #drupal (irc) and the suggestion was to use some of the code from the core Search module, which is independent on the PCRE libraries... as I understand it is nearly impossible to detect what PCRE options the installed PCRE library has (which in your case, I assume, is missing some options that mine and others' have). Search.module bypasses this using a long list of multibyte values to check against =)

Glad you could circumvent it for now; again, I would not depend on that code being 100% correct, though, since it WILL drop some characters with diacritics in them (like "Motley Crüe", "The effects of el Niño"); well, at least at the beginning and end of strings.

Currently looks like this:

function millennium_trim_marc_value($value) {
  $newvalue = $value;
  $newvalue = trim($value);
  $newvalue = preg_replace('/[\.,\/:;]+$/', '', $newvalue);
  return $newvalue;
}

Perhaps I need to take a page from #768040: truncate_utf8() only works for latin languages (and drupal_substr has a bug) and truncate ending word-boundary and other characters, like so?

$newvalue = preg_replace('/[' . PREG_CLASS_UNICODE_WORD_BOUNDARY . ']+$/u', '', $newvalue );

Closing out 1.x branch.