PayPal specifies that the DESC field has "Character length and limitations: 127 single-byte alphanumeric characters" however, uc_paypal module is inserting the multiple-byte × "times" character (hex UTF-8 bytes C3 97). This unexpected multiple-byte data results in corrupted and truncated "Notification of payment received" e-mails sent to site administrators by PayPal.com.

NOTE: Many PayPal fields specify single-byte characters. But in Drupal, any user-entered data could contain multiple-byte UTF-8 characters. Aside from fixing this bug, it would be a good idea to sanitize this and other fields which could contain non-ASCII characters before submitting to PayPal. Transliteration module could be used to convert UTF-8 strings to ASCII strings.

CommentFileSizeAuthor
#1 uc_paypal-7-multiple-byte-character.patch487 bytesmfb
uc_paypal-6-multiple-byte-character.patch483 bytesmfb

Comments

mfb’s picture

mfb
they or he
commented
Version: 6.x-2.x-dev » 7.x-3.x-dev
StatusFileSize
new uc_paypal-7-multiple-byte-character.patch487 bytes

Patch for 7.x-3.x branch

longwave’s picture

longwave
he/him
Primary language English
Location UK
commented
Priority: Normal » Minor
Status: Needs review » Active

Committed to both branches, thanks.

Leaving active but minor to fix the remaining issue with other data being sent to PayPal. I wonder how this works in countries that use extended character sets in (e.g.) billing or delivery addresses, does PayPal just not accept these?

longwave’s picture

longwave
he/him
Primary language English
Location UK
commented

Can we just use utf8_decode() on any fields that may contain multibyte characters? (description, addresses, etc)

mfb’s picture

mfb
they or he
commented

I tested utf8_decode() with non-ASCII characters and no, it doesn't work. I believe when PayPal says "single-byte characters" they mean single-byte UTF-8 characters, not a legacy single-byte encoding like ISO-8859-1.

When testing non-ASCII ISO-8859-1 characters in the description, in some test cases I received a PayPal Error 10001: Internal Error and the payment failed. Or else the payment went thru but the payment notification e-mail was corrupted.

Here's an idea for a helper function:

function uc_paypal_convert_to_ascii($text) {
  // If Transliteration module is installed, use it to convert to ASCII.
  if (function_exists('transliteration_get')) {
    return transliteration_get($text);
  }
  // Strip characters aside from tab, line feed, carriage return, and ASCII printable characters.
  return preg_replace('/[^\x09\x0A\x0D\x20-\x7E]/', '', $text);
}
tr’s picture

tr commented
Issue summary: View changes
Status: Active » Closed (outdated)

This is no longer an issue because PayPal changed their API at some point and they now accept UTF-8 strings.

https://developer.paypal.com/docs/classic/api/NVPAPIOverview/

The PayPal API assumes that all data in requests is in Unicode, specifically, the Unicode (or UCS) Transformation Format, 8-bit encoding form (UTF-8).