While using text delimiters, escaped text delimiters are not handled correctly in some particular cases.

The problem happens since data is read from the csv in chunks of 1024 bytes. This can result in situations where the escape character has been read and the delimiter not. However the current code starts looking for escaped delimiters only in the newly read data and thus misses it. Sample data is attached. You can see the problem in the step 3 itself where all the fields are not correctly separated.

The same problem also occurs in some cases when field separator is multi-character.

Drupal: 6.19
PHP: 5.3.2

CommentFileSizeAuthor
#1 escaped-text-delimiter-multichar-fs-1169280-1.patch379 bytesabhaga
error.txt1.25 KBabhaga

Comments

abhaga’s picture

abhaga commented
StatusFileSize
new escaped-text-delimiter-multichar-fs-1169280-1.patch379 bytes

The issue can be solved by re-scanning the complete buffer whenever we read more data in. A patch is attached.