If a value contains newlines it will corrupt the CSV export because they do not get quoted and end up interpreted as the start of a new record.
Also, though less serious, a value with leading or trailing spaces will lose that spacing in the export process without the value being enclosed in quotes.
There's no such thing as an official CSV specification but a general rule is that all values may be quoted even if unnecessarily. I think the safest, easiest way to solve this problem is to quote every value.
To do so, I've changed function _webform_report_format_csv_column in webform_report.inc from the following:
function _webform_report_format_csv_column($value, $delim) {
// if value contains a delimiter, it should be enclosed in quotes
if (strpos($value, $delim) !== FALSE) {
// if value contains quotes, double the quotes
if (strpos($value, '"')) {
return '"' . str_replace('"', '""', $value) . '"';
}
else {
return '"' . $value . '"';
}
}
return $value;
}
into:
function _webform_report_format_csv_column($value, $delim) {
// if value contains quotes, double the quotes
if (strpos($value, '"')) {
return '"' . str_replace('"', '""', $value) . '"';
}
else {
return '"' . $value . '"';
}
return $value;
}
So far it seems to work well.
My only problem is that when opened in Excel (after exporting with either Text or Excel option) those newlines appear as unknown characters (rectangle with question mark inside). I know that Excel can handle newlines in a CSV, so this must be an encoding issue. Unfortunately I'm not well versed enough in encoding to know how to solve this. Hopefully someone has a suggestion about the encoding, but even so this is better than a corrupted CSV.
Attached a patch against 6.x-2.0 with the above change.
| Comment | File | Size | Author |
|---|---|---|---|
| #5 | webform_report-csv-quote-all-crlf.patch | 903 bytes | msielski |
| #1 | webform_report-csv-quote-all.patch | 838 bytes | msielski |
| webform_report-csv-quote-all.patch | 858 bytes | msielski |
Comments
Comment #1
msielskiUh oh, I messed up the return statements there. Here's the new version:
New patch attached.
Comment #2
jimbullington commentedThank you for the report and the patch. I wonder if you could try replacing CRLF with LF characters.
That is mentioned here:
http://stackoverflow.com/questions/1241220/generating-csv-file-for-excel...
Also, did you try the Excel download option?
Comment #3
msielskiThank you! It was apparently the CRs in Excel showing up as unrecognized. The replacement of CRLF to LF fixed it up. Do you think this should be added to the export code?
I was using both the Excel and Text options and both had the unrecognized character symbol. Both are now fixed with the substitution.
Comment #4
jimbullington commentedYes - when you have time, roll up a patch with this change as well. When you feel confident in the code, I'll push to the repository.
Comment #5
msielskiOk, great. What I'm using is this:
I removed the strpos test. I could be wrong but I think it's unnecessary to test for the string before attempting to replace it.
I'll be testing this with some other unrelated items on our dev site this week and will probably have a good idea how it's doing by the start of next.
Comment #6
msielskiI have tested this most recent patch and taken it into production recently. I'm happy with the way it works.
Comment #7
msielskiForgot to update status. I've reviewed my own patch, but should get maintainer or 3rd party review as well.
Comment #8
jimbullington commentedWorks for me!
Comment #9
jimbullington commentedCommitted patch to repository.