If a value contains newlines it will corrupt the CSV export because they do not get quoted and end up interpreted as the start of a new record.

Also, though less serious, a value with leading or trailing spaces will lose that spacing in the export process without the value being enclosed in quotes.

There's no such thing as an official CSV specification but a general rule is that all values may be quoted even if unnecessarily. I think the safest, easiest way to solve this problem is to quote every value.

To do so, I've changed function _webform_report_format_csv_column in webform_report.inc from the following:

function _webform_report_format_csv_column($value, $delim) {

  // if value contains a delimiter, it should be enclosed in quotes
  if (strpos($value, $delim) !== FALSE) {
      // if value contains quotes, double the quotes
      if (strpos($value, '"')) {
        return '"' . str_replace('"', '""', $value) . '"';
      }
      else {
        return '"' . $value . '"';
      }
  }
  return $value;
}

into:

function _webform_report_format_csv_column($value, $delim) {

  // if value contains quotes, double the quotes
  if (strpos($value, '"')) {
    return '"' . str_replace('"', '""', $value) . '"';
  }
  else {
    return '"' . $value . '"';
  }
  return $value;
}

So far it seems to work well.

My only problem is that when opened in Excel (after exporting with either Text or Excel option) those newlines appear as unknown characters (rectangle with question mark inside). I know that Excel can handle newlines in a CSV, so this must be an encoding issue. Unfortunately I'm not well versed enough in encoding to know how to solve this. Hopefully someone has a suggestion about the encoding, but even so this is better than a corrupted CSV.

Attached a patch against 6.x-2.0 with the above change.

Comments

msielski’s picture

StatusFileSize
new838 bytes

Uh oh, I messed up the return statements there. Here's the new version:

function _webform_report_format_csv_column($value, $delim) {

  // if value contains quotes, double the quotes
  if (strpos($value, '"')) {
    $value = str_replace('"', '""', $value);
  }
  return '"' . $value . '"';
}

New patch attached.

jimbullington’s picture

Thank you for the report and the patch. I wonder if you could try replacing CRLF with LF characters.

$value = str_replace("\r\n", "\n", $value);

That is mentioned here:

http://stackoverflow.com/questions/1241220/generating-csv-file-for-excel...

Also, did you try the Excel download option?

msielski’s picture

Thank you! It was apparently the CRs in Excel showing up as unrecognized. The replacement of CRLF to LF fixed it up. Do you think this should be added to the export code?

I was using both the Excel and Text options and both had the unrecognized character symbol. Both are now fixed with the substitution.

jimbullington’s picture

Yes - when you have time, roll up a patch with this change as well. When you feel confident in the code, I'll push to the repository.

msielski’s picture

StatusFileSize
new903 bytes

Ok, great. What I'm using is this:

function _webform_report_format_csv_column($value, $delim) {

  // Replace single quotes with double quotes, replace CRLF with LF
  $value = str_replace(array('"', "\r\n"),
                       array('""', "\n"),
                       $value);

  return '"' . $value . '"';
}

I removed the strpos test. I could be wrong but I think it's unnecessary to test for the string before attempting to replace it.

I'll be testing this with some other unrelated items on our dev site this week and will probably have a good idea how it's doing by the start of next.

msielski’s picture

I have tested this most recent patch and taken it into production recently. I'm happy with the way it works.

msielski’s picture

Status: Active » Needs review

Forgot to update status. I've reviewed my own patch, but should get maintainer or 3rd party review as well.

jimbullington’s picture

Status: Needs review » Reviewed & tested by the community

Works for me!

jimbullington’s picture

Status: Reviewed & tested by the community » Fixed

Committed patch to repository.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.