Provide correct CSV encoding and headers

The mb_convert_encoding thing worked flawlessly. I have tested on Excel 2003 SP2.
Thanks!

$value = mb_convert_encoding($value, "Windows-1252")

This worked for me too.
Didn't do extensive testing... but it works for me.
Office 2000

Does this work for other CSV-reading apps? Numbers? OpenOffice?

Also, it would be nice if someone could provide a patch.

What are the changes we need to do to the module to have an exportale excel friendy version???
Patchak

#344875: Update CSV plugin to use RFC 4180 text/csv MIME type. for the MIME part.
As far as the separator goes... In 6-dev this is fixed by providing the seperator as an option.

In 5 you'll need to overide the theme this is important. I think the default should stick to commas as the file type name implies.

I'm not sure about using the encoding stuff yet. I'm hoping using the utf8 encoding on text/csv will fix any oddness surrounding that. If this turns out to be false we'll deal with it then.

Just to confirm the mime type encoding is not enough for Excel (2007 SP1), but the mb_convert_encoding trick works fine

--- views_bonus_export.theme.inc.orig   2009-09-08 17:48:49.000000000 +0200
+++ views_bonus_export.theme.inc        2009-09-08 17:42:19.000000000 +0200
@@ -28,6 +28,7 @@

   foreach ($vars['themed_rows'] as $i => $values) {
     foreach ($values as $j => $value) {
+      $value = mb_convert_encoding($value, "Windows-1252");
       $vars['themed_rows'][$i][$j] = $wrap . str_replace('"', $replace_value, decode_entities(strip_tags($value))) . $wrap;
     }
   }

#8 worked for me exporting to excel 2003 sp3.

#8 Works perfectly here as well. Only now (in 6.x-1.0) this needs to be done in views_bonus_export.module (not in views_bonus_export.theme.inc). Could it be applied to the module? (I am marking this RTBC so that it is not overlooked, even though this is a solution to only part of the problem discussed above.)

Adjusting the title.

Y reopen this? The encoding issue is fixed in a separate issue and the separator is handled in the Views UI.

Its a no fix.

@Scott Reynolds - Encoding did not work in this version and I spent several days testing and tinkering. Eventually I solved the problem perfectly as described in #10 above. If you say this was fixed elsewhere in a different or more comprehensive manner, please do provide precise pointers and/or explain why "won't fix". Thank you.

A client stuck on Windows has reported similar problems to me. I agree that re-encoding to Windows-1252 doesn't make sense. Now that the content type header declares the file as UTF-8 this in theory should work everywhere but apparently Windows needs the redundant UTF-8 byte order mark to correctly read these files *sigh*

I propose the following trivial patch, which prints the BOM at the beginning of the template. If this is considered harmful I think it should be at least made an option for the CSV format but I won't write that code for now.

Reference: http://en.wikipedia.org/wiki/Byte_order_mark

patch in #14 doesn't fox in windows for me. Instead #1 technique does work for Excel but breaks in openoffice, i think because some kind of double encoding because wrong chars are as rhombus (�).

This seems fairly reasonable and I'm leaning toward committing it. Much better solution for sure. Documentation of BOMs does suggest that they may be needed in files like plain text where the encoding may not be detected or properly handled by some programs. This seems like exactly that case and we aren't going to be doing things like parsing this with a php compiler that would blow up.

I'd like to get some review by someone that might be more familiar with BOMs/UTF than I am though and any possible side effects.

I have created two small sample CSV output files with and without the BOM to help people test this in Windows without having to apply this patch. The version without BOM was created by this module before my patch, and the version with BOM was created with the patch. If these work for you than my approach is correct.

Both are text files in MS DOS format (CRLF line endings) and other than the BOM contain only character in the US ASCII character range (0-127).

Neither of them show correctly in MS Excel 2000 in my Windows Vista 64b.

So far, for me, the only working solution has been #8 (see #10).

While I don't have access to a Windows machine to test this further, I would like to very strongly suggest that if possible this module *not* apply the patch in #8 since it changes the character set to Windows-1252 despite declaring the character set UTF-8 in the headers. I suspect this will break every operating system which actually respects the character set declaration in the header (apparently, everything except some versions of Windows). I don't think is would be sufficient to make this a configuration option for the View, since not all users will be using the same operating system. This would have to be set per-user or by detecting the operating system based on user agent, which of course would be annoying, difficult, and error prone.

I leave it to Windows users to come up with a solution here that will respect standards and work for everyone, and am happy to test any sample files posted in this issue.

I have no intention of applying #8 because it is just wrong. If we're providing utf-8 data, we're providing utf-8 data. I don't have a vista machine with office so that'll be difficult but I do have a 7 secondary boot with a recent version of office(newer that 2000 I'm certain unfortunately) that I can do some testing on at some point.

#14 works here. Windows XP SP3, Excel 2007

Thanks just tested #14 and it works for me too (Excel 2007 on Vista).
Will be testing it with our corporate customers and report if there are issues.
Also agreed that it's a cleaner approach than the encoding workaround, would love to see a solution commited...

Subscribe. Using OpenOffice to convert cvs export for excel while waiting for this fix.

Also tested the patch in #14 and it solves the problem with Excel 2007 / Windows 7.