EDIT: I’ve made a proper release on GitHub: https://github.com/gpakosz/UnicodeBOMInputStream Here is a class I coded a while ago, I just edited the package name before pasting. Nothing special, it is quite similar to solutions posted in SUN’s bug database. Incorporate it in your code and you’re fine. /* ____________________________________________________________________________ * * File: UnicodeBOMInputStream.java * Author: … Read more
Three words for you: Byte Order Mark (BOM) That’s the representation for the UTF-8 BOM in ISO-8859-1. You have to tell your editor to not use BOMs or use a different editor to strip them out. To automatize the BOM’s removal you can use awk as shown in this question. As another answer says, the … Read more
BOM or Byte Order Mark is sometimes quite annoying. Visual Studio does not change the file unless you save it (as Hans said). And here is the solution to your problem: If you want to save a file with other encodings, select save as and extend the save button in file dialog and select “Save … Read more
I have the same (or similar) problem. In my case, if I add a BOM to the output, it works: header(‘Content-Encoding: UTF-8’); header(‘Content-type: text/csv; charset=UTF-8’); header(‘Content-Disposition: attachment; filename=Customers_Export.csv’); echo “\xEF\xBB\xBF”; // UTF-8 BOM I believe this is a pretty ugly hack, but it worked for me, at least for Excel 2007 Windows. Not sure it’ll … Read more
I believe the problem is that codecs.BOM_UTF8 is a byte string, not a Unicode string. I suspect the file handler is trying to guess what you really mean based on “I’m meant to be writing Unicode as UTF-8-encoded text, but you’ve given me a byte string!” Try writing the Unicode string for the byte order … Read more
Using .NET’s UTF8Encoding class and passing $False to the constructor seems to work: $MyRawString = Get-Content -Raw $MyPath $Utf8NoBomEncoding = New-Object System.Text.UTF8Encoding $False [System.IO.File]::WriteAllLines($MyPath, $MyRawString, $Utf8NoBomEncoding)
The UTF-8 BOM is a sequence of bytes at the start of a text stream (0xEF, 0xBB, 0xBF) that allows the reader to more reliably guess a file as being encoded in UTF-8. Normally, the BOM is used to signal the endianness of an encoding, but since endianness is irrelevant to UTF-8, the BOM is … Read more
The UTF-8 BOM is a sequence of bytes at the start of a text stream (0xEF, 0xBB, 0xBF) that allows the reader to more reliably guess a file as being encoded in UTF-8. Normally, the BOM is used to signal the endianness of an encoding, but since endianness is irrelevant to UTF-8, the BOM is … Read more