Remove a BOM character in a file
If you look in the same menu. Click “Convert to UTF-8.”
If you look in the same menu. Click “Convert to UTF-8.”
Using VIM Open file in VIM: vi text.xml Remove BOM encoding: :set nobomb Save and quit: :wq For a non-interactive solution, try the following command line: vi -c “:set nobomb” -c “:wq” text.xml That should remove the BOM, save the file and quit, all from the command line.
With ruby >= 1.9.2 you can use the mode r:bom|utf-8 This should work (I haven’t test it in combination with json): json = nil #define the variable outside the block to keep the data File.open(‘file.txt’, “r:bom|utf-8”){|file| json = JSON.parse(file.read) } It doesn’t matter, if the BOM is available in the file or not. Andrew remarked, … Read more
It turns out, the answer is that what I’m seeing is a Byte Order Mark, which is a character that tells whatever is loading the document what it is encoded in. In my case, it’s encoded in utf-8, so the corresponding BOM was EF BB BF, as shown below. To remove it, I opened it … Read more
You have to tell open that this is UTF-8 with BOM. I know that works with io.open: import io . . . inputFile = io.open(“test.csv”, “r”, encoding=’utf-8-sig’) . . . And you have to open the file in text mode, “r” instead of “rb”.
Setting coding-system-for-write directly is a bit of a hack. The user command for changing a file’s encoding (in this case from utf-8-with-signature to plain utf-8) is set-buffer-file-coding-system. It is bound by default to C-x RET f.
With ruby 1.9.2 you can use the mode r:bom|utf-8 text_without_bom = nil #define the variable outside the block to keep the data File.open(‘file.txt’, “r:bom|utf-8”){|file| text_without_bom = file.read } or text_without_bom = File.read(‘file.txt’, encoding: ‘bom|utf-8’) or text_without_bom = File.read(‘file.txt’, mode: ‘r:bom|utf-8′) It doesn’t matter, if the BOM is available in the file or not. You may … Read more
Use an XmlTextWriter and pass that to the XDocument’s Save() method, that way you can have more control over the type of encoding used: var doc = new XDocument( new XDeclaration(“1.0”, “utf-8”, null), new XElement(“root”, new XAttribute(“note”, “boogers”)) ); using (var writer = new XmlTextWriter(“.\\boogers.xml”, new UTF8Encoding(false))) { doc.Save(writer); } The UTF8Encoding class constructor has … Read more
Try like this: public ActionResult Download() { var data = Encoding.UTF8.GetBytes(“some data”); var result = Encoding.UTF8.GetPreamble().Concat(data).ToArray(); return File(result, “application/csv”, “foo.csv”); } The reason is that the UTF8Encoding constructor that takes a boolean parameter doesn’t do what you would expect: byte[] bytes = new UTF8Encoding(true).GetBytes(“a”); The resulting array would contain a single byte with the value … Read more