Open up your file using regular old vanilla Notepad that comes with Windows. It will show you the encoding of the file when you click “Save As…“. It’ll look like this: Whatever the default-selected encoding is, that is what your current encoding is for the file. If it is UTF-8, you can change it to … Read more
Convert from String to byte[]: String s = “some text here”; byte[] b = s.getBytes(StandardCharsets.UTF_8); Convert from byte[] to String: byte[] b = {(byte) 99, (byte)97, (byte)116}; String s = new String(b, StandardCharsets.US_ASCII); You should, of course, use the correct encoding name. My examples used US-ASCII and UTF-8, two commonly-used encodings.
function _arrayBufferToBase64( buffer ) { var binary = ”; var bytes = new Uint8Array( buffer ); var len = bytes.byteLength; for (var i = 0; i < len; i++) { binary += String.fromCharCode( bytes[ i ] ); } return window.btoa( binary ); } but, non-native implementations are faster e.g. https://gist.github.com/958841 see http://jsperf.com/encoding-xhr-image-data/6 Updated benchmarks: https://jsben.ch/wnaZC
Yes, it’s frustrating—sometimes type and other programs print gibberish, and sometimes they do not. First of all, Unicode characters will only display if the current console font contains the characters. So use a TrueType font like Lucida Console instead of the default Raster Font. But if the console font doesn’t contain the character you’re trying … Read more
In Python 3, all strings are sequences of Unicode characters. There is a bytes type that holds raw bytes. In Python 2, a string may be of type str or of type unicode. You can tell which using code something like this: def whatisthis(s): if isinstance(s, str): print “ordinary string” elif isinstance(s, unicode): print “unicode … Read more
You can’t detect the codepage, you need to be told it. You can analyse the bytes and guess it, but that can give some bizarre (sometimes amusing) results. I can’t find it now, but I’m sure Notepad can be tricked into displaying English text in Chinese. Anyway, this is what you need to read: The … Read more
Your ”.join() expression is filtering, removing anything non-ASCII; you could use a conditional expression instead: return ”.join([i if ord(i) < 128 else ‘ ‘ for i in text]) This handles characters one by one and would still use one space per character replaced. Your regular expression should just replace consecutive non-ASCII characters with a space: … Read more
… is the unicode for “…” so just replace it. It’s better to have it as one char/symbol than three dots.
Here’s a nice fun LINQ example. public static byte[] StringToByteArray(string hex) { return Enumerable.Range(0, hex.Length) .Where(x => x % 2 == 0) .Select(x => Convert.ToByte(hex.Substring(x, 2), 16)) .ToArray(); }
If you apply utf8_encode() to an already UTF-8 string, it will return garbled UTF-8 output. I made a function that addresses all this issues. It´s called Encoding::toUTF8(). You don’t need to know what the encoding of your strings is. It can be Latin1 (ISO 8859-1), Windows-1252 or UTF-8, or the string can have a mix … Read more