I want to allow any valid UTF-8 characters in usernames and passwords. Abandon all hope. Basic Authentication and Unicode don’t mix. There is no standard(*) for how to encode non-ASCII characters into a Basic Authentication username:password token before base64ing it. Consequently every browser does something different: Opera uses UTF-8; IE uses the system’s default codepage … Read more
I got it solved as the following. fileName = dateString+”_マイページ情報.xls”; fileName = URLEncoder.encode(fileName,”UTF-8″); try { response.setContentType(“application/ms-excel; charset=UTF-8”); response.setCharacterEncoding(“UTF-8”); if(browserType.equals(“IE”)||browserType.equals(“Chrome”)) response.setHeader(“Content-Disposition”,”attachment; filename=”+fileName); if(browserType.endsWith(“Firefox”)) response.setHeader(“Content-Disposition”,”attachment; filename*=UTF-8””+fileName); } catch (Exception e1) { // TODO Auto-generated catch block e1.printStackTrace(); }
Upgrade to Ruby 2.0, as it makes UTF-8 the default encoding, removing the need for magic comments.
— is wrong. When you use numeric character references, the number refers to the Unicode codepoint. For numbers below 256 that is the same as the codepoint in ISO-8859-1. In 8859-1, character 151 is amongst the “C1 control codes”, and not a dash or any other visible character. The confusion arises because character 151 is … Read more
ALTER TABLE tbl_name CONVERT TO CHARACTER SET utf8;
There are a bunch: LF: Line Feed, U+000A (UTF-8 in hex: 0A) VT: Vertical Tab, U+000B (UTF-8 in hex: 0B) FF: Form Feed, U+000C (UTF-8 in hex: 0C) CR: Carriage Return, U+000D (UTF-8 in hex: 0D) CR+LF: CR (U+000D) followed by LF (U+000A) (UTF-8 in hex: 0D 0A) NEL: Next Line, U+0085 (UTF-8 in hex: … Read more
I use this simple and sturdy approach: function cleanString(input) { var output = “”; for (var i=0; i<input.length; i++) { if (input.charCodeAt(i) <= 127) { output += input.charAt(i); } } return output; } Basically all you really want are the ASCII chars 0-127 so just rebuild the string char by char. If it’s a good … Read more
Try LOAD DATA INFILE ‘file’ IGNORE INTO TABLE table CHARACTER SET UTF8 FIELDS TERMINATED BY ‘;’ OPTIONALLY ENCLOSED BY ‘”‘ LINES TERMINATED BY ‘\n’
As the other poster mentioned, you might try: df = pd.read_csv(‘1459966468_324.csv’, encoding=’utf8′) However this could still leave you looking at ‘object’ when you print the dtypes. To confirm they are utf8, try this line after reading the CSV: df.apply(lambda x: pd.lib.infer_dtype(x.values)) Example output: args unicode date datetime64 host unicode kwargs unicode operation unicode
>>> c=”\xe7\x9a\x84″.decode(‘utf8′) >>> c u’\u7684’ >>> print c 的 though Unicode encodes it in 16 bits, utf8 breaks it down to 3 bytes.