I found the same problem with spanish, solved it with with “latin1” encoding: import pandas as pd pd.read_csv(“Openhealth_S-Grippal.csv”,delimiter=”;”, encoding=’latin1′) Hope it helps!
see capture file below for a detailed explanation. Hope this helps (if still relevant)…
Can use other option again: string value = “\u00C4 \uD802\u0033 \u00AE”; byte[] bytes= System.Text.Encoding.UTF8.GetBytes(value); For more information can look on Encoding.UTF8 Property
This is possible since Python 3.3: import gzip gzip.open(‘file.gz’, ‘rt’, encoding=’utf-8′) Notice that gzip.open() requires you to explicitly specify text mode (‘t’).
I think that your MYSQLdb python library doesn’t know it’s supposed to encode to utf8, and is encoding to the default python system-defined charset latin1. When you connect() to your database, pass the charset=”utf8″ parameter. This should also make a manual SET NAMES or SET character_set_client unnecessary.
There is no charset parameter defined for this media type. For the encoding guidelines, see https://url.spec.whatwg.org/#application/x-www-form-urlencoded . The application/x-www-form-urlencoded standard implies UTF-8 and percent-encoding. Though: A legacy server-oriented implementation might have to support encodings other than UTF-8 as well as have special logic for tuples of which the name is _charset. Such logic is not … Read more
Check http://www.regular-expressions.info/unicode.html and http://xregexp.com/plugins/ You would need to use \p{L} to match any letter character if you want to include unicode. Speaking unicode, alternative of \w is [\p{L}\p{N}_] then.