Git quotes any non-ascii character by default, not only asian ones. There’s an option to disable this quoting behaviour. You can disable it using the following command: git config –global core.quotepath false Or, alternatively, by adding the following snippet to your git config file ($HOME/.gitconfig usually) [core] quotepath = false After this, git should show … Read more
There is a way to ““deaccent” the string being compared” without the use of a substitution function that lists all the accents you want to remove… Here is the easiest solution I can think about to remove accents (and other diacritics) from a string. See it in action: var string = ‘Ça été Mičić. ÀÉÏÓÛ’; … Read more
Here is a solution for the single column search using PATINDEX. It also displays the StartPosition, InvalidCharacter and ASCII code. select line, patindex(‘%[^ !-~]%’ COLLATE Latin1_General_BIN,Line) as [Position], substring(line,patindex(‘%[^ !-~]%’ COLLATE Latin1_General_BIN,Line),1) as [InvalidCharacter], ascii(substring(line,patindex(‘%[^ !-~]%’ COLLATE Latin1_General_BIN,Line),1)) as [ASCIICode] from staging.APARMRE1 where patindex(‘%[^ !-~]%’ COLLATE Latin1_General_BIN,Line) >0
That character is U+2028 Line Separator, which is a kind of newline character. Think of it as the Unicode equivalent of HTML’s <br>. As to why it shows up here: my guess would be that an internal database uses LSEP to not conflict with literal newlines or HTML tags (which might break the database or … Read more
These days, a slightly better approach is to use the stringi package which provides a function for general unicode conversion. This allows you to preserve the original text as much as possible: x <- c(“Ekstr\u00f8m”, “J\u00f6reskog”, “bi\u00dfchen Z\u00fcrcher”) x #> [1] “Ekstrøm” “Jöreskog” “bißchen Zürcher” stringi::stri_trans_general(x, “latin-ascii”) #> [1] “Ekstrom” “Joreskog” “bisschen Zurcher”
You should define source code encoding, add this to the top of your script: # -*- coding: utf-8 -*- The reason why it works differently in console and in the IDE is, likely, because of different default encodings set. You can check it by running: import sys print sys.getdefaultencoding() Also see: Why declare unicode by … Read more
ASCII is in range of 0 to 127, so: str.replace(/[^\x00-\x7F]/g, “”);
I have tried all sorts based on the variations listed in the answers, but the following worked: $unwanted_array = array( ‘Š’=>’S’, ‘š’=>’s’, ‘Ž’=>’Z’, ‘ž’=>’z’, ‘À’=>’A’, ‘Á’=>’A’, ‘Â’=>’A’, ‘Ã’=>’A’, ‘Ä’=>’A’, ‘Å’=>’A’, ‘Æ’=>’A’, ‘Ç’=>’C’, ‘È’=>’E’, ‘É’=>’E’, ‘Ê’=>’E’, ‘Ë’=>’E’, ‘Ì’=>’I’, ‘Í’=>’I’, ‘Î’=>’I’, ‘Ï’=>’I’, ‘Ñ’=>’N’, ‘Ò’=>’O’, ‘Ó’=>’O’, ‘Ô’=>’O’, ‘Õ’=>’O’, ‘Ö’=>’O’, ‘Ø’=>’O’, ‘Ù’=>’U’, ‘Ú’=>’U’, ‘Û’=>’U’, ‘Ü’=>’U’, ‘Ý’=>’Y’, ‘Þ’=>’B’, ‘ß’=>’Ss’, ‘à’=>’a’, … Read more
This expression will search for non-ASCII values: [^\x00-\x7F]+ Tick off ‘Search Mode = Regular expression’, and click Find Next. Source: Regex any ASCII character
This will match a single non-ASCII character: [^\x00-\x7F] This is a valid PCRE (Perl-Compatible Regular Expression). You can also use the POSIX shorthands: [[:ascii:]] – matches a single ASCII char [^[:ascii:]] – matches a single non-ASCII char [^[:print:]] will probably suffice for you.**