The StreamReader.CurrentEncoding property rarely returns the correct text file encoding for me. I’ve had greater success determining a file’s endianness, by analyzing its byte order mark (BOM). If the file does not have a BOM, this cannot determine the file’s encoding. *UPDATED 4/08/2020 to include UTF-32LE detection and return correct encoding for UTF-32BE /// <summary> … Read more
As per RFC 1738: Unsafe: Characters can be unsafe for a number of reasons. The space character is unsafe because significant spaces may disappear and insignificant spaces may be introduced when URLs are transcribed or typeset or subjected to the treatment of word-processing programs. The characters “<” and “>” are unsafe because they are used … Read more
It is common to simply swap alphabet for use in urls, so that no %-encoding is necessary; only 3 of the 65 characters are problematic – +, / and =. the most common replacements are – in place of + and _ in place of /. As for the padding: just remove it (the =); … Read more
You cannot determine the encoding of a arbitrary byte stream. This is the nature of encodings. A encoding means a mapping between a byte value and its representation. So every encoding “could” be the right. The getEncoding() method will return the encoding which was set up (read the JavaDoc) for the stream. It will not … Read more
Uri.EscapeDataString or HttpUtility.UrlEncode is the correct way to escape a string meant to be part of a URL. Take for example the string “Stack Overflow”: HttpUtility.UrlEncode(“Stack Overflow”) –> “Stack+Overflow” Uri.EscapeUriString(“Stack Overflow”) –> “Stack%20Overflow” Uri.EscapeDataString(“Stack + Overflow”) –> Also encodes “+” to “%2b” —->Stack%20%2B%20%20Overflow Only the last is correct when used as an actual part of … Read more
I noticed the exact same issue when logging onto servers running Red Hat from an OSX Lion machine. Try adding or editing the ~/.profile file for it to correctly export your locale settings upon initiating a new session. export LC_ALL=en_US.UTF-8 export LANG=en_US.UTF-8 These two lines added to the file should suffice to set the locale … Read more
Your conclusion that padding is unnecessary is right. It’s always possible to determine the length of the input unambiguously from the length of the encoded sequence. However, padding is useful in situations where base64 encoded strings are concatenated in such a way that the lengths of the individual sequences are lost, as might happen, for … Read more
So what’s the problem, It’s a ’ (RIGHT SINGLE QUOTATION MARK – U+2019) character which is being decoded as CP-1252 instead of UTF-8. If you check the encodings table, then you see that this character is in UTF-8 composed of bytes 0xE2, 0x80 and 0x99. If you check the CP-1252 code page layout, then you’ll … Read more
This way of specifying the encoding of a Python file comes from PEP 0263 – Defining Python Source Code Encodings. It is also recognized by GNU Emacs (see Python Language Reference, 2.1.4 Encoding declarations), though I don’t know if it was the first program to use that syntax.
Encoding transforms data into another format using a scheme that is publicly available so that it can easily be reversed. Encryption transforms data into another format in such a way that only specific individual(s) can reverse the transformation. For Summary – Encoding is for maintaining data usability and uses schemes that are publicly available. Encryption … Read more