更新时间:2022-10-15 16:33:13
UTF-8 中的所有单字节编码字符都必须在 [0x00 .. 0x7F] (https://en.wikipedia.org/wiki/UTF-8).这些相当于 7 位 ASCII.
对于德语 ß
,你会得到 2 个 UTF-8 字节:
b'\xc3\x9f'
解码时也能正常工作:
b'\xc3\x9f'.decode("utf-8")
'ß'
I have a bytestring b"\xDF"
. When I try to decode it to UTF-8, a UnicodeDecodeError is thrown. Decoding to CP1252 works fine. In both charsets, 0xDF is represented by the character "ß". So why the Error?
>>> hex(ord("ß"))
'0xdf'
>>> b"\xDF".decode("utf-8")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xdf in position 0: unexpected end of data
>>> b"\xDF".decode("cp1252")
'ß'
All single-byte encoded characters in UTF-8 have to be in the range [0x00 .. 0x7F] (https://en.wikipedia.org/wiki/UTF-8). Those are equivalent to 7-bit ASCII.
For the german ß
, you'd get 2 bytes in UTF-8:
>>> "ß".encode("utf-8")
b'\xc3\x9f'
Which also works correctly when decoding:
b'\xc3\x9f'.decode("utf-8")
'ß'