Skip to content
Text & Encoding
-
Text & Encoding
NFC vs NFD: Unicode Normalization, Mac Files and Zips
Why "café" == "café" can be False: NFC, NFD, NFKC and NFKD measured in Python and PHP, what APFS really stores, Mac zips without the UTF-8 flag, and git.
-
Text & Encoding
Encoding Detection Is Guesswork: Accuracy and Speed
mb_detect_encoding and a stdlib guesser tested on 9,000 samples: right on long text, wrong on short. Plus 10 MB conversion speed and the chunk-split decode bug.
-
Text & Encoding
Characters, Code Points and UTF-8 Bytes, Seen in xxd
Why len('café') is 4 but the file is 5 bytes: code points vs UTF-8 bytes in xxd, Python len vs PHP strlen, a hand-written decoder and real size measurements.
-
Text & Encoding
String Length in 8 Languages: Emoji, Graphemes, Bytes
What length() returns for café, 👍🏽, 👨👩👧 and 🇯🇵 in Python, JS, Swift, Dart, Java, C#, PHP and SQLite, measured, plus truncation bugs and DB column limits.
-
Text & Encoding
Mojibake: Why ’ Appears and How to Fix It in Python
Reproduce ’ and é on purpose, read the pattern, repair double-encoded text with encode/decode, and measure which mojibake is reversible.
-
Text & Encoding
CSV Encoding: UTF-8, the BOM and Excel, Byte by Byte
Write the same CSV as UTF-8, UTF-8 with BOM and cp1252, read it back with Python, PHP and sqlite3, measure the BOM's side effects and pick the right encoding.
↑