Filename encodings on Darwin
Describe the feature
Use correct filename encoding on Darwin
Is there a prototype?
No.
Describe the feature in more detail
I think the general plan is to change the default filename encoding
from :iso8859-1
to :utf-8
everywhere. This should work fine on
Linux, FreeBSD, and Solaris.
But Darwin is quite complicated.
Some links explaining how it's done:
- https://developer.apple.com/library/archive/technotes/tn/tn1150.html#UnicodeSubtleties
- https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/APFS_Guide/FAQ/FAQ.html
- https://web.archive.org/web/20140812023313/http://developer.apple.com/library/ios/documentation/MacOSX/Conceptual/BPInternational/Articles/FileEncodings.html
- https://eclecticlight.co/2021/05/08/explainer-unicode-normalization-and-apfs/
In particular the third link says
Mac OS Extended (HFS+) uses canonically decomposed Unicode 3.2 in UTF-16 format, which consists of a sequence of 16-bit codes. (Characters in the ranges U2000-U2FFF, UF900-UFA6A, and U2F800-U2FA1D are not decomposed.)
Presumably on x86 machines it's UTF-16 little-endian.
The first link says
In addition, the Korean Hangul characters with codes in the range u+AC00 through u+D7A3 are illegal and must be replaced with the equivalent sequence of conjoining jamos, as described in the Unicode 2.0 book, section 3.10.
This makes it pretty complicated. For now maybe it's good enough just to do UTF-16.
However, I think we're using :iso8859-1
which seems to work, so I'm
not really sure how this works at all....