I am currently refactoring filesystem::path_view::compare() along SG16's
feedback, and I want to check approval for the following algorithm. Be
aware that path_view can refer to source path data with the following
1. byte (don't interpret, just pass through as native filesystem encoding)
2. char (native narrow encoding)
3. wchar_t (native wide encoding)
4. char8_t (utf8)
5. char16_t (utf16)
The proposed comparison algorithm:
1. Splitting a path into components is done by
codepoint-value-comparison for '/' (and
filesystem::path::preferred_separator if not on POSIX). This is not
Unicode aware, and could be problematic on weird native encodings.
2. Each path component in between separator-codepoint-values is
individually compared as follows:
a) If the encodings are the same, via char_traits<CharT>::compare().
b) If the encodings are dissimilar, codecvt<CharT, char8_t> is used to
convert each non-char8_t source character to utf8. The char8_t values
are compared using char_traits<char8_t>::compare().
That's a problem -- the Unicode rules say that you cannot compare bytes to do less-than comparisons (which they term "collation"), or even for equality. Just for equality, you must normalize. Consider the combiner C1 and joiner J that have a combined code point C2 (so, concretely, say that C1 is "A", J is a combining circle, and C2 is the Angstrom symbol). You must not use comparisons in which result in C1J == C2 being false. Normalization gets you there.
Collation is even harder. There is a default collation, and it works for a lot of comparison operations just as you'd want. However, there are different levels of collation (there's one that considers everything -- probably what you want) and there are collations that work for different languages, and even variants of languages (the ordering of strings in the phonebook is different from the one you would use when searching within a document, for instance).
Did I mention that Unicode is hard?
I think you can just use the default collation, especially if you document that this is what you are doing. However, on some filesystems this might lead to the wrong answer -- FAT and NTFS and apparently now some Linux FSs are supposed to ignore case, right?
In any case, the algorithm above is not Unicode-friendly.
Sorry to be the bearer of this awfulest of news. Collation sucks.