On 8/26/25 9:30 PM, Thiago Macieira via Std-Proposals wrote:
> On Tuesday, 26 August 2025 18:04:33 Pacific Daylight Time Oliver Hunt wrote:
>> I don’t know about rust, but with the exception of the NSString bridge
>> (which I assume does something exciting, because bridging), Swift operates
>> entirely in utf-8. In addition to that I know many string implementations
>> that are ostensibly utf16 or ucs2 use either ascii (not utf8 - this is
>> generally because they’re often stuck with some char16* API and
>> utf8->utf16->utf8 is expensive) or utf16 internally depending on the
>> content.
> Would you be able to find out how Swift implements collation? I don't know
> where to begin the search. For Rust, it appears to be ICU4X [1] which is a
> full reimplementation of ICU4C in Rust.
>
> Either way, if I am writing C++ platform, Swift or Rust don't matter much. My
> options are to use either ICU or an OS-provided framework. The latter case on
> an Apple platform is either UCCompareText[2], which operates on UniChar
> arrays, or NSString's localizedCompare[3]. Likewise for Windows: you'd either
> use ICU or CompareStringEx[4] - a wchar_t-only API, there's no "A" version.
>
> So if you're dealing with Unicode today, you are using UTF-16 almost without
> exception. The C++ Standard's Unicode support needs to take this legacy into
> account. Forcing everyone to use char8_t is not the way.

I'm obviously biased towards char8_t, but I agree 100%. There are many
existing non-legacy applications that use UTF-16 internally and they
aren't going to disappear or be rewritten any time soon.

Tom.

>
> [1] <https://github.com/unicode-org/icu4x
> [2]
> https://developer.apple.com/documentation/coreservices/1390642-uccomparetext
> [3] https://developer.apple.com/documentation/foundation/nsstring/
> localizedcompare(_:)?language=objc
> [4] https://learn.microsoft.com/en-us/windows/win32/api/stringapiset/nf-stringapiset-comparestringex
>
>