I agree with Peter’s points. I don’t think we should be moving in the direction of a new, Unicode-aware string type to provide Unicode support. I don’t think C++ is going to move away from a world where, as far as standard string types go, effectively the only thing that the type system tells you is what type it uses for its code units. char_traits is essentially legacy; we shouldn’t be thinking about making changes to it. If users want more features out of their text types— for example, a type like Rust’s String that enforces UTF-8 validity, or a string with an associated runtime collation order— they can implement their own libraries that use the building blocks the standard library gives them. (This is my own view, which may or may not reflect SG16’s current consensus.)
I can also provide some additional details about my longer-term plans for standard library Unicode support.
I want to use the ranges library as the vehicle for providing additional Unicode functionality that users of standard string types can leverage.
So far my work on this has been scoped entirely to UTF transcoding (P2728), but Zach Laine also wrote two additional papers that I’m planning to try to revive once my work on P2728 is complete.
Those are P2729, which will handle normalization, and the unpublished draft D2745, which will handle grapheme clustering and word/sentence/line breaking.
In terms of how these proposals work with strings:
std::string (or std::string_view) that contains UTF-8 text to “a container of code points,” you’d just convert it to a std::u32string. With P2728’s transcoding facilities, you’d spell that as std::u32string u32s = my_string | views::as_char8_t | views::to_utf32 | ranges::to<std::u32string>().std::string (or std::string_view) that contains UTF-8 text to NFC, once I update P2729, I expect the API to be shaped something like std::string normalized = my_string | views::as_char8_t | views::to_nfc<char8_t> | views::as_char | ranges::to<std::string>().std::string first_word = my_string | views::as_char8_t | views::to_unicode_words | views::take(1) | views::join | views::as_char | ranges::to<std::string>().(Note that, for P2729 and P2745, my examples don’t reflect the current status of the paper, and are hypothetical APIs that will arrive in updated revisions.)
Thanks,
Eddie
Hi Alisdair,
On Tuesday, July 7th, 2026 at 6:15 PM, Alisdair Meredith via SG16 <sg16@lists.isocpp.org> wrote:
> Raising a distinct concern from the thread on best practice for Unicode today, does this group have a position on whether `basic_string` is an appropriate type for text processing in Unicode?
For text *storage*, it's the right type to use. It stores code units in whatever underlying character type it uses, and for u8/u16/u32 it's a Unicode type. All storage happens in terms of code units regardless.
For text *processing*, we almost always want to view them as code points, or higher-level constructs on top of that. Unicode properties that we want to have aren't necessarily retained in simple code unit changes, such as "I have a whole number of code points", "I have valid utf8", "my text is NFC normalized" or "concatenating two strings of X + Y grapheme clusters results in X+Y grapheme clusters". We need to have a view transform that provides this code point view from the underlying code unit storage, and we need to have a view transform that is usable for storing a code point range of some kind into a code unit storage.
With those fundamentals, we can build code point algorithms and grapheme algorithms on top of them.
> From my perspective, `basic_string` (and its type aliases) is a container of code units, where for text processing we want a container of code points. Likewise, is it parameterized on `char_traits` where we do not provide a traits type following Unicode rules, and in fact some things like collation order we would want to customize at runtime, but are baked into a compile-time decision through this traits parameter. The various functions that sound like processing text are not well suited to any multi-byte encoding system, not just Unicode,but as long as we live in ASCII plus the various codepage extensions native to our environment, we do not run into issues.
Char_traits is fundamentally broken and unsuitable for anything Unicode. We need to think of it as ossified. Collation is a property of a collection or sort invocation, not of a string type. The cstring_view proposal has char_traits so that it can convert from string to string_view through it without data loss, but beyond that SG16 agreed that it's useless.
> The counterpoint is that `std::string` has been our established vocabulary for the last 30 years, and `std::string_view` for the last 10. Does that weight of that legacy make it intractable to provide a “better” solution for a world that has slowly standardized on Unicode as its preferred encoding scheme (even if adherence to semantics like collation order may be less universal)?
They're okay storage types. They have a bunch of functions that have no Unicode meaning, and they have a lot of ossified features that we cannot modify and that serve no purpose, but any replacement at best would do the same thing modulo char_traits and string-like functions, and would necessarily face an uphill battle against the existing types that work just as well.
This is my understanding, please correct me if you disagree.
--
SG16 mailing list
SG16@lists.isocpp.org
https://lists.isocpp.org/mailman/listinfo.cgi/sg16
Link to this post: http://lists.isocpp.org/sg16/2026/07/4799.php