> On Jul 7, 2026, at 12:44 PM, Peter Bindels <cpp_at_[hidden]> wrote:
>
> Hi Alisdair,
>
> On Tuesday, July 7th, 2026 at 6:15 PM, Alisdair Meredith via SG16 <sg16_at_[hidden]> wrote:
>
>> Raising a distinct concern from the thread on best practice for Unicode today, does this group have a position on whether `basic_string` is an appropriate type for text processing in Unicode?
>
> For text *storage*, it's the right type to use. It stores code units in whatever underlying character type it uses, and for u8/u16/u32 it's a Unicode type. All storage happens in terms of code units regardless.
>
> For text *processing*, we almost always want to view them as code points, or higher-level constructs on top of that. Unicode properties that we want to have aren't necessarily retained in simple code unit changes, such as "I have a whole number of code points", "I have valid utf8", "my text is NFC normalized" or "concatenating two strings of X + Y grapheme clusters results in X+Y grapheme clusters". We need to have a view transform that provides this code point view from the underlying code unit storage, and we need to have a view transform that is usable for storing a code point range of some kind into a code unit storage.
>
> With those fundamentals, we can build code point algorithms and grapheme algorithms on top of them.
>
>> From my perspective, `basic_string` (and its type aliases) is a container of code units, where for text processing we want a container of code points. Likewise, is it parameterized on `char_traits` where we do not provide a traits type following Unicode rules, and in fact some things like collation order we would want to customize at runtime, but are baked into a compile-time decision through this traits parameter. The various functions that sound like processing text are not well suited to any multi-byte encoding system, not just Unicode,but as long as we live in ASCII plus the various codepage extensions native to our environment, we do not run into issues.
>
> Char_traits is fundamentally broken and unsuitable for anything Unicode. We need to think of it as ossified. Collation is a property of a collection or sort invocation, not of a string type. The cstring_view proposal has char_traits so that it can convert from string to string_view through it without data loss, but beyond that SG16 agreed that it's useless.
>
>> The counterpoint is that `std::string` has been our established vocabulary for the last 30 years, and `std::string_view` for the last 10. Does that weight of that legacy make it intractable to provide a “better” solution for a world that has slowly standardized on Unicode as its preferred encoding scheme (even if adherence to semantics like collation order may be less universal)?
>
> They're okay storage types. They have a bunch of functions that have no Unicode meaning, and they have a lot of ossified features that we cannot modify and that serve no purpose, but any replacement at best would do the same thing modulo char_traits and string-like functions, and would necessarily face an uphill battle against the existing types that work just as well.
>
>
> This is my understanding, please correct me if you disagree.

Thanks, that sounds like a good clarification on top of my understanding.

Would we be interested in a type with a smaller, simpler interface named for such purpose, e.g., `u8_code_untils`, that wrapped a `basic_string<char8_t>` and could convert to a reference to the `basic_string`? Removing confusing distraction form the interface is a feature for folks trying to understand where we are toda without the baggage of how we got here.

AlisdairM