Date: Tue, 7 Jul 2026 12:15:04 -0400
Raising a distinct concern from the thread on best practice for Unicode today, does this group have a position on whether `basic_string` is an appropriate type for text processing in Unicode?
From my perspective, `basic_string` (and its type aliases) is a container of code units, where for text processing we want a container of code points. Likewise, is it parameterized on `char_traits` where we do not provide a traits type following Unicode rules, and in fact some things like collation order we would want to customize at runtime, but are baked into a compile-time decision through this traits parameter. The various functions that sound like processing text are not well suited to any multi-byte encoding system, not just Unicode,but as long as we live in ASCII plus the various codepage extensions native to our environment, we do not run into issues.
The counterpoint is that `std::string` has been our established vocabulary for the last 30 years, and `std::string_view` for the last 10. Does that weight of that legacy make it intractable to provide a “better” solution for a world that has slowly standardized on Unicode as its preferred encoding scheme (even if adherence to semantics like collation order may be less universal)?
I am sure we have multiple answers to this in past papers, I am mostly trying to catch up on what our current consensus might be closer to.
AlisdairM
From my perspective, `basic_string` (and its type aliases) is a container of code units, where for text processing we want a container of code points. Likewise, is it parameterized on `char_traits` where we do not provide a traits type following Unicode rules, and in fact some things like collation order we would want to customize at runtime, but are baked into a compile-time decision through this traits parameter. The various functions that sound like processing text are not well suited to any multi-byte encoding system, not just Unicode,but as long as we live in ASCII plus the various codepage extensions native to our environment, we do not run into issues.
The counterpoint is that `std::string` has been our established vocabulary for the last 30 years, and `std::string_view` for the last 10. Does that weight of that legacy make it intractable to provide a “better” solution for a world that has slowly standardized on Unicode as its preferred encoding scheme (even if adherence to semantics like collation order may be less universal)?
I am sure we have multiple answers to this in past papers, I am mostly trying to catch up on what our current consensus might be closer to.
AlisdairM
Received on 2026-07-07 16:15:20
