> In which way may my approach not be supported?
In whatever which way in the future a decision needs to be made that is incompatible with your usage.
Ex.
Should string literals with large Unicode code points be allowed? Nope, those are UB.
If I'm to create a plain transcoding function that converts from utf-8 to utf-16 which is the type used to represent utf-8? char8_t, char should not be supported. You want to use std::string? That's on you, you
can reinterpret_cast it if you want, but you are ultimately responsible to figure out what the right thing is, and it has been discussed your approach will not work on every system.
Essentially on any feature that we are yet to come up with in the future every time a decision needs to be made, char8_t is utf-8 and char isn't. What that is we may not know yet, but the decision is already
made.
> If neither IO streams nor std::print supports u8string, it is simply a show-stopper to me. I know many people who think the same way, including some committee members.
> A wild thought: maybe we should make u8string a derived class of string, and that will solve many usage problems. Of course, there are many implementation/compatibility issues too, so it is
just a thought at this moment.
Absolutely Not!
The introduction of char8_t has solved portability problems that char has. The wishy washiness of the arithmetic properties of char makes algorithms involving it not portable, char8_t was a god send for portability.
And I like that char8_t creates a clear type system distinction from char, so that I know which domain I’m operating at and what to expect.
Aliasing u8string to string, it wouldn’t just be inconvenient, it would actually break code.
The lack of IO and formatting support in the standard makes for a usability problem. But that is because the formatting and IO in the standard are bad, it’s not because char8_t is bad.
We should fix the bad parts, not throw way the good parts because they don ‘t work with the bad parts.
The analogy here is the current standard for io/formatting is like “banging up rocks together”, if all Grug does is bang up rocks together all day, handing them a screwdriver does not feel useful. Grug thinks
why can’t screwdriver be more like rock?
But what I’m trying to say is “let’s not bang rock, give ‘CAR’ a try”. But that is hard when “Grug never seen car”.
And I think that’s maybe the problem we are facing and there hasn’t been much progress. Everybody is bickering over small stuff and have different ideas about things, but nobody has seen what the “good thing
looks like”, so there’s no consensus over in which direction to move towards.
I don’t think the SG16 can fix it.
Maybe what we need is for a group of people to just go ahead make a library that works independent of the standard, have that be widely adopted, and when that happens then people will get it. And then people
will go like “oh, that’s how you are supposed to use it, we want to do like library X” and it clicks, and then you can adopt something in the standard because everyone gets it.
From: Yongwei Wu <wuyongwei@gmail.com>
Sent: Wednesday, July 8, 2026 3:26:27 AM
To: Tiago Freire <tmiguelf@hotmail.com>
Cc: sg16@lists.isocpp.org <sg16@lists.isocpp.org>; Christopher Nelson <nadiasvertex@gmail.com>; Tom Honermann <tom@honermann.net>
Subject: Re: [isocpp-sg16] UTF-8 support status
In which way may my approach not be supported? I do not see such a possibility. My approach does not in any way violate rules in the standard.
If neither IO streams nor std::print supports u8string, it is simply a show-stopper to me. I know many people who think the same way, including some committee members.
A wild thought: maybe we should make u8string a derived class of string, and that will solve many usage problems. Of course, there are many implementation/compatibility issues too, so it is just
a thought at this moment.
On Wed, 8 Jul 2026 at 01:53, Tiago Freire <tmiguelf@hotmail.com> wrote:
I think the topic is meandering a little bit.
The official position of the standard is char8_t for utf-8.
How you decide to use it is of course up to you, but it doesn't mean that your usage is going to be supported in the future.
No, I won't use std::u8string, as long as IO streams and std::print do not support it.
EBCDIC is a non-issue, and I do not care about it: I have never touched such a system in my 30+ year career. I am not expecting to port a program to such a system.
There are always trade-offs in life, and one chooses the lesser evil. Sticking to std::string and requiring everyone to use UTF-8 work the best.
On Tue, 7 Jul 2026 at 23:04, Tiago Freire via SG16 <sg16@lists.isocpp.org> wrote:
Correct me if I'm wrong but the internet of std::string is to be "system defined encoding" not even superset of ASCII. Ex. on an EBDIC system it's just EBDIC encoding.
std::u8string is the utf-8 one, and you should just use that if your text is utf-8.
I think a significant detail that we'd like to understand is: what does the committee intend std::string to mean in the future? In many places it seems like it is becoming idiomatic to say that std::string is utf-8. Which makes us wonder
what the design intent of the other string containers are. We're just trying to be good C++ citizens and not create more problems for ourselves in the future.
On Fri, Jul 3, 2026 at 11:00 AM Tom Honermann <tom@honermann.net> wrote:
I haven’t had a chance to review this thread yet and only a couple of regular attendees of SG16 meetings have weighed in so far. I suggest waiting at least a few more days for additional advice, especially considering the holiday weekend
in the US.
The summary below reflects one reasonable approach, but it is not the only one.
Tom.
On Jul 3, 2026, at 10:48 AM, Christopher Nelson via SG16 <sg16@lists.isocpp.org> wrote:
Thanks folks, I really appreciate all of the guidance.
The specific codebase we are working on is group of libraries that has bindings for several programming languages. C++(17/23/26) , C#, nodejs, and Rust. It runs on Linux, macOS, and Windows. We control all the applications it is embedded in.
May I summarize what I understood from the conversation and you can tell me if I am on the right track?
1. Use std::string
2. Assume that all internal string data is UTF-8
3. Enforce that invariant at input boundaries using helper functions
4. Be explicit about output encoding if it matters, default to UTF-8
5. Make formatting helpers that understand intent (display/file/etc.)
6. For Windows, explicitly opt into UTF-8 with the manifest flag (where possible) and compiler flag
7. Realize that you are mostly on your own here, so investing in a small library that will "do the right thing" for your app makes sense
Again, thank you so much for your time. I appreciate all the hard work you folks put into this.
On Fri, Jul 3, 2026 at 2:50 AM Tiago Freire <tmiguelf@hotmail.com> wrote:
I can’t speak for everybody, but I can tell you what I do.
On Windows apps I just set the console code page to be utf8. The exact method to do that has changed as some terminal applications have broken certain features. But you can do
it in 3 ways:
-
There’s a global Windows setting that sets the default windows code page buried deep in the localization options.
-
You can embed the preferred code page in the manifest file for the application
-
You can set the code page at runtime using a function
To be safe, you can do all 3, but ultimately it is going to depend on the terminal’s type setter to respect your preference (sometimes it doesn’t… *looks at windows terminal*).
As for “filesystem::path” they are not Unicode and never will be no matter how many times people say otherwise and it is best to acknowledge that upfront. To deal with that there
is not one but multiple strategies depending on the purpose of the “filesystem::path”.
If the purpose is for printing you can create a transcoding function that makes certain invalid Unicode characters to be visibly printable indicating that “there’s a character there
that has been altered for visibility but is not the actual character” and that makes “humans with eyes happy”.
If the purpose is to pass trough code that uses [bytes] char8_t to store or transfer things, but you would really like to perfectly preserve the path, there is a different transcoding
function that I use that makes the “path round-trip-able” that is “mostly utf8 compatible” but not truly utf8 in order to encode the invalid code points (i.e. UTF-16 surrogates are representable).
So, the right answer is “filesystem::path” is a path, not exactly text but text-like. And “what you do with it” depends on “what you want to achieve with it”, and Unicode in this
context is only relevant if you want to print it for a user to see and it is otherwise just bits.
As for “[it doesn’t] work with std::format and streams”, I’ve re-written my own output, formatting, and encoding library pipeline from scratch and I don’t suffer from such problems.
The C++ standard has come a long way but is comparatively speaking “still just banging rocks together”.
I can share more details regarding what I have done on this front, and even tough some concepts can be cannibalized to improve the standard, I’m afraid that the C++ status-quo has
painted itself too much into a corner that is a monumental task to even just start to fix it.
Best regards,
Tiago
Hi C++ Unicode folks, literally asking for a friend (Christopher, on the To: line so please Reply-All)…
His project team wants to use UTF-8 in their code base and they hoped to switch to u8string[_view] and char8_t throughout, but they encountered two sets of limitations:
-
In the standard, those types don’t seem to work with std::format and streams.
-
On Windows, platform APIs interpret narrow characters using the active code page (e.g., std::cout emits mojibake, filesystem::path{std::string} UTF-8 paths are mangled or throw).
What’s the current best guidance for adopting UTF-8 in C++ code?
Thank you,
Herb
--
SG16 mailing list
SG16@lists.isocpp.org
https://lists.isocpp.org/mailman/listinfo.cgi/sg16
Link to this post:
http://lists.isocpp.org/sg16/2026/07/4786.php
--
SG16 mailing list
SG16@lists.isocpp.org
https://lists.isocpp.org/mailman/listinfo.cgi/sg16
Link to this post:
http://lists.isocpp.org/sg16/2026/07/4792.php
--
Yongwei Wu
URL: http://wyw.dcweb.cn/
--
Yongwei Wu
URL: http://wyw.dcweb.cn/