Date: Tue, 7 Jul 2026 14:00:55 -0400
Thank you, everyone.
I think from our standpoint the concern we had was that it appeared that
the committee might be reconsidering the uXXstring classes and backing away
from supporting them in the future. If I understand correctly, the general
view is that there will be more friction at this point because things are
still in development. But long term, the uXXstring classes are going to be
fully supported so if we use them now it's not likely something we will
have to undo in the future. The experience will simply get better over time.
Is that fair?
On Tue, Jul 7, 2026 at 1:53 PM Tiago Freire <tmiguelf_at_[hidden]> wrote:
> I think the topic is meandering a little bit.
>
> The official position of the standard is char8_t for utf-8.
> How you decide to use it is of course up to you, but it doesn't mean that
> your usage is going to be supported in the future.
>
>
>
> ------------------------------
> *From:* Yongwei Wu <wuyongwei_at_[hidden]>
> *Sent:* Tuesday, July 7, 2026 6:41:13 PM
> *To:* sg16_at_[hidden] <sg16_at_[hidden]>
> *Cc:* Christopher Nelson <nadiasvertex_at_[hidden]>; Tom Honermann <
> tom_at_[hidden]>; Tiago Freire <tmiguelf_at_[hidden]>
> *Subject:* Re: [isocpp-sg16] UTF-8 support status
>
> No, I won't use std::u8string, as long as IO streams and std::print do not
> support it.
>
> EBCDIC is a non-issue, and I do not care about it: I have never touched
> such a system in my 30+ year career. I am not expecting to port a program
> to such a system.
>
> There are always trade-offs in life, and one chooses the lesser evil.
> Sticking to std::string and requiring everyone to use UTF-8 work the best.
>
> On Tue, 7 Jul 2026 at 23:04, Tiago Freire via SG16 <sg16_at_[hidden]>
> wrote:
>
> Correct me if I'm wrong but the internet of std::string is to be "system
> defined encoding" not even superset of ASCII. Ex. on an EBDIC system it's
> just EBDIC encoding.
> std::u8string is the utf-8 one, and you should just use that if your text
> is utf-8.
>
>
> ------------------------------
> *From:* Christopher Nelson <nadiasvertex_at_[hidden]>
> *Sent:* Tuesday, July 7, 2026 4:59:17 PM
> *To:* Tom Honermann <tom_at_[hidden]>
> *Cc:* sg16_at_[hidden] <sg16_at_[hidden]>; Tiago Freire <
> tmiguelf_at_[hidden]>
> *Subject:* Re: [isocpp-sg16] UTF-8 support status
>
> I think a significant detail that we'd like to understand is: what does
> the committee intend std::string to mean in the future? In many places it
> seems like it is becoming idiomatic to say that std::string is utf-8. Which
> makes us wonder what the design intent of the other string containers
> are. We're just trying to be good C++ citizens and not create more problems
> for ourselves in the future.
>
> On Fri, Jul 3, 2026 at 11:00 AM Tom Honermann <tom_at_[hidden]> wrote:
>
> I haven’t had a chance to review this thread yet and only a couple of
> regular attendees of SG16 meetings have weighed in so far. I suggest
> waiting at least a few more days for additional advice, especially
> considering the holiday weekend in the US.
>
> The summary below reflects one reasonable approach, but it is not the only
> one.
>
> Tom.
>
> On Jul 3, 2026, at 10:48 AM, Christopher Nelson via SG16 <
> sg16_at_[hidden]> wrote:
>
>
> Thanks folks, I really appreciate all of the guidance.
>
> The specific codebase we are working on is group of libraries that has
> bindings for several programming languages. C++(17/23/26) , C#, nodejs, and
> Rust. It runs on Linux, macOS, and Windows. We control all the applications
> it is embedded in.
>
> May I summarize what I understood from the conversation and you can tell
> me if I am on the right track?
>
> 1. Use std::string
> 2. Assume that all internal string data is UTF-8
> 3. Enforce that invariant at input boundaries using helper functions
> 4. Be explicit about output encoding if it matters, default to UTF-8
> 5. Make formatting helpers that understand intent (display/file/etc.)
> 6. For Windows, explicitly opt into UTF-8 with the manifest flag (where
> possible) and compiler flag
> 7. Realize that you are mostly on your own here, so investing in a small
> library that will "do the right thing" for your app makes sense
>
> Again, thank you so much for your time. I appreciate all the hard work you
> folks put into this.
>
> On Fri, Jul 3, 2026 at 2:50 AM Tiago Freire <tmiguelf_at_[hidden]> wrote:
>
> I can’t speak for everybody, but I can tell you what I do.
>
>
>
> On Windows apps I just set the console code page to be utf8. The exact
> method to do that has changed as some terminal applications have broken
> certain features. But you can do it in 3 ways:
>
> 1. There’s a global Windows setting that sets the default windows code
> page buried deep in the localization options.
> 2. You can embed the preferred code page in the manifest file for the
> application
> 3. You can set the code page at runtime using a function
>
> To be safe, you can do all 3, but ultimately it is going to depend on the
> terminal’s type setter to respect your preference (sometimes it doesn’t…
> *looks at windows terminal*).
>
>
>
> As for “filesystem::path” they are not Unicode and never will be no matter
> how many times people say otherwise and it is best to acknowledge that
> upfront. To deal with that there is not one but multiple strategies
> depending on the purpose of the “filesystem::path”.
>
> If the purpose is for printing you can create a transcoding function that
> makes certain invalid Unicode characters to be visibly printable indicating
> that “there’s a character there that has been altered for visibility but is
> not the actual character” and that makes “humans with eyes happy”.
>
> If the purpose is to pass trough code that uses [bytes] char8_t to store
> or transfer things, but you would really like to perfectly preserve the
> path, there is a different transcoding function that I use that makes the
> “path round-trip-able” that is “mostly utf8 compatible” but not truly utf8
> in order to encode the invalid code points (i.e. UTF-16 surrogates are
> representable).
>
> So, the right answer is “filesystem::path” is a path, not exactly text but
> text-like. And “what you do with it” depends on “what you want to achieve
> with it”, and Unicode in this context is only relevant if you want to print
> it for a user to see and it is otherwise just bits.
>
>
>
>
>
> As for “[it doesn’t] work with std::format and streams”, I’ve re-written
> my own output, formatting, and encoding library pipeline from scratch and I
> don’t suffer from such problems. The C++ standard has come a long way but
> is comparatively speaking “still just banging rocks together”.
>
> I can share more details regarding what I have done on this front, and
> even tough some concepts can be cannibalized to improve the standard, I’m
> afraid that the C++ status-quo has painted itself too much into a corner
> that is a monumental task to even just start to fix it.
>
>
>
> Best regards,
>
> Tiago
>
>
>
> *From:* SG16 <sg16-bounces_at_[hidden]> *On Behalf Of *Herb Sutter
> via SG16
> *Sent:* Friday, July 3, 2026 01:48
> *To:* sg16_at_[hidden]; 'Christopher Nelson' <nadiasvertex_at_[hidden]>
> *Cc:* Herb Sutter <herb.sutter_at_[hidden]>
> *Subject:* [isocpp-sg16] UTF-8 support status
>
>
>
> Hi C++ Unicode folks, literally asking for a friend (Christopher, on the
> To: line so please Reply-All)…
>
>
>
> His project team wants to use UTF-8 in their code base and they hoped to
> switch to u8string[_view] and char8_t throughout, but they encountered two
> sets of limitations:
>
>
>
> - In the standard, those types don’t seem to work with std::format and
> streams.
>
>
>
> - On Windows, platform APIs interpret narrow characters using the
> active code page (e.g., std::cout emits mojibake,
> filesystem::path{std::string} UTF-8 paths are mangled or throw).
>
>
>
> What’s the current best guidance for adopting UTF-8 in C++ code?
>
>
>
> Thank you,
>
>
>
> Herb
>
>
>
> --
> SG16 mailing list
> SG16_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/sg16
> Link to this post: http://lists.isocpp.org/sg16/2026/07/4786.php
>
>
> --
> SG16 mailing list
> SG16_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/sg16
> Link to this post: http://lists.isocpp.org/sg16/2026/07/4792.php
>
>
>
> --
> Yongwei Wu
> URL: http://wyw.dcweb.cn/
>
>
I think from our standpoint the concern we had was that it appeared that
the committee might be reconsidering the uXXstring classes and backing away
from supporting them in the future. If I understand correctly, the general
view is that there will be more friction at this point because things are
still in development. But long term, the uXXstring classes are going to be
fully supported so if we use them now it's not likely something we will
have to undo in the future. The experience will simply get better over time.
Is that fair?
On Tue, Jul 7, 2026 at 1:53 PM Tiago Freire <tmiguelf_at_[hidden]> wrote:
> I think the topic is meandering a little bit.
>
> The official position of the standard is char8_t for utf-8.
> How you decide to use it is of course up to you, but it doesn't mean that
> your usage is going to be supported in the future.
>
>
>
> ------------------------------
> *From:* Yongwei Wu <wuyongwei_at_[hidden]>
> *Sent:* Tuesday, July 7, 2026 6:41:13 PM
> *To:* sg16_at_[hidden] <sg16_at_[hidden]>
> *Cc:* Christopher Nelson <nadiasvertex_at_[hidden]>; Tom Honermann <
> tom_at_[hidden]>; Tiago Freire <tmiguelf_at_[hidden]>
> *Subject:* Re: [isocpp-sg16] UTF-8 support status
>
> No, I won't use std::u8string, as long as IO streams and std::print do not
> support it.
>
> EBCDIC is a non-issue, and I do not care about it: I have never touched
> such a system in my 30+ year career. I am not expecting to port a program
> to such a system.
>
> There are always trade-offs in life, and one chooses the lesser evil.
> Sticking to std::string and requiring everyone to use UTF-8 work the best.
>
> On Tue, 7 Jul 2026 at 23:04, Tiago Freire via SG16 <sg16_at_[hidden]>
> wrote:
>
> Correct me if I'm wrong but the internet of std::string is to be "system
> defined encoding" not even superset of ASCII. Ex. on an EBDIC system it's
> just EBDIC encoding.
> std::u8string is the utf-8 one, and you should just use that if your text
> is utf-8.
>
>
> ------------------------------
> *From:* Christopher Nelson <nadiasvertex_at_[hidden]>
> *Sent:* Tuesday, July 7, 2026 4:59:17 PM
> *To:* Tom Honermann <tom_at_[hidden]>
> *Cc:* sg16_at_[hidden] <sg16_at_[hidden]>; Tiago Freire <
> tmiguelf_at_[hidden]>
> *Subject:* Re: [isocpp-sg16] UTF-8 support status
>
> I think a significant detail that we'd like to understand is: what does
> the committee intend std::string to mean in the future? In many places it
> seems like it is becoming idiomatic to say that std::string is utf-8. Which
> makes us wonder what the design intent of the other string containers
> are. We're just trying to be good C++ citizens and not create more problems
> for ourselves in the future.
>
> On Fri, Jul 3, 2026 at 11:00 AM Tom Honermann <tom_at_[hidden]> wrote:
>
> I haven’t had a chance to review this thread yet and only a couple of
> regular attendees of SG16 meetings have weighed in so far. I suggest
> waiting at least a few more days for additional advice, especially
> considering the holiday weekend in the US.
>
> The summary below reflects one reasonable approach, but it is not the only
> one.
>
> Tom.
>
> On Jul 3, 2026, at 10:48 AM, Christopher Nelson via SG16 <
> sg16_at_[hidden]> wrote:
>
>
> Thanks folks, I really appreciate all of the guidance.
>
> The specific codebase we are working on is group of libraries that has
> bindings for several programming languages. C++(17/23/26) , C#, nodejs, and
> Rust. It runs on Linux, macOS, and Windows. We control all the applications
> it is embedded in.
>
> May I summarize what I understood from the conversation and you can tell
> me if I am on the right track?
>
> 1. Use std::string
> 2. Assume that all internal string data is UTF-8
> 3. Enforce that invariant at input boundaries using helper functions
> 4. Be explicit about output encoding if it matters, default to UTF-8
> 5. Make formatting helpers that understand intent (display/file/etc.)
> 6. For Windows, explicitly opt into UTF-8 with the manifest flag (where
> possible) and compiler flag
> 7. Realize that you are mostly on your own here, so investing in a small
> library that will "do the right thing" for your app makes sense
>
> Again, thank you so much for your time. I appreciate all the hard work you
> folks put into this.
>
> On Fri, Jul 3, 2026 at 2:50 AM Tiago Freire <tmiguelf_at_[hidden]> wrote:
>
> I can’t speak for everybody, but I can tell you what I do.
>
>
>
> On Windows apps I just set the console code page to be utf8. The exact
> method to do that has changed as some terminal applications have broken
> certain features. But you can do it in 3 ways:
>
> 1. There’s a global Windows setting that sets the default windows code
> page buried deep in the localization options.
> 2. You can embed the preferred code page in the manifest file for the
> application
> 3. You can set the code page at runtime using a function
>
> To be safe, you can do all 3, but ultimately it is going to depend on the
> terminal’s type setter to respect your preference (sometimes it doesn’t…
> *looks at windows terminal*).
>
>
>
> As for “filesystem::path” they are not Unicode and never will be no matter
> how many times people say otherwise and it is best to acknowledge that
> upfront. To deal with that there is not one but multiple strategies
> depending on the purpose of the “filesystem::path”.
>
> If the purpose is for printing you can create a transcoding function that
> makes certain invalid Unicode characters to be visibly printable indicating
> that “there’s a character there that has been altered for visibility but is
> not the actual character” and that makes “humans with eyes happy”.
>
> If the purpose is to pass trough code that uses [bytes] char8_t to store
> or transfer things, but you would really like to perfectly preserve the
> path, there is a different transcoding function that I use that makes the
> “path round-trip-able” that is “mostly utf8 compatible” but not truly utf8
> in order to encode the invalid code points (i.e. UTF-16 surrogates are
> representable).
>
> So, the right answer is “filesystem::path” is a path, not exactly text but
> text-like. And “what you do with it” depends on “what you want to achieve
> with it”, and Unicode in this context is only relevant if you want to print
> it for a user to see and it is otherwise just bits.
>
>
>
>
>
> As for “[it doesn’t] work with std::format and streams”, I’ve re-written
> my own output, formatting, and encoding library pipeline from scratch and I
> don’t suffer from such problems. The C++ standard has come a long way but
> is comparatively speaking “still just banging rocks together”.
>
> I can share more details regarding what I have done on this front, and
> even tough some concepts can be cannibalized to improve the standard, I’m
> afraid that the C++ status-quo has painted itself too much into a corner
> that is a monumental task to even just start to fix it.
>
>
>
> Best regards,
>
> Tiago
>
>
>
> *From:* SG16 <sg16-bounces_at_[hidden]> *On Behalf Of *Herb Sutter
> via SG16
> *Sent:* Friday, July 3, 2026 01:48
> *To:* sg16_at_[hidden]; 'Christopher Nelson' <nadiasvertex_at_[hidden]>
> *Cc:* Herb Sutter <herb.sutter_at_[hidden]>
> *Subject:* [isocpp-sg16] UTF-8 support status
>
>
>
> Hi C++ Unicode folks, literally asking for a friend (Christopher, on the
> To: line so please Reply-All)…
>
>
>
> His project team wants to use UTF-8 in their code base and they hoped to
> switch to u8string[_view] and char8_t throughout, but they encountered two
> sets of limitations:
>
>
>
> - In the standard, those types don’t seem to work with std::format and
> streams.
>
>
>
> - On Windows, platform APIs interpret narrow characters using the
> active code page (e.g., std::cout emits mojibake,
> filesystem::path{std::string} UTF-8 paths are mangled or throw).
>
>
>
> What’s the current best guidance for adopting UTF-8 in C++ code?
>
>
>
> Thank you,
>
>
>
> Herb
>
>
>
> --
> SG16 mailing list
> SG16_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/sg16
> Link to this post: http://lists.isocpp.org/sg16/2026/07/4786.php
>
>
> --
> SG16 mailing list
> SG16_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/sg16
> Link to this post: http://lists.isocpp.org/sg16/2026/07/4792.php
>
>
>
> --
> Yongwei Wu
> URL: http://wyw.dcweb.cn/
>
>
Received on 2026-07-07 18:01:15