I think a significant detail that we'd like to understand is: what does the
committee intend std::string to mean in the future? In many places it seems
like it is becoming idiomatic to say that std::string is utf-8. Which makes
us wonder what the design intent of the other string containers are. We're
just trying to be good C++ citizens and not create more problems for
ourselves in the future.

On Fri, Jul 3, 2026 at 11:00 AM Tom Honermann <tom_at_[hidden]> wrote:

> I haven’t had a chance to review this thread yet and only a couple of
> regular attendees of SG16 meetings have weighed in so far. I suggest
> waiting at least a few more days for additional advice, especially
> considering the holiday weekend in the US.
>
> The summary below reflects one reasonable approach, but it is not the only
> one.
>
> Tom.
>
> On Jul 3, 2026, at 10:48 AM, Christopher Nelson via SG16 <
> sg16_at_[hidden]> wrote:
>
> 
> Thanks folks, I really appreciate all of the guidance.
>
> The specific codebase we are working on is group of libraries that has
> bindings for several programming languages. C++(17/23/26) , C#, nodejs, and
> Rust. It runs on Linux, macOS, and Windows. We control all the applications
> it is embedded in.
>
> May I summarize what I understood from the conversation and you can tell
> me if I am on the right track?
>
> 1. Use std::string
> 2. Assume that all internal string data is UTF-8
> 3. Enforce that invariant at input boundaries using helper functions
> 4. Be explicit about output encoding if it matters, default to UTF-8
> 5. Make formatting helpers that understand intent (display/file/etc.)
> 6. For Windows, explicitly opt into UTF-8 with the manifest flag (where
> possible) and compiler flag
> 7. Realize that you are mostly on your own here, so investing in a small
> library that will "do the right thing" for your app makes sense
>
> Again, thank you so much for your time. I appreciate all the hard work you
> folks put into this.
>
> On Fri, Jul 3, 2026 at 2:50 AM Tiago Freire <tmiguelf_at_[hidden]> wrote:
>
>> I can’t speak for everybody, but I can tell you what I do.
>>
>>
>>
>> On Windows apps I just set the console code page to be utf8. The exact
>> method to do that has changed as some terminal applications have broken
>> certain features. But you can do it in 3 ways:
>>
>> 1. There’s a global Windows setting that sets the default windows
>> code page buried deep in the localization options.
>> 2. You can embed the preferred code page in the manifest file for the
>> application
>> 3. You can set the code page at runtime using a function
>>
>> To be safe, you can do all 3, but ultimately it is going to depend on the
>> terminal’s type setter to respect your preference (sometimes it doesn’t…
>> *looks at windows terminal*).
>>
>>
>>
>> As for “filesystem::path” they are not Unicode and never will be no
>> matter how many times people say otherwise and it is best to acknowledge
>> that upfront. To deal with that there is not one but multiple strategies
>> depending on the purpose of the “filesystem::path”.
>>
>> If the purpose is for printing you can create a transcoding function that
>> makes certain invalid Unicode characters to be visibly printable indicating
>> that “there’s a character there that has been altered for visibility but is
>> not the actual character” and that makes “humans with eyes happy”.
>>
>> If the purpose is to pass trough code that uses [bytes] char8_t to store
>> or transfer things, but you would really like to perfectly preserve the
>> path, there is a different transcoding function that I use that makes the
>> “path round-trip-able” that is “mostly utf8 compatible” but not truly utf8
>> in order to encode the invalid code points (i.e. UTF-16 surrogates are
>> representable).
>>
>> So, the right answer is “filesystem::path” is a path, not exactly text
>> but text-like. And “what you do with it” depends on “what you want to
>> achieve with it”, and Unicode in this context is only relevant if you want
>> to print it for a user to see and it is otherwise just bits.
>>
>>
>>
>>
>>
>> As for “[it doesn’t] work with std::format and streams”, I’ve re-written
>> my own output, formatting, and encoding library pipeline from scratch and I
>> don’t suffer from such problems. The C++ standard has come a long way but
>> is comparatively speaking “still just banging rocks together”.
>>
>> I can share more details regarding what I have done on this front, and
>> even tough some concepts can be cannibalized to improve the standard, I’m
>> afraid that the C++ status-quo has painted itself too much into a corner
>> that is a monumental task to even just start to fix it.
>>
>>
>>
>> Best regards,
>>
>> Tiago
>>
>>
>>
>> *From:* SG16 <sg16-bounces_at_[hidden]> *On Behalf Of *Herb Sutter
>> via SG16
>> *Sent:* Friday, July 3, 2026 01:48
>> *To:* sg16_at_[hidden]; 'Christopher Nelson' <nadiasvertex_at_[hidden]
>> >
>> *Cc:* Herb Sutter <herb.sutter_at_[hidden]>
>> *Subject:* [isocpp-sg16] UTF-8 support status
>>
>>
>>
>> Hi C++ Unicode folks, literally asking for a friend (Christopher, on the
>> To: line so please Reply-All)…
>>
>>
>>
>> His project team wants to use UTF-8 in their code base and they hoped to
>> switch to u8string[_view] and char8_t throughout, but they encountered two
>> sets of limitations:
>>
>>
>>
>> - In the standard, those types don’t seem to work with std::format
>> and streams.
>>
>>
>>
>> - On Windows, platform APIs interpret narrow characters using the
>> active code page (e.g., std::cout emits mojibake,
>> filesystem::path{std::string} UTF-8 paths are mangled or throw).
>>
>>
>>
>> What’s the current best guidance for adopting UTF-8 in C++ code?
>>
>>
>>
>> Thank you,
>>
>>
>>
>> Herb
>>
>>
>>
> --
> SG16 mailing list
> SG16_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/sg16
> Link to this post: http://lists.isocpp.org/sg16/2026/07/4786.php
>
>