I haven’t had a chance to review this thread yet and only a couple of regular attendees of SG16 meetings have weighed in so far. I suggest waiting at least a few more days for additional advice, especially considering the holiday weekend in the US. 

The summary below reflects one reasonable approach, but it is not the only one. 

Tom.

On Jul 3, 2026, at 10:48 AM, Christopher Nelson via SG16 <sg16@lists.isocpp.org> wrote:



Thanks folks, I really appreciate all of the guidance.

The specific codebase we are working on is group of libraries that has bindings for several programming languages. C++(17/23/26) , C#, nodejs, and Rust. It runs on Linux, macOS, and Windows. We control all the applications it is embedded in.

May I summarize what I understood from the conversation and you can tell me if I am on the right track?

1. Use std::string
2. Assume that all internal string data is UTF-8

3. Enforce that invariant at input boundaries using helper functions

4. Be explicit about output encoding if it matters, default to UTF-8

5. Make formatting helpers that understand intent (display/file/etc.)

6. For Windows, explicitly opt into UTF-8 with the manifest flag (where possible) and compiler flag

7. Realize that you are mostly on your own here, so investing in a small library that will "do the right thing" for your app makes sense

Again, thank you so much for your time. I appreciate all the hard work you folks put into this.

On Fri, Jul 3, 2026 at 2:50 AM Tiago Freire <tmiguelf@hotmail.com> wrote:

I can’t speak for everybody, but I can tell you what I do.

 

On Windows apps I just set the console code page to be utf8. The exact method to do that has changed as some terminal applications have broken certain features.  But you can do it in 3 ways:

1. There’s a global Windows setting that sets the default windows code page buried deep in the localization options.
2. You can embed the preferred code page in the manifest file for the application
3. You can set the code page at runtime using a function

To be safe, you can do all 3, but ultimately it is going to depend on the terminal’s type setter to respect your preference (sometimes it doesn’t… *looks at windows terminal*).

 

As for “filesystem::path” they are not Unicode and never will be no matter how many times people say otherwise and it is best to acknowledge that upfront. To deal with that there is not one but multiple strategies depending on the purpose of the “filesystem::path”.

If the purpose is for printing you can create a transcoding function that makes certain invalid Unicode characters to be visibly printable indicating that “there’s a character there that has been altered for visibility but is not the actual character” and that makes “humans with eyes happy”.

If the purpose is to pass trough code that uses [bytes] char8_t to store or transfer things, but you would really like to perfectly preserve the path, there is a different transcoding function that I use that makes the “path round-trip-able” that is “mostly utf8 compatible” but not truly utf8 in order to encode the invalid code points (i.e. UTF-16 surrogates are representable).

So, the right answer is “filesystem::path” is a path, not exactly text but text-like. And “what you do with it” depends on “what you want to achieve with it”, and Unicode in this context is only relevant if you want to print it for a user to see and it is otherwise just bits.

 

 

As for “[it doesn’t] work with std::format and streams”, I’ve re-written my own output, formatting, and encoding library pipeline from scratch and I don’t suffer from such problems. The C++ standard has come a long way but is comparatively speaking “still just banging rocks together”.

I can share more details regarding what I have done on this front, and even tough some concepts can be cannibalized to improve the standard, I’m afraid that the C++ status-quo has painted itself too much into a corner that is a monumental task to even just start to fix it.

 

Best regards,

Tiago

 

From: SG16 <sg16-bounces@lists.isocpp.org> On Behalf Of Herb Sutter via SG16
Sent: Friday, July 3, 2026 01:48
To: sg16@lists.isocpp.org; 'Christopher Nelson' <nadiasvertex@gmail.com>
Cc: Herb Sutter <herb.sutter@gmail.com>
Subject: [isocpp-sg16] UTF-8 support status

 

Hi C++ Unicode folks, literally asking for a friend (Christopher, on the To: line so please Reply-All)…

 

His project team wants to use UTF-8 in their code base and they hoped to switch to u8string[_view] and char8_t throughout, but they encountered two sets of limitations:

 

• In the standard, those types don’t seem to work with std::format and streams.

 

• On Windows, platform APIs interpret narrow characters using the active code page (e.g., std::cout emits mojibake, filesystem::path{std::string} UTF-8 paths are mangled or throw).

 

What’s the current best guidance for adopting UTF-8 in C++ code?

 

Thank you,

 

Herb

 

--
SG16 mailing list
SG16@lists.isocpp.org
https://lists.isocpp.org/mailman/listinfo.cgi/sg16
Link to this post: http://lists.isocpp.org/sg16/2026/07/4786.php