Date: Tue, 7 Jul 2026 15:39:56 +0000
There's lack of transcoding features for sure, but that doesn't make "what a user might do" correct.
Visibility of this issue might be hard on popular systems (which are mostly ASCII supersets), but when the system diverges from this it becomes a problem.
On an EBDIC system you wouldn't be able to reinterpret cast, as "string" would be bitwise different from u8"string". This would be very noticeably a problem. You will need to thread very gingerly let's say text that comes from the command line, which should obviously be std::string, and if you are reading from a file which you know is utf-8 it should be std: u8string.
If you want portability to these potential systems then use std::u8string.
________________________________
From: Thiago Macieira <thiago_at_[hidden]>
Sent: Tuesday, July 7, 2026 5:18:29 PM
To: Christopher Nelson <nadiasvertex_at_[hidden]>; Tom Honermann <tom_at_[hidden]>; sg16_at_[hidden] <sg16_at_[hidden]>
Cc: Tiago Freire <tmiguelf_at_[hidden]>
Subject: Re: [isocpp-sg16] UTF-8 support status
On Tuesday, 7 July 2026 08:04:45 Pacific Daylight Time Tiago Freire via SG16
wrote:
> Correct me if I'm wrong but the internet of std::string is to be "system
> defined encoding" not even superset of ASCII. Ex. on an EBDIC system it's
> just EBDIC encoding. std::u8string is the utf-8 one, and you should just
> use that if your text is utf-8.
That might be the Standard answer from the SG: std::string is the arbitrary-
encoding and anything that interprets the bytes as characters will do so under
the current execution charset, while std::u8string is always UTF-8.
But given the lack of APIs operating on char8_t and u8string, in practice
users will reinterpret_cast back and forth and will store UTF-8 in std::string
anyway. If you own the application, setting the manifest flag in Windows .exe
turning the execution encoding to UTF-8 solves practically all problems too.
If you don't own the application, then you have to deploy your own methods and
functions, in which case you may as well use std::string for UTF-8 too.
Visibility of this issue might be hard on popular systems (which are mostly ASCII supersets), but when the system diverges from this it becomes a problem.
On an EBDIC system you wouldn't be able to reinterpret cast, as "string" would be bitwise different from u8"string". This would be very noticeably a problem. You will need to thread very gingerly let's say text that comes from the command line, which should obviously be std::string, and if you are reading from a file which you know is utf-8 it should be std: u8string.
If you want portability to these potential systems then use std::u8string.
________________________________
From: Thiago Macieira <thiago_at_[hidden]>
Sent: Tuesday, July 7, 2026 5:18:29 PM
To: Christopher Nelson <nadiasvertex_at_[hidden]>; Tom Honermann <tom_at_[hidden]>; sg16_at_[hidden] <sg16_at_[hidden]>
Cc: Tiago Freire <tmiguelf_at_[hidden]>
Subject: Re: [isocpp-sg16] UTF-8 support status
On Tuesday, 7 July 2026 08:04:45 Pacific Daylight Time Tiago Freire via SG16
wrote:
> Correct me if I'm wrong but the internet of std::string is to be "system
> defined encoding" not even superset of ASCII. Ex. on an EBDIC system it's
> just EBDIC encoding. std::u8string is the utf-8 one, and you should just
> use that if your text is utf-8.
That might be the Standard answer from the SG: std::string is the arbitrary-
encoding and anything that interprets the bytes as characters will do so under
the current execution charset, while std::u8string is always UTF-8.
But given the lack of APIs operating on char8_t and u8string, in practice
users will reinterpret_cast back and forth and will store UTF-8 in std::string
anyway. If you own the application, setting the manifest flag in Windows .exe
turning the execution encoding to UTF-8 solves practically all problems too.
If you don't own the application, then you have to deploy your own methods and
functions, in which case you may as well use std::string for UTF-8 too.
-- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Principal Engineer - Intel Data Center - Platform & Sys. Eng.
Received on 2026-07-07 15:40:04