Date: Tue, 07 Jul 2026 10:00:30 -0700
On Tuesday, 7 July 2026 08:39:56 Pacific Daylight Time Tiago Freire wrote:
> There's lack of transcoding features for sure, but that doesn't make "what a
> user might do" correct. Visibility of this issue might be hard on popular
> systems (which are mostly ASCII supersets), but when the system diverges
> from this it becomes a problem.
>
> On an EBDIC system you wouldn't be able to reinterpret cast, as "string"
> would be bitwise different from u8"string". This would be very noticeably a
> problem. You will need to thread very gingerly let's say text that comes
> from the command line, which should obviously be std::string, and if you
> are reading from a file which you know is utf-8 it should be std: u8string.
>
> If you want portability to these potential systems then use std::u8string.
Strictly speaking, that's true, but the number of people who will ever touch
an EBCDIC system and don't work for IBM is virtually indistiguishable from
zero. You don't need UTF-8 to build in dependencies on ASCII; a simple
if (c >= 'a' && c <= 'z')
is wrong under EBCDIC.
No, the biggest danger are assumptions that break with Windows, namely:
1) the narrow character execution charset is UTF-8
which causes
2) all file names and environment variables can be encoded under that charset
std::filesystem being UTF-16 aware on Windows introduces more issues, because
now it's possible to get a directory listing and get fs::path objects for files
whose names cannot be encoded in the narrow character charset.
> There's lack of transcoding features for sure, but that doesn't make "what a
> user might do" correct. Visibility of this issue might be hard on popular
> systems (which are mostly ASCII supersets), but when the system diverges
> from this it becomes a problem.
>
> On an EBDIC system you wouldn't be able to reinterpret cast, as "string"
> would be bitwise different from u8"string". This would be very noticeably a
> problem. You will need to thread very gingerly let's say text that comes
> from the command line, which should obviously be std::string, and if you
> are reading from a file which you know is utf-8 it should be std: u8string.
>
> If you want portability to these potential systems then use std::u8string.
Strictly speaking, that's true, but the number of people who will ever touch
an EBCDIC system and don't work for IBM is virtually indistiguishable from
zero. You don't need UTF-8 to build in dependencies on ASCII; a simple
if (c >= 'a' && c <= 'z')
is wrong under EBCDIC.
No, the biggest danger are assumptions that break with Windows, namely:
1) the narrow character execution charset is UTF-8
which causes
2) all file names and environment variables can be encoded under that charset
std::filesystem being UTF-16 aware on Windows introduces more issues, because
now it's possible to get a directory listing and get fs::path objects for files
whose names cannot be encoded in the narrow character charset.
-- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Principal Engineer - Intel Data Center - Platform & Sys. Eng.
Received on 2026-07-07 17:00:41
