Date: Wed, 08 Jul 2026 08:50:11 -0700
On Wednesday, 8 July 2026 08:09:50 Pacific Daylight Time Tiago Freire wrote:
> > That's not correct. Since C++11, char is required to support all UTF-8
> > code units (that means the minimum size for a byte is 8 bits). The
> > addition of char8_t has not changed the requirement on char. Therefore,
> > it can't be UB.
> > Whether the code unit stored in the char makes sense when interpreted
> > under the narrow character set, it's a completely different story.
>
> First of, isn't the "whether the code unit stored in the char makes sense"
> the whole point? Second "that it can fit" doesn't mean "that's how it is
> interpreted".
That's exactly my point. It can fit and therefore you can write code that
interprets differently.
> How do you explain EBCDIC systems?
Just interpret differently. A byte is a byte, and it doesn't matter when
scanning for 0x40 whether it is an at sign or the space character. The string
functions will operate either way and are agnostic to encoding.
The locale-aware functions (ctype.h, etc.) and I/O are a different problem. I
don't know how EBCDIC developers routinely deal with this, other than "use
ASCII-mode".
https://www.ibm.com/docs/en/zos/3.2.0?topic=table-enhanced-ascii-support
They must have working solutions. I'm just happy I don't have to worry about
them myself.
> There was never a promise that every Unicode character would be translated
> as utf-8 for char. It can't, otherwise EBCDIC wouldn't work.
There was a promise that char is big enough to hold every single UTF-8 code
unit. What wasn't promised is that u8'A' would have the same byte value as
'A'. I'm just saying this isn't a problem for encoding-agnostic API.
And I am saying that one of these two situations applies to you when it comes
to I/O:
a) it's UTF-8 anyway and you don't have a problem that needs solving
b) it's not UTF-8, and there currently is no portable solution in either char
or char8_t
For IBM systems affected by (b), they have their existing, non-portable, char-
based solutions to support ASCII and UTF-8. For Windows affected by (b), the
solution is char16_t (under the guise of wchar_t, which everyone
reinterpret_casts to and from). In neither case was char8_t necessary.
Introducing new char8_t APIs may be the way going forward, but it's not
standardising any existing practice.
> `char` lives in a limbo state, when it compiles and runs it must definitely
> have an encoding, but exactly what that encoding is is not defined
> anywhere, and worse it can actually be many things at once and nobody knows
> exactly.
Partially agreed. What you're saying is technically true... except for the
99.99% case where it's known to be UTF-8. Even Microsoft STL's <print> has
code to detect the use of /utf-8 because it's so common nowadays.
Look, I don't have a problem with fixing the remaining 0.01%. Ideally it would
be done by Microsoft switching to UTF-8 by default for the narrow character
encoding and EBCDIC finally fading away. But short of that, what do we do?
Mind you, the solution has to be something that developers will want to use.
And since 99% of whom think there is no problem that still needs solving, how
do we convince them to adopt yet more APIs? That includes the API shared with
WG14 and the need for it to be *usable* (and we know how C APIs tend to be
complex).
> Problems that do absolutely exist on linux systems. I'm not talking about
> the poor choice of Windows using wchar_t for their native interfaces. I'm
> talking about problems like char doesn't have a signedness defined, and
> that you can change it to be signed or unsigned with a compiler setting, a
> problem that makes checks like " val < 'A' " ill-formed. This isn't the
> case with char8_t, char8_t is unsigned. " val < u8'A' " is well-formed.
Agreed that's very unfortunate, but that's what we have unsigned char for. It
has the same problem on encoding (or lack of one). In Qt, we have "uchar" as
typedef, which you can simultaneously read as "unsigned char" or "Unicode
char".
Signedness is an issue, but it's a known one and has been solved for ages. So
of all the problems that char has, its signedness ranks lowest.
> > That's not correct. Since C++11, char is required to support all UTF-8
> > code units (that means the minimum size for a byte is 8 bits). The
> > addition of char8_t has not changed the requirement on char. Therefore,
> > it can't be UB.
> > Whether the code unit stored in the char makes sense when interpreted
> > under the narrow character set, it's a completely different story.
>
> First of, isn't the "whether the code unit stored in the char makes sense"
> the whole point? Second "that it can fit" doesn't mean "that's how it is
> interpreted".
That's exactly my point. It can fit and therefore you can write code that
interprets differently.
> How do you explain EBCDIC systems?
Just interpret differently. A byte is a byte, and it doesn't matter when
scanning for 0x40 whether it is an at sign or the space character. The string
functions will operate either way and are agnostic to encoding.
The locale-aware functions (ctype.h, etc.) and I/O are a different problem. I
don't know how EBCDIC developers routinely deal with this, other than "use
ASCII-mode".
https://www.ibm.com/docs/en/zos/3.2.0?topic=table-enhanced-ascii-support
They must have working solutions. I'm just happy I don't have to worry about
them myself.
> There was never a promise that every Unicode character would be translated
> as utf-8 for char. It can't, otherwise EBCDIC wouldn't work.
There was a promise that char is big enough to hold every single UTF-8 code
unit. What wasn't promised is that u8'A' would have the same byte value as
'A'. I'm just saying this isn't a problem for encoding-agnostic API.
And I am saying that one of these two situations applies to you when it comes
to I/O:
a) it's UTF-8 anyway and you don't have a problem that needs solving
b) it's not UTF-8, and there currently is no portable solution in either char
or char8_t
For IBM systems affected by (b), they have their existing, non-portable, char-
based solutions to support ASCII and UTF-8. For Windows affected by (b), the
solution is char16_t (under the guise of wchar_t, which everyone
reinterpret_casts to and from). In neither case was char8_t necessary.
Introducing new char8_t APIs may be the way going forward, but it's not
standardising any existing practice.
> `char` lives in a limbo state, when it compiles and runs it must definitely
> have an encoding, but exactly what that encoding is is not defined
> anywhere, and worse it can actually be many things at once and nobody knows
> exactly.
Partially agreed. What you're saying is technically true... except for the
99.99% case where it's known to be UTF-8. Even Microsoft STL's <print> has
code to detect the use of /utf-8 because it's so common nowadays.
Look, I don't have a problem with fixing the remaining 0.01%. Ideally it would
be done by Microsoft switching to UTF-8 by default for the narrow character
encoding and EBCDIC finally fading away. But short of that, what do we do?
Mind you, the solution has to be something that developers will want to use.
And since 99% of whom think there is no problem that still needs solving, how
do we convince them to adopt yet more APIs? That includes the API shared with
WG14 and the need for it to be *usable* (and we know how C APIs tend to be
complex).
> Problems that do absolutely exist on linux systems. I'm not talking about
> the poor choice of Windows using wchar_t for their native interfaces. I'm
> talking about problems like char doesn't have a signedness defined, and
> that you can change it to be signed or unsigned with a compiler setting, a
> problem that makes checks like " val < 'A' " ill-formed. This isn't the
> case with char8_t, char8_t is unsigned. " val < u8'A' " is well-formed.
Agreed that's very unfortunate, but that's what we have unsigned char for. It
has the same problem on encoding (or lack of one). In Qt, we have "uchar" as
typedef, which you can simultaneously read as "unsigned char" or "Unicode
char".
Signedness is an issue, but it's a known one and has been solved for ages. So
of all the problems that char has, its signedness ranks lowest.
-- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Principal Engineer - Intel Data Center - Platform & Sys. Eng.
Received on 2026-07-08 15:50:22