To address your point directly.

Agreed on all counts

- https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2023/p1885r12.pdf has a mapping on page 37 (!!)

- We did at the time note that we could add a "is_whatwg" function if we ever wanted to - which could even be constexpr, and therefore used as a constraint or something.

- The windows1252 situation is a bit hairy but do we care? ie, we are not vendoring a brother and so we do not need to map it to ascii.

Another concern I have is that Big5 is not a unicode subset and dealing with encodings that are not unicode subsets can be challenging.

Also, if a platform does not have a way to transcode to/from Big5 (or any other non-utf whatwg encoding), do implementers have to implement that in the standard library? It's a tough ask.

The argument "these are useful on the web" must confront that we have statistics for the web https://w3techs.com/technologies/overview/character_encoding

Wikipedia:

Big5 is another popular non-UTF encoding meant for traditional Chinese characters (though GB 18030 works for those too, is a full UTF), and is next-most popular in Taiwan after UTF-8 at 97.5%,[10] and it's also second-most used in Hong Kong, while there as elsewhere, UTF-8 is even more dominant at 98.7%.

I do however support transcoding facilities based on iconv (or icu, or any other platform mechanism that exposes text converters through a name or an id that can be mapped to a text_encoding).

These are probably better exposed as batch interfaces, I can't imagine doing iconv calls one code unit at a time is gonna go very well (and if the goal is to _expose_ iconv, which I think is the only reasonable ask we can make of implementers,

then we are limited by what iconv provides)

On Tue, 27 Oct 2026 at 19:01, Jonathan Wakely via SG16 <sg16@lists.isocpp.org> wrote:

I checked every one of them, and all the codec enumerators are already present as enumerators of std::text_encoding::id, except for the special values "replacement" and "x-user-defined", and maybe x-mac-cyrillic (is that the same as csISO111ECMACyrillic a.k.a ECMA-cyrillic a.k.a KOI8-E?)

Surely we could reuse std::text_encoding::id? Maybe specify the mapping between the names in https://encoding.spec.whatwg.org/#names-and-labels and std::text_encoding::id values (for the subset of id values that correspond to one of the WhatWG encodings). Then the lookup can be specified to return a value of std::text_encoding::id. Most of the labels in the WhatWG spec already match an alias of the corresponding text_encoding::id, but we could just require that the lookup for this feature maps the labels to an id.

On Tue, 27 Oct 2026 at 17:34, Jonathan Wakely <cxx@kayari.org> wrote:

On Tue, 27 Oct 2026 at 17:24, Jonathan Wakely <cxx@kayari.org> wrote:
I know the paper mentions Eddie's P2728 (Unicode in the library pt 1) and compares them, but I'm not convinced that a completely different API with entirely different names is needed. That seems likely to confuse users.

There seems to be a huge amount of overlap between the codec enum and std::text_encoding::id, why do we need a new enum? Won't that confuse users too?

Why do we need a novel mechanism for mapping encoding labels/names to IDs, isn't that exactly what std::text_encoding does?

Is 'codec' a suitable name? (I don't see that term anywhere in the WhatWG Encoding spec). The paper proposes 'codec' as an enum for identifying an encoding, but then decode_codec and encode_codec as concepts that are modelled by classers that encode and decode ... so is a codec an enum identifying an encoding or a class performing it? It seems to be both. The WhatWG spec uses "encoder" and "decoder", but not "codec".

Oh, also the paper claims that the values of the codec enumerators are not specified, which contradicts the wording. If you want the values of the enumerators to be unspecified, you need to say so in the wording.

"Each enumerator names the encoding of the same name in the WHATWG Encoding Standard, whose table of encodings is normative and is not restated here: an encoding is what that standard says it is, and a specification that copied the list would be a second one to keep current."

This is not wording. An enumerator doesn't name something from another document, it names a constant. The rest of the sentence seems like rationale for the format of the specification, but is not specification.

After the <transode> synopsis there are a load of Remarks: and Constraints: paragraphs that don't seem attached to anything. Were the paragraphs in the wording subjected to a random shuffle?

--
SG16 mailing list
SG16@lists.isocpp.org
https://lists.isocpp.org/mailman/listinfo.cgi/sg16
Link to this post: http://lists.isocpp.org/sg16/2026/10/4891.php