Date: Tue, 27 Oct 2026 21:40:41 +0000
On Tue, 27 Oct 2026, 19:39 Jan Schultke, <janschultke_at_[hidden]> wrote:
> I have previously brought up very similar concerns on Mattermost:
> https://chat.isocpp.org/general/pl/k698q5qg93n3bnjq5renm859zr
>
> The very first thing I brought up is why we are making a new enum. That
> concern was partially addressed by Steve:
> https://chat.isocpp.org/general/pl/861afkhr57dwmphrgjjz3kuy1a
>
> The labels from text_encoding are a non-strict superset of what the WhatWG
>> provides, and the IANA registry doesn't provide actual algorithms.
>
>
Which is a good thing for us.
It was one of the core reasons the browser ecosystem had to produce the
>> standard. There was a lot of divergence on Big5 implementation, and in some
>> places still is. You can sometimes get different answers out of iconv and
>> whatwg, and for purposes of web interchange, iconv is then wrong.
>
>
That's fine. std::text_encoding::id doesn't imply any algorithm, it's just
an identifier. The whatwg encoder and decoder can still Do The Right Thing
for the encoding identified by text_encoding::Big5, because that's what the
WhatWG spec says to do for Big5.
The point is just that we already have an identifier for that encoding, and
no pre-existing "this has to be encoded the way iconv does it" meaning. So
why add another identifier?
> I also brought up that I don't see why this API needs to be based on
> views. This seems much more intended for transcoding a file or network
> packet once, at the program boundaries, in order to obtain Unicode. Views
> are useful for e.g. temporarily iterating over the code points of a UTF-8
> string, without needing to allocate. The API being view-based actually
> hurts its usability in my opinion.
>
> On Tue, 27 Oct 2026 at 18:24, Jonathan Wakely via SG16 <
> sg16_at_[hidden]> wrote:
>
>> I know the paper mentions Eddie's P2728 (Unicode in the library pt 1) and
>> compares them, but I'm not convinced that a completely different API with
>> entirely different names is needed. That seems likely to confuse users.
>>
>> There seems to be a huge amount of overlap between the codec enum and
>> std::text_encoding::id, why do we need a new enum? Won't that confuse users
>> too?
>>
>> Why do we need a novel mechanism for mapping encoding labels/names to
>> IDs, isn't that exactly what std::text_encoding does?
>>
>> Is 'codec' a suitable name? (I don't see that term anywhere in the WhatWG
>> Encoding spec). The paper proposes 'codec' as an enum for identifying an
>> encoding, but then decode_codec and encode_codec as concepts that are
>> modelled by classers that encode and decode ... so is a codec an enum
>> identifying an encoding or a class performing it? It seems to be both. The
>> WhatWG spec uses "encoder" and "decoder", but not "codec".
>> --
>> SG16 mailing list
>> SG16_at_[hidden]
>> https://lists.isocpp.org/mailman/listinfo.cgi/sg16
>> Link to this post: http://lists.isocpp.org/sg16/2026/10/4889.php
>>
>
> I have previously brought up very similar concerns on Mattermost:
> https://chat.isocpp.org/general/pl/k698q5qg93n3bnjq5renm859zr
>
> The very first thing I brought up is why we are making a new enum. That
> concern was partially addressed by Steve:
> https://chat.isocpp.org/general/pl/861afkhr57dwmphrgjjz3kuy1a
>
> The labels from text_encoding are a non-strict superset of what the WhatWG
>> provides, and the IANA registry doesn't provide actual algorithms.
>
>
Which is a good thing for us.
It was one of the core reasons the browser ecosystem had to produce the
>> standard. There was a lot of divergence on Big5 implementation, and in some
>> places still is. You can sometimes get different answers out of iconv and
>> whatwg, and for purposes of web interchange, iconv is then wrong.
>
>
That's fine. std::text_encoding::id doesn't imply any algorithm, it's just
an identifier. The whatwg encoder and decoder can still Do The Right Thing
for the encoding identified by text_encoding::Big5, because that's what the
WhatWG spec says to do for Big5.
The point is just that we already have an identifier for that encoding, and
no pre-existing "this has to be encoded the way iconv does it" meaning. So
why add another identifier?
> I also brought up that I don't see why this API needs to be based on
> views. This seems much more intended for transcoding a file or network
> packet once, at the program boundaries, in order to obtain Unicode. Views
> are useful for e.g. temporarily iterating over the code points of a UTF-8
> string, without needing to allocate. The API being view-based actually
> hurts its usability in my opinion.
>
> On Tue, 27 Oct 2026 at 18:24, Jonathan Wakely via SG16 <
> sg16_at_[hidden]> wrote:
>
>> I know the paper mentions Eddie's P2728 (Unicode in the library pt 1) and
>> compares them, but I'm not convinced that a completely different API with
>> entirely different names is needed. That seems likely to confuse users.
>>
>> There seems to be a huge amount of overlap between the codec enum and
>> std::text_encoding::id, why do we need a new enum? Won't that confuse users
>> too?
>>
>> Why do we need a novel mechanism for mapping encoding labels/names to
>> IDs, isn't that exactly what std::text_encoding does?
>>
>> Is 'codec' a suitable name? (I don't see that term anywhere in the WhatWG
>> Encoding spec). The paper proposes 'codec' as an enum for identifying an
>> encoding, but then decode_codec and encode_codec as concepts that are
>> modelled by classers that encode and decode ... so is a codec an enum
>> identifying an encoding or a class performing it? It seems to be both. The
>> WhatWG spec uses "encoder" and "decoder", but not "codec".
>> --
>> SG16 mailing list
>> SG16_at_[hidden]
>> https://lists.isocpp.org/mailman/listinfo.cgi/sg16
>> Link to this post: http://lists.isocpp.org/sg16/2026/10/4889.php
>>
>
Received on 2026-10-27 21:41:01
