Date: Tue, 27 Oct 2026 20:05:10 +0000
> I also brought up that I don't see why this API needs to be based on views. This seems much more intended for transcoding a file or network packet once, at the program boundaries, in order to obtain Unicode. Views are useful for e.g. temporarily iterating over the code points of a UTF-8 string, without needing to allocate. > The API being view-based actually hurts its usability in my opinion.
How so? What alternative API would you use?
From: SG16 <sg16-bounces_at_[hidden]> On Behalf Of Jan Schultke via SG16
Sent: Tuesday, October 27, 2026 20:40
To: sg16_at_[hidden]
Cc: Jan Schultke <janschultke_at_[hidden]>; Jonathan Wakely <cxx_at_[hidden]>
Subject: Re: [isocpp-sg16] P4246R0 Transcoding Text Views and Ranges for WhatWG and iconv
I have previously brought up very similar concerns on Mattermost: https://chat.isocpp.org/general/pl/k698q5qg93n3bnjq5renm859zr
The very first thing I brought up is why we are making a new enum. That concern was partially addressed by Steve: https://chat.isocpp.org/general/pl/861afkhr57dwmphrgjjz3kuy1a
The labels from text_encoding are a non-strict superset of what the WhatWG provides, and the IANA registry doesn't provide actual algorithms. It was one of the core reasons the browser ecosystem had to produce the standard. There was a lot of divergence on Big5 implementation, and in some places still is. You can sometimes get different answers out of iconv and whatwg, and for purposes of web interchange, iconv is then wrong.
I also brought up that I don't see why this API needs to be based on views. This seems much more intended for transcoding a file or network packet once, at the program boundaries, in order to obtain Unicode. Views are useful for e.g. temporarily iterating over the code points of a UTF-8 string, without needing to allocate. The API being view-based actually hurts its usability in my opinion.
On Tue, 27 Oct 2026 at 18:24, Jonathan Wakely via SG16 <sg16_at_[hidden]<mailto:sg16_at_[hidden]>> wrote:
I know the paper mentions Eddie's P2728 (Unicode in the library pt 1) and compares them, but I'm not convinced that a completely different API with entirely different names is needed. That seems likely to confuse users.
There seems to be a huge amount of overlap between the codec enum and std::text_encoding::id, why do we need a new enum? Won't that confuse users too?
Why do we need a novel mechanism for mapping encoding labels/names to IDs, isn't that exactly what std::text_encoding does?
Is 'codec' a suitable name? (I don't see that term anywhere in the WhatWG Encoding spec). The paper proposes 'codec' as an enum for identifying an encoding, but then decode_codec and encode_codec as concepts that are modelled by classers that encode and decode ... so is a codec an enum identifying an encoding or a class performing it? It seems to be both. The WhatWG spec uses "encoder" and "decoder", but not "codec".
--
SG16 mailing list
SG16_at_[hidden]<mailto:SG16_at_[hidden].org>
https://lists.isocpp.org/mailman/listinfo.cgi/sg16
Link to this post: http://lists.isocpp.org/sg16/2026/10/4889.php
How so? What alternative API would you use?
From: SG16 <sg16-bounces_at_[hidden]> On Behalf Of Jan Schultke via SG16
Sent: Tuesday, October 27, 2026 20:40
To: sg16_at_[hidden]
Cc: Jan Schultke <janschultke_at_[hidden]>; Jonathan Wakely <cxx_at_[hidden]>
Subject: Re: [isocpp-sg16] P4246R0 Transcoding Text Views and Ranges for WhatWG and iconv
I have previously brought up very similar concerns on Mattermost: https://chat.isocpp.org/general/pl/k698q5qg93n3bnjq5renm859zr
The very first thing I brought up is why we are making a new enum. That concern was partially addressed by Steve: https://chat.isocpp.org/general/pl/861afkhr57dwmphrgjjz3kuy1a
The labels from text_encoding are a non-strict superset of what the WhatWG provides, and the IANA registry doesn't provide actual algorithms. It was one of the core reasons the browser ecosystem had to produce the standard. There was a lot of divergence on Big5 implementation, and in some places still is. You can sometimes get different answers out of iconv and whatwg, and for purposes of web interchange, iconv is then wrong.
I also brought up that I don't see why this API needs to be based on views. This seems much more intended for transcoding a file or network packet once, at the program boundaries, in order to obtain Unicode. Views are useful for e.g. temporarily iterating over the code points of a UTF-8 string, without needing to allocate. The API being view-based actually hurts its usability in my opinion.
On Tue, 27 Oct 2026 at 18:24, Jonathan Wakely via SG16 <sg16_at_[hidden]<mailto:sg16_at_[hidden]>> wrote:
I know the paper mentions Eddie's P2728 (Unicode in the library pt 1) and compares them, but I'm not convinced that a completely different API with entirely different names is needed. That seems likely to confuse users.
There seems to be a huge amount of overlap between the codec enum and std::text_encoding::id, why do we need a new enum? Won't that confuse users too?
Why do we need a novel mechanism for mapping encoding labels/names to IDs, isn't that exactly what std::text_encoding does?
Is 'codec' a suitable name? (I don't see that term anywhere in the WhatWG Encoding spec). The paper proposes 'codec' as an enum for identifying an encoding, but then decode_codec and encode_codec as concepts that are modelled by classers that encode and decode ... so is a codec an enum identifying an encoding or a class performing it? It seems to be both. The WhatWG spec uses "encoder" and "decoder", but not "codec".
--
SG16 mailing list
SG16_at_[hidden]<mailto:SG16_at_[hidden].org>
https://lists.isocpp.org/mailman/listinfo.cgi/sg16
Link to this post: http://lists.isocpp.org/sg16/2026/10/4889.php
Received on 2026-10-27 20:05:18