On Tue, Oct 27, 2026, 16:05 Tiago Freire via SG16 <sg16_at_[hidden]>
wrote:

> > I also brought up that I don't see why this API needs to be based on
> views. This seems much more intended for transcoding a file or network
> packet once, at the program boundaries, in order to obtain Unicode. Views
> are useful for e.g. temporarily iterating over the code points of a UTF-8
> string, without needing to allocate. > The API being view-based actually
> hurts its usability in my opinion.
>
>
>
> How so? What alternative API would you use?
>
>
>
Presumably one that produces a filled buffer? I'm always in a situation
where I have the text sitting in memory already, and need to do some
additional operations, and early benchmarks indicated that a view piped
into `to<vector>` was not significantly different than a bulk API in
performance. Some testing with Book of Genji and Journey to the West in
legacy native encodings might be suggesting otherwise, but they still
aren't awful, and there's more than a little room behind the API to improve
some parts, I think.


>
>
> *From:* SG16 <sg16-bounces_at_[hidden]> *On Behalf Of *Jan Schultke
> via SG16
> *Sent:* Tuesday, October 27, 2026 20:40
> *To:* sg16_at_[hidden]
> *Cc:* Jan Schultke <janschultke_at_[hidden]>; Jonathan Wakely <
> cxx_at_[hidden]>
> *Subject:* Re: [isocpp-sg16] P4246R0 Transcoding Text Views and Ranges
> for WhatWG and iconv
>
>
>
> I have previously brought up very similar concerns on Mattermost:
> https://chat.isocpp.org/general/pl/k698q5qg93n3bnjq5renm859zr
>
> The very first thing I brought up is why we are making a new enum. That
> concern was partially addressed by Steve:
> https://chat.isocpp.org/general/pl/861afkhr57dwmphrgjjz3kuy1a
>
> The labels from text_encoding are a non-strict superset of what the WhatWG
> provides, and the IANA registry doesn't provide actual algorithms. It was
> one of the core reasons the browser ecosystem had to produce the standard.
> There was a lot of divergence on Big5 implementation, and in some places
> still is. You can sometimes get different answers out of iconv and whatwg,
> and for purposes of web interchange, iconv is then wrong.
>
>
>
> I also brought up that I don't see why this API needs to be based on
> views. This seems much more intended for transcoding a file or network
> packet once, at the program boundaries, in order to obtain Unicode. Views
> are useful for e.g. temporarily iterating over the code points of a UTF-8
> string, without needing to allocate. The API being view-based actually
> hurts its usability in my opinion.
>
>
>
> On Tue, 27 Oct 2026 at 18:24, Jonathan Wakely via SG16 <
> sg16_at_[hidden]> wrote:
>
> I know the paper mentions Eddie's P2728 (Unicode in the library pt 1) and
> compares them, but I'm not convinced that a completely different API with
> entirely different names is needed. That seems likely to confuse users.
>
>
>
> There seems to be a huge amount of overlap between the codec enum and
> std::text_encoding::id, why do we need a new enum? Won't that confuse users
> too?
>
>
>
> Why do we need a novel mechanism for mapping encoding labels/names to IDs,
> isn't that exactly what std::text_encoding does?
>
>
>
> Is 'codec' a suitable name? (I don't see that term anywhere in the WhatWG
> Encoding spec). The paper proposes 'codec' as an enum for identifying an
> encoding, but then decode_codec and encode_codec as concepts that are
> modelled by classers that encode and decode ... so is a codec an enum
> identifying an encoding or a class performing it? It seems to be both. The
> WhatWG spec uses "encoder" and "decoder", but not "codec".
>
> --
> SG16 mailing list
> SG16_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/sg16
> Link to this post: http://lists.isocpp.org/sg16/2026/10/4889.php
>
> --
> SG16 mailing list
> SG16_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/sg16
> Link to this post: http://lists.isocpp.org/sg16/2026/10/4895.php
>