C++ Logo

sg16

Advanced search

Re: [isocpp-sg16] P4246R0 Transcoding Text Views and Ranges for WhatWG and iconv

From: Jan Schultke <janschultke_at_[hidden]>
Date: Tue, 27 Oct 2026 20:39:30 +0100
I have previously brought up very similar concerns on Mattermost:
https://chat.isocpp.org/general/pl/k698q5qg93n3bnjq5renm859zr

The very first thing I brought up is why we are making a new enum. That
concern was partially addressed by Steve:
https://chat.isocpp.org/general/pl/861afkhr57dwmphrgjjz3kuy1a

The labels from text_encoding are a non-strict superset of what the WhatWG
> provides, and the IANA registry doesn't provide actual algorithms. It was
> one of the core reasons the browser ecosystem had to produce the standard.
> There was a lot of divergence on Big5 implementation, and in some places
> still is. You can sometimes get different answers out of iconv and whatwg,
> and for purposes of web interchange, iconv is then wrong.


I also brought up that I don't see why this API needs to be based on views.
This seems much more intended for transcoding a file or network packet
once, at the program boundaries, in order to obtain Unicode. Views are
useful for e.g. temporarily iterating over the code points of a UTF-8
string, without needing to allocate. The API being view-based actually
hurts its usability in my opinion.

On Tue, 27 Oct 2026 at 18:24, Jonathan Wakely via SG16 <
sg16_at_[hidden]> wrote:

> I know the paper mentions Eddie's P2728 (Unicode in the library pt 1) and
> compares them, but I'm not convinced that a completely different API with
> entirely different names is needed. That seems likely to confuse users.
>
> There seems to be a huge amount of overlap between the codec enum and
> std::text_encoding::id, why do we need a new enum? Won't that confuse users
> too?
>
> Why do we need a novel mechanism for mapping encoding labels/names to IDs,
> isn't that exactly what std::text_encoding does?
>
> Is 'codec' a suitable name? (I don't see that term anywhere in the WhatWG
> Encoding spec). The paper proposes 'codec' as an enum for identifying an
> encoding, but then decode_codec and encode_codec as concepts that are
> modelled by classers that encode and decode ... so is a codec an enum
> identifying an encoding or a class performing it? It seems to be both. The
> WhatWG spec uses "encoder" and "decoder", but not "codec".
> --
> SG16 mailing list
> SG16_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/sg16
> Link to this post: http://lists.isocpp.org/sg16/2026/10/4889.php
>

Received on 2026-10-27 19:39:47