Date: Tue, 27 Oct 2026 21:11:06 +0000
Did you mean ranges?
An algorithm isn’t the data, it is what you use to transform data which can come in the forms (contiguous block of memory, iteratable sequence of discontinued characters, i.e. views, ranges, etc.)
Like P2728?
I agree with the observation that “passing in a discrete runtime codec code to a singular function handles multiple transcoders” isn’t ideal, that information should just be part of the type system, and if the user needs a runtime switch case, they can implement it themselves.
I also don’t like the fact that it returns an heap allocated object (either it be an std::string or std::vector) and there is no option to use a stack allocated buffer or a user provided recycled buffer.
But as it comes to the structure of the data, it is my opinion that views are superior to ranges, and should be the de facto interface, for 2 simple reasons.
1. The syntax of the range solution is confusing, even if it is more versatile.
2. The overhead for the range state machine far exceeds the complexity of the transcoding itself (that is added on top of the transcoding itself). So, I would expect the best solution using ranges to be at least 2x to 3x slower than what you could achieve with views.
I would expect that the vast majority of the source data that would require transcoding would come in the form of a consecutive memory block, and you would want to transcode the whole text block instead of just discrete chunks of it, which highly favors views over ranges.
I think a proposal to be successful must settle this question.
And I would like to be able to convince you that views would have good enough ergonomics, and that the performance improvements are significant enough to be worth it.
From: Jan Schultke <janschultke_at_[hidden]>
Sent: Tuesday, October 27, 2026 21:20
To: Tiago Freire <tmiguelf_at_hotmail.com>
Cc: sg16_at_[hidden]; Jonathan Wakely <cxx_at_kayari.org>
Subject: Re: [isocpp-sg16] P4246R0 Transcoding Text Views and Ranges for WhatWG and iconv
On Tue, 27 Oct 2026 at 21:05, Tiago Freire <tmiguelf_at_[hidden]<mailto:tmiguelf_at_[hidden]>> wrote:
> I also brought up that I don't see why this API needs to be based on views. This seems much more intended for transcoding a file or network packet once, at the program boundaries, in order to obtain Unicode. Views are useful for e.g. temporarily iterating over the code points of a UTF-8 string, without needing to allocate. > The API being view-based actually hurts its usability in my opinion.
How so? What alternative API would you use?
I would use an algorithm rather than a view. Range goes in, code units come out.
Part of the problem is that if you have views (and notably a distinct type for each codec), you end up with:
std::u32string decode(std::string_view text, std::codec c) {
switch (c) {
case codec::utf8: return decode_with<codec::utf8>(text);
// 199 more cases ...
}
}
Where std::codec c is a runtime parameter obtained based on, say, the contents of <meta charset=...> HTML elements. The paper seems to have evolved a bit since I made my Mattermost comments and now has some ways to work with runtime codecs. It was just views last time I saw it, but that was an early draft.
Still, I find the use case for views contrived. Why wouldn't you just transcode a single time at program boundaries, in which case an algorithm is better suited than a view?
An algorithm isn’t the data, it is what you use to transform data which can come in the forms (contiguous block of memory, iteratable sequence of discontinued characters, i.e. views, ranges, etc.)
Like P2728?
I agree with the observation that “passing in a discrete runtime codec code to a singular function handles multiple transcoders” isn’t ideal, that information should just be part of the type system, and if the user needs a runtime switch case, they can implement it themselves.
I also don’t like the fact that it returns an heap allocated object (either it be an std::string or std::vector) and there is no option to use a stack allocated buffer or a user provided recycled buffer.
But as it comes to the structure of the data, it is my opinion that views are superior to ranges, and should be the de facto interface, for 2 simple reasons.
1. The syntax of the range solution is confusing, even if it is more versatile.
2. The overhead for the range state machine far exceeds the complexity of the transcoding itself (that is added on top of the transcoding itself). So, I would expect the best solution using ranges to be at least 2x to 3x slower than what you could achieve with views.
I would expect that the vast majority of the source data that would require transcoding would come in the form of a consecutive memory block, and you would want to transcode the whole text block instead of just discrete chunks of it, which highly favors views over ranges.
I think a proposal to be successful must settle this question.
And I would like to be able to convince you that views would have good enough ergonomics, and that the performance improvements are significant enough to be worth it.
From: Jan Schultke <janschultke_at_[hidden]>
Sent: Tuesday, October 27, 2026 21:20
To: Tiago Freire <tmiguelf_at_hotmail.com>
Cc: sg16_at_[hidden]; Jonathan Wakely <cxx_at_kayari.org>
Subject: Re: [isocpp-sg16] P4246R0 Transcoding Text Views and Ranges for WhatWG and iconv
On Tue, 27 Oct 2026 at 21:05, Tiago Freire <tmiguelf_at_[hidden]<mailto:tmiguelf_at_[hidden]>> wrote:
> I also brought up that I don't see why this API needs to be based on views. This seems much more intended for transcoding a file or network packet once, at the program boundaries, in order to obtain Unicode. Views are useful for e.g. temporarily iterating over the code points of a UTF-8 string, without needing to allocate. > The API being view-based actually hurts its usability in my opinion.
How so? What alternative API would you use?
I would use an algorithm rather than a view. Range goes in, code units come out.
Part of the problem is that if you have views (and notably a distinct type for each codec), you end up with:
std::u32string decode(std::string_view text, std::codec c) {
switch (c) {
case codec::utf8: return decode_with<codec::utf8>(text);
// 199 more cases ...
}
}
Where std::codec c is a runtime parameter obtained based on, say, the contents of <meta charset=...> HTML elements. The paper seems to have evolved a bit since I made my Mattermost comments and now has some ways to work with runtime codecs. It was just views last time I saw it, but that was an early draft.
Still, I find the use case for views contrived. Why wouldn't you just transcode a single time at program boundaries, in which case an algorithm is better suited than a view?
Received on 2026-10-27 21:11:09
