Hi Tom,

On 24/10/2023 07.11, Tom Honermann via SG16 wrote:
> Hang on, this is going to be a bumpy ride.

Thanks for the write-up.

We should ask the implementers whether their basic_stream support
for charN_t is intentional or accidental, and investigate a little
more whether it works.

Hyrum's law suggests that this will be used in the wild.

My opinion:

Let's make sure the ground is clear for a future extension
to charN_t for basic_stream, but let's not try to address
any of the deeper troubles (in particular the 1:N mapping for
basic_fstream). In particular:

Let's fix "int_type". The ABI of the standard library
itself will not be broken, we just risk ABI breakage
of user components, I think?

Let's deprecate

    std::codecvt<char16_t, char8_t, std::mbstate_t>
    std::codecvt<char32_t, char8_t, std::mbstate_t>
    std::codecvt_byname<char16_t, char8_t, std::mbstate_t>
    std::codecvt_byname<char32_t, char8_t, std::mbstate_t>

Those might come back when a proper solution arrives.


std::codecvt<char16_t, char, std::mbstate_t> # Deprecated.
std::codecvt<char32_t, char, std::mbstate_t> # Deprecated.

"Since iostreams does not support charN_t in the standard today and since the char16_t and char32_t specializations have already been deprecated for two release cycles, perhaps it is even reasonable to change their behavior so that they convert to and from the locale encoding rather than UTF-8."

That might work for

std::codecvt<char32_t, char, std::mbstate_t>

but

std::codecvt<char16_t, char, std::mbstate_t>

runs afoul of the 1:N mapping issue, unless on a platform where everything
fits into 16-bit Unicode, right?

Best to leave those functions alone; I'm also ok with removing them.

Jens