On Mon, Apr 19, 2021 at 11:01 AM Peter Brett via SG16 <sg16_at_[hidden]>
wrote:

> Good, I think that deliberately forbidding implicit transcoding is the way
> to go here.
>
>
I would like to challenge that

Converting between UTF-X and UTF-Y is a lossless operation.
Say users want to debug a u32_string, they should not have to have to use a
u32 format string: requires more storage for the format string, there might
be extra conversion in the opposite direction to be able to do anything
useful with it (no system will be able to print a utf32 string natively).

So what is it that we gain by not allowing format(u8"{}", u"");
and format(u8"{}", U"");?

I do agree that format(u8"{}", ""); is a bit dicey and not allowing it is a
good starting point.

(and of course, format("{}", u8""); would have strong opposition from me)



>
>
> What should the following code do?
>
>
>
> #include <locale>
>
> #include <format>
>
> std::locale::global(std::locale("en_GB"));
>
> auto a = std::format("{:} {:L}", 10000, 20000);
>
> auto b = std::format(u8"{:} {:L}", 10000, 20000);
>
>
>
> Options:
>
>
>
> 1. *a = "10000 20,000", b = u8"10000 20,000"*. In effect, mandate that
> implementations supply UTF-8/16/32 locale.
> 2. *a = "10000 20,000", b = u8"10000 20000"*. Silently ignore the ‘L’
> option, setting up for an ABI break if we want to introduce Better Global
> Locale at some point in the future. It may also cause non-obvious changes
> in behaviour for users who start using std::format with 'L', but then want
> to u8-ify it later.
> 3. *a = "10000 20,000", b = <std::format_error>.* Forbid implementers
> from even attempting to make 'L' option work in UTF-8/16/32 formatting.
> 4. *A = "10000 20,000", b = <conditionally-supported>.* Many
> implementers will choose to throw std::format_error. However, if wchar_t
> is 32-bit, and the wide execution encoding is UTF-32, then some
> implementers will actually be able to ‘L’ Just Work in UTF-32 formatting,
> and it doesn’t seem unreasonable to allow them to do that.
>
>
>
> “std::locale in its current form is pretty much useless,” may be a true
> statement but it doesn’t help me make progress.
>

I interpret it as: lets tread carefully and lets not back ourselves into a
corner.
However I think that Victor's point that we should not create
inconsistencies between the different overloads is also a very good point


>
>
> Peter
>
>
>
> *From:* Victor Zverovich <victor.zverovich_at_[hidden]>
> *Sent:* 18 April 2021 23:23
> *To:* Peter Brett <pbrett_at_[hidden]>
> *Cc:* SG16 <sg16_at_[hidden]>
> *Subject:* Re: Wording strategy for Unicode std::format
>
>
>
> EXTERNAL MAIL
>
> > {fmt} already does exactly this, right?
>
>
>
> {fmt} explicitly disallows any implicit transcoding at the formatting
> level.
>
>
>
> > How can we word this so as to make `{L}` substitutions for UTF-8/16/32
>
> > formatting conditionally-supported, depending on whether the
> > implementation provides the necessary specializations of <locale>
> > facilities?
>
>
>
> I'm less concerned with how we word this and more with the fact that
> std::locale in its current form is pretty much useless. I would recommend
> looking into this instead of blindly extending it to new code unit types.
>
>
>
> Cheers,
>
> Victor
>
>
>
> On Fri, Apr 16, 2021 at 10:08 AM Peter Brett <pbrett_at_[hidden]> wrote:
>
> Hi all (esp. Victor),
>
> We discussed adding C++23 support for homogeneous formatting in UTF-8,
> UTF-16 and UTF-32. For C++23, we would like to allow UTF-8 format
> strings with UTF-8 substitutions, UTF-16 format strings with UTF-16
> substitutions, etc. In a future version of the standard (where UTF
> transcoding is guaranteed to be available) we would like to extend this
> to allowing e.g. UTF-32 substitutions into UTF-8 format strings.
>
> Victor: {fmt} already does exactly this, right?
>
> As far as I can tell, most of the wording is already in place for this,
> and it will only be necessary to mandate the addition of specific
> overloads and template specialisations.
>
> My current sticking point is the way we have specified the
> locale-specific form (with the `L` option). Take the `{L}` substitution
> for bool, for example. In P1892 I chose to specify this in terms of
> std::numpunct<charT>, but the standard only requires the standard
> library to provide numpunct<char> and numpunct<wchar_t> specializations.
> Similar problems arise for `L` with other standard format specifiers.
>
> How can we word this so as to make `{L}` substitutions for UTF-8/16/32
> formatting conditionally-supported, depending on whether the
> implementation provides the necessary specializations of <locale>
> facilities?
>
> Advice appreciated.
>
> Peter
>
> --
> SG16 mailing list
> SG16_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/sg16
>