On Tue, 14 Jan 2020 at 23:57, Steven R. Loomis <srl295_at_[hidden]> wrote:

> Hi Corentin.,
>
> --
> Steven R. Loomis | @srl295 | git.io/srl295
>
>
>
> El ene. 13, 2020, a las 5:46 p. m., Corentin Jabot <
> corentinjabot_at_[hidden]> escribió:
>
> + message-format-wg_at_[hidden]
>
> Hello.
> Let me (try to) describe the current state of things in C++ and future
> directions.
>
> Of course this will be my opinions and not necessarily that of SG16 or
> WG21.
>
>
> Thank you!
>
>
> C++20 (which is on course to be approved next month) will provide a new
> feature in the name of std::format derived from the popular fmt library (
> https://fmt.dev/), itself heavily inspired and sharing the syntax of
> python's format function.
>
> std::format("Hello {}", "World") -> "Hello World";
> std::format("{2} + {1} = {0}", 3, 1.0, 2) -> "1.0 + 2 = 3";
>
> Of interest to Unicode and localization:
>
> - For now this function is mostly byte based, in that it is encoding
> agnostic.
> - However we made the interesting decision that padding is based on
> display width (which is fuzzily specified), as we realized the primary use
> case for padding was the creation of console interface
>
>
> display width is complex… Unicode’s East Asian Width is often used for
> character width, but there’s more to that (and see <
> https://www.unicode.org/reports/tr11/#Scope> … see for example
> https://github.com/nodejs/node/blob/b0a762157793b0d9143eaa7c270da91932f2a64f/src/node_i18n.cc#L729 in
> Node.js — going beyond wcwidth, etc which do not reflect many terminal
> emulators’ behavior
>

Nice, I think that might be of interest to Victor!


>
> Is the function itself mostly designed for the console or generalized use
> in application (such as non-terminal UI)?
>
Both - although padding/boxing in UI will probably handled more accurately
at the font rendering which is out of scope for us :)


>
> - By default this function will format all types, notably numbers
> using the C locale.. locale is explicitly opt-in : std::format(locale("fr_FR"),
> "{:L}", 1.0) -> "1,0";
> - It is not a translation facility, but does support positional
> arguments with index. *I am afraid that identifier based positional
> arguments would result in more cumbersome and less efficient APIS for C++
> as it would require some kind of dictionary*
>
> Positional with index is then localizable, something like “User {}
> requests {}.” is not as localizable because the order may need to change.
>

My point is that {0} is much easier to deal with than {users_count} from an
api standpoint in c++ (can be implemented more efficiently) - if an
identifier is desirable,
something like {0-users_count} or similar syntax would work better than
forcing the instantiation of a map or similar structure


>
> - Each type of entity can have a set of options which are determined
> by its type. Formatters are user defined and the standard does provide
> formatting for numbers, strings, date/time and a few other things.
>
>
>
> There is some consensus that we should in the future extend that interface
> rather than iostream as it is much more efficient and easier to use.
> I think that sharing a syntax which is easy to use between C++ and Python
> is a great benefit and it would be interesting to see if Unicode can build
> on it too.
>
>
> Yes. Please provide input to the MFWG then. You might update or create an
> issue at https://github.com/unicode-org/message-format-wg/issues
>
>
> That is for C++20.
> As for the future and things we might benefit from:
>
> I don't think any one is looking at translation s in the C++ standard and
> to be honest we are spread a bit thin. we do have std::messages which is
> a wrapper over gettext,does not support pluralization and, as far as I
> can tell, has very little uses.
>
>
> That’s why MessageFormat was created, first in Java and soon after in ICU
> C++/Java. (Note that the MFWG is not standardizing ICU, it’s creating a
> follow-on to ICU’s format.) Localization of these messages is a critical
> requirement. The message formats represented by the working group are in
> heavy use.
>
> *If we were to look at translations someday, it is clear that having a
> spec we can reference would be almost necessary. *
> And we wouldn't want to create something new specific to standard C++,
> implementing a Unicode specification would have a lot more value.
>
>
> Very good.
>
> At the same time we are currently looking at measurements and units apis.
> I tried to make the point that we should provide localized formatting for
> measurements and units if such api is provided.
> Alas, I found that there is no spec for that, nor UAX and the CLDR was not
> complete (some unit would have kilo versions, some not, things like that),
>
>
> CLDR specifies arbitrary SI prefixes for all units, see
> https://unicode-org.atlassian.net/browse/CLDR-13057 So kilo-anything is
> supported. What are the other shortcomings?
>

Interesting.
So if the CLDR defines both joule and mega for example, one should be able
to localize megajoule?
What about missing units? for example farad, coulomb, newton?
I have no idea what will ultimately end up in the standard though, just a
few things that I found in this library https://github.com/mpusz/units ,
which is an implementation of this proposal
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1935r0.html
My guess is that it will take quite a while until anything comes of it
though


>
> CLDR units are implemented in ICU, and are part of Ecma402
> https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/NumberFormat
>
>

This is super interesting - I think it is rather easy for the C++ standard
to refer to ECMA specification


>
> new Intl.NumberFormat("pt-PT", { style: 'unit',
> unit: "mile-per-hour"}).format(50);
> // → 50 mi/h
>
>
> which would add quite a burden
> for the C++ committee to specify and we would most likely get it wrong.
> *I think it would be tremendously helpful for us to have a specification
> on how to format measurement units.*
>
>
> I would recommend CLDR,
> http://unicode.org/reports/tr35/tr35-general.html#Unit_Elements
>
> Unit specification (as part of the message format) is in-scope for the
> message format working group. I think ICU already supports units in message
> format (I can’t find it on a quick search).
>
> Similarly, all string to number and number to string conversions in the
> standard, including integral and floating points assume the Hindu-Arabic
> numerals system.
> *A specification telling us when and how use other numeral systems would
> be beneficial.*
>
>
> Please see
> https://unicode.org/reports/tr35/tr35-numbers.html#otherNumberingSystems for
> example. EcmaScript and many others have adopted this.
>

Ok, so it seems ECMA 402 is the thing of interest in both instances.
I understands tr-35 describes the format of the locale data from which one
is able to infer the set of localization and rules but just from a
specification perspective it seems
much simpler to refer to something that already did the work of turning
into an interface.
With the hope that ICU and ECMA have the same behavior?


>
> I have no idea if either of these points fall into the purview of your
> group.
>
> I may be forgetting many things, but i think it's a fair overview of the
> current state of things in C++ as far as formatting is concerned.
> I hope that helps.
>
>
> Thanks!
>
>
>
> Regards,
>
>
> Corentin
>
>
>
>
>
> On Fri, 10 Jan 2020 at 23:54, Steven R. Loomis via SG16 <
> sg16_at_[hidden]> wrote:
>
>> FYI. This might be of interest as far as std::format goes.
>>
>> Steven, IBM/ICU
>>
>>
>> --
>> Steven R. Loomis | @srl295 | git.io/srl295
>>
>>
>>
>> Inicio del mensaje reenviado:
>>
>> *De: *announcements_at_[hidden]
>> *Asunto: **New Unicode Working Group: Message Formatting*
>> *Fecha: *10 de enero de 2020, 1:55:35 p. m. PST
>> *Para: *announcements_at_[hidden]
>> *Responder a: *root_at_[hidden]
>>
>> <msg-wg-annc-large.jpg>One of the challenges in adapting programs to
>> work with different languages is message formatting. This is the process of
>> formatting and inserting data values into messages in the user’s language.
>> For example, “The package will arrive at *{time}* on *{date}*” could be
>> translated into German as “Das Paket wird am *{date}* um *{time}*
>> geliefert”, and the particular *{time}* and *{date}* variables would be
>> automatically formatted for German, and inserted in the right places.
>>
>> The Unicode Consortium has provided message formatting for some time via
>> the ICU programming libraries and CLDR locale data repository. But until
>> now we have not had a syntax for localizable message strings standardized
>> by Unicode. Furthermore, the current ICU MessageFormat is relatively
>> complex for existing operations, such as plural forms, and it does not
>> scale well to other language properties, such as gender and inflections.
>>
>> The Unicode CLDR Technical Committee is formalizing a new working group
>> to develop a technical specification for message format that addresses
>> these issues. That working group is called the Message Format Working Group
>> and is chaired by Romulo Cintra from CaixaBank. Other participants
>> currently represented are Amazon, Dropbox, Facebook, Google, IBM, Mozilla,
>> OpenJSF, and Paypal.
>>
>> For information on how to get involved, visit the working group’s GitHub
>> page: https://github.com/unicode-org/message-format-wg
>>
>> Open discussions will take place on GitHub, and written notes will be
>> posted after every meeting.
>>
>> ------------------------------
>> *Over 130,000 characters are available for adoption
>> <http://unicode.org/consortium/adopt-a-character.html>, to help the Unicode
>> Consortium’s work on digitally disadvantaged languages.*
>>
>> <ynh-infinity.png> <http://unicode.org/consortium/adopt-a-character.html>
>>
>> http://blog.unicode.org/2020/01/new-unicode-working-group-message.html
>>
>>
>> --
>> SG16 mailing list
>> SG16_at_[hidden]
>> https://lists.isocpp.org/mailman/listinfo.cgi/sg16
>>
>
>