On Wed, Nov 30, 2022 at 12:09 AM Tom Honermann <tom@honermann.net> wrote:
On 11/29/22 6:04 PM, Corentin Jabot wrote:


On Tue, Nov 29, 2022 at 11:47 PM Tom Honermann <tom@honermann.net> wrote:
On 11/29/22 5:32 PM, Corentin Jabot wrote:


On Tue, Nov 29, 2022 at 11:13 PM Tom Honermann via SG16 <sg16@lists.isocpp.org> wrote:

Corentin, I'm having a hard time understanding what the screenshots included in P2675R0 are showing. The paper doesn't explain (or I missed it) what was done to produce those screenshots. I can't tell what code points the displayed glyphs correspond to. The set of characters displayed seems to differ; the number of rows displayed is not consistent. It doesn't seem to be possible to dive into the data to determine which characters are being rendered differently. Basically, I'm unable to evaluate whether the screenshots support the proposal.

It would also be helpful if the screenshots included information about the terminal encoding (almost always UTF-8 I would guess/hope) and the font(s) used. I recognize gathering such data could be challenging.


Well, given the first collection took many days and involved a large number of people, that may prove challenging.
I did intend to have the characters separated by |, but by the time I realized that it was too late.
Knowing the fonts used would be even more challenging as in the absence of a codepoint renderers will try to find some other similar font.
For east asian characters, they will be aligned no matter the font, for emoji they may or may not be rendered on the grid.

The paper contains a script that was used to generate the list of codepoint rendered.
I intended to include that list however it proved difficult to handle for google docs, and apparently subsequentely forgot to put a reference
Feel free to try on your system.

The TL;DR is that on a conforming system with sufficient fonts - which is not all systems, in part because Unicode 15 is recent and not all terminals display wide tofu properly - what is proposed is coherent with existing terminal behaviors. And we should not try to placate over temporary deficiency of one specific terminal over another.

Thanks, could you please add that kind of description to the next revision of the paper? Not for tomorrow obviously.

Having taken a look at the gist linked above, I have to conclude that the screenshots currently in the paper are not beneficial. Some screenshots show rows from the top of the gist and others from the bottom, but I can only tell that by guessing based on glyph shape and being locally able to look at the entire gist.

Having a large number of screenshots is not important to me. I would be happy to have examples for just the following. But I would like screenshots that provide an apples-to-apples comparison and that show the entire gist.

  • Linux with Konsole.
  • Linux with gnome-terminal.
  • Linux with non-graphical terminal.
  • Windows with cmd.exe.
  • Windows with Windows Terminal.
  • Windows with Putty.
  • macOS with its default terminal.

All of these are provided in the paper.
They are present, but they are incomplete and it is not possible for me to make sense of them.
You are free to collect your own of course, not everyone is able to show every codepoint at once given screen size, etc.
I understand that taking multiple screenshots would likely be required to show a complete presentation.
And it's beside the point.
My point is that the standard currently has arbitrary rules that do not follow any existing practice.
Victor obtained the list by looking at existing practice.

Let's continue that tomorrow.
That is explained in the paper, Victor looked at existing practice long before Unicode 15 (his work was based on 13),
and based on an implementation of wcwidth which I also reference and explain.

You can also look at other existing practice

https://github.com/microsoft/terminal/blob/main/src/types/CodepointWidthDetector.cpp
https://github.com/KDE/konsole/blob/b8325d2f1842e8f5b4999e7e4510093895818dee/tools/uni2characterwidth/uni2characterwidth.cpp#L892
https://github.com/GNOME/vte/blob/master/doc/ambiguous.txt

(Note that terminals are much smarter than the standard but they all consider W and F codepoints as double width)

 
Can you justify the current list, as it is in the standard?

I'm trying to evaluate your proposal against the status quo. Unfortunately, the paper is not proving all that helpful in doing so.

Given the list of codepoints I sent you (the gist), would you argue any of them should have a narrow rendering?
 

Tom.

 

Tom.


Tom.

On 11/29/22 3:40 PM, Tom Honermann via SG16 wrote:

SG16 will hold a telecon on Wednesday, November 30th, at 19:30 UTC (timezone conversion).

This message will also serve as your friendly reminder that this meeting is taking place tomorrow. I'm sorry for publishing an agenda so very late.

For participants in the USA, please note that daylight savings time ended 2022-11-06, so this telecon will start one hour earlier than our last telecon.

The agenda follows. We won't get through all of these. These are all of the NB comments we have left to address. Whatever we don't get to in this meeting will be scheduled for the December 14th meeting.

P2713R0 (Escaping improvements in std::format) implements the SG16 proposed resolutions for US 38-098 (see the 2022-10-19 SG16 meeting summary) and FR 005-134 (see the 2022-11-02 SG16 meeting summary). We'll review the wording and then poll forwarding to LEWG as the resolution of the two NB comments.

Candidate Poll 1: P2713R0: Forward to LEWG as the recommended resolution of US 38-098 and FR 005-134 [amended to ...].

P2693R0 (Formatting thread::id and stacktrace) is intended to resolve FR-008-011. I did not initially tag this NB comment as needing SG16 review, but Bryce requested that SG16 take a look, specifically with regard to narrow vs wide formatting. Bryce has indicated this paper will need to be approved soon in order for it to appear in the electronic polling that will be conducted in January.

Candidate Poll 2: P2693R0: Forward to LEWG as the recommended resolution of FR-008-011 [amended to ...].

FR-010-133 and FR-021-013 were discussed during the 2022-11-02 SG16 meeting and concluded with a recommendation to discuss with the project editor the possibility of preferring the Unicode Standard over ISO/IEC 10646 within the C++ standard. The project editor approved this direction and we can now move forward with drafting wording changes. This will require a paper produced in short order if it is to be accepted for C++23.

P2675R0 (LWG3780: The Paper (format's width estimation is too approximate and not forward compatible)) is intended to resolve LWG #3780 and FR-007-012. It seeks to replace the explicit list of code point ranges in [format.string.std]p12 with wording that derives substantially the same set of code points using Unicode database properties.

Candidate Poll 3.1: P2675R0: Forward to LEWG as the recommended resolution of FR-007-012 [amended to ...].
Candidate Poll 3.2: P2675R0: Forward to LEWG for C++26 [amended to ...].
Candidate Poll 3.3: Recommend to LEWG that FR-007-012 be rejected.

FR-020-014 raises concerns that were discussed as part of the reviews of P2314 and P2297 during the 2021-03-24 SG16 meeting. The comment does not appear to present new information. If we choose to accept, a paper will need to be quickly produced.

Candidate poll 4.1: Recommend to CWG that FR-020-014 be accepted.
Candidate poll 4.2: Recommend to CWG that FR-020-014 be rejected.

Tom.


--
SG16 mailing list
SG16@lists.isocpp.org
https://lists.isocpp.org/mailman/listinfo.cgi/sg16