SG16 will hold a telecon on Wednesday, April 12th, at 19:30 UTC (timezone conversion).

For those in central Europe, please note that daylight savings time began since we last met, so this telecon will begin one hour later relative to the last telecon.

The agenda follows.

Discussion during the 2023-03-22 SG16 telecon included the following topics:

Note that many of these topics are more LEWG concerns than they are SG16 concerns. I think that is ok; the designs we forward should be guided by our expectations of what LEWG will find agreeable.

My impression of current consensus based on recent discussion is that we wish to be forward looking and focus on support for charN_t types with support for other types provided by wrappers, adapters, casts, etc... I'd like to poll this.

With regard to segmented data and handling of partial code unit sequences at the end of a segment, there are at least two concerns; 1) how to transition the boundary without treating the partial sequence as an error, and 2) how to handle the transition efficiently. Network buffers or data structure segments may provide contiguous data that can be processed optimally, but such optimizations cannot be applied to the entire sequence due to the segmentation. JeanHeyd's work in WG14 N3095 (Restartable and Non-Restartable Functions for Efficient Character Conversions) enables such data to be optimally processed by storing partial sequences in mbstate_t instances and allowing for continuation with another buffer; these are not iterator-based interfaces. The interfaces proposed in P2728R0 cannot support such optimizations; at least not until support for segmented data concepts is added to the ranges library to allow for the identification of contiguous segments (we could recognize range-of-ranges designs, but not range designs where segmentation is an internal iterator detail). I'd like to discuss whether we are comfortable with these limitations or whether we would prefer to wait for a partially-contiguous range specification so that maximally performant functionality can be provided in a range-based interface.

I'd like to spend time discussing the viability of transcoding output iterators like utf_8_to_32_out_iterator and utf_16_to_32_out_iterator. The issue is that writing a partial code unit sequence to them doesn't produce an output, so it isn't clear what happens if no further input is ever provided. Is the partial sequence silently lost? Does the iterator's destructor throw an exception or otherwise signal an error?

 Candidate polls:

  1. UTF transcoding interfaces provided by the C++ standard library should operate on charN_t types with support for other types provided by adapters.
  2. The association of a UTF-8 encoding with a sequence of char must be explicit in the source code unless the literal encoding is UTF-8.
  3. The association of a UTF-16 or UTF-32 encoding with a sequence of wchar_t must be explicit in the source code unless the wide literal encoding is UTF-16 or UTF-32.
  4. char32_t should be used as the Unicode code point type within the C++ standard library.
  5. Low level transcoding facilities (WG14 N3095) suffice for high speed handling of segmented data structures with contiguous segments; high level facilities can rely on iterators to abstract such structures.
  6. MxN conversions where M is larger than N (e.g., UTF-8 -> UTF-32) shall be performed by view/iterator input adapters, not by output adapters.

Tom.