I'm concerned that this approach will be hard to understand by people who
have not followed the discussions, on top of preexisting obfuscations (the
translation set indirection).
It's also very repetitive but maybe we can massage that a bit.
Lastly, I really don't like the " There are no end-of-line indicators apart
from the content of the UTF-8 code unit sequence" which is more confusing
than enlightening.
It's also unfortunate that the utf-8-ness is tied to a medium rather than
the content, and that we can't agree that source code is text, or that any
textual data consumed by an implementation has an associated encoding.
I'd also prefer using "input" in lieu of "physical source files", as we
established physical source files may not be files nor be physical.

That being said, as this wording seems to have more consensus, maybe we can
go with some form of it, it achieves the intent of the paper.

---
An implementation shall support source files that are a sequence of UTF-8
code units (UTF-8 source files). It may also support an
implementation-defined set of
other kinds of source files, and, if so, the kind of a source file is
determined in an implementation-defined manner which includes a means of
designating a file as a UTF-8 source file, independent of the contents of
the source files. [Note: In other words, recognizing the U+FEFF Byte Order
Mark is not sufficient. --end note]
If a physical source file is designated or otherwise determined to be a
UTF-8 source file, then it shall be a well-formed UTF-8 code unit sequence
and it is decoded to produce a sequence of UCS scalar values that
constitutes the sequence of elements of the translation character set.
For any other kind of physical source file supported by the implementation,
characters are mapped, in an implementation-defined manner, to a sequence
of translation character set elements.
---
On Fri, Jun 10, 2022 at 1:07 AM Hubert Tong <
hubert.reinterpretcast_at_[hidden]> wrote:
> I also prefer the direction of Option 2. I share Mike's concerns that
> "UTF-8 source file" comes out of nowhere in Option 2 as presented.
>
> Additionally, I previously gave feedback that there was no requirement out
> of SG16 for there to be an ability to individually designate files as UTF-8
> source files (as opposed to having a mode where all source files are
> considered UTF-8 source files).
>
> Corentin, if you have concrete objections to the following, please express
> them:
> An implementation shall support physical source files that are a sequence
> of UTF-8 code units. It may also support an implementation-defined set of
> other kinds of source files, and, if so, the kind of a source file is
> determined in an implementation-defined manner which includes a means of
> causing the determination to interpret files as sequences of UTF-8 code
> units, independent of the contents of the source files. [Note: In other
> words, recognizing the U+FEFF Byte Order Mark is not sufficient. --end note]
>
> If a physical source file is determined to consist of a sequence of UTF-8
> code units, then it shall be a well-formed UTF-8 code unit sequence and its
> content is decoded to produce a sequence of UCS scalar values that
> constitutes the sequence of elements of the translation character set. [
> Note: There are no end-of-line indicators apart from the content of the
> UTF-8 code unit sequence. — end note ]
>
> For any other kind of physical source file supported by the
> implementation, characters are mapped, in an implementation-defined manner,
> to a sequence of translation character set elements.
>
> On Thu, Jun 9, 2022 at 10:23 AM Corentin via Core <core_at_[hidden]>
> wrote:
>
>>
>> Hello folks,
>> We have not talked about P2295 for a while, but given that multiple
>> people have signaled to me they are interested in seeing progress,
>> I would like to see whether we can find a majority consensus on wording.
>> We have 2 options to choose from, I have a very strong preference for
>> option 1 which is a more direct description of reality ("a kind of source
>> file" as suggested by option 2 is a bit too vacuous for my taste).
>>
>> The last sentence of both wordings is extracted from P2348 - Whitespaces
>> Wording Revamp, as this avoids having to retain a note about "end of line
>> indicator" for the non utf-8 case, and a note saying there are no such "end
>> of line indicator" for the ut-8 case. The term "end of line indicator" was
>> never defined, and because the mapping is implementation defined, it is a
>> given that implementations can introduce whatever characters they like.
>>
>> I tweaked option 2 slightly from what was suggested by Mike/Huber to
>> avoid repetition of the definition of a UTF-8 source file.
>>
>> It is important to me that, in addition to achieving the design goals of
>> P2295, the wording remains as clear as possible.
>>
>> Let me know what you think.
>>
>> Regards,
>>
>> Corentin
>>
>>
>> *Option 1*
>> A source file is a sequence of integers with an associated encoding
>> scheme that is determined in an implementation-defined manner.
>> An implementation shall support the UTF-8 encoding scheme, and may
>> support an implementation-defined set of additional encoding schemes.
>> If encoding schemes other than UTF-8 are supported, an implementation
>> shall provide a means by which the UTF-8 encoding scheme can be specified,
>> independent of the content of that source file. [Note: In other words,
>> recognizing the U+FEFF Byte Order Mark is not sufficient. --end note]
>>
>> If the encoding scheme of a source file is determined to be UTF-8, then
>> the source file shall be a well-formed UTF-8 code unit sequence. The source
>> file is decoded to produce a sequence of UCS scalar values that constitutes
>> the sequence of elements of the translation character set.
>>
>> For any other encoding scheme supported by the implementation, source
>> file characters are mapped, in an implementation-defined manner, to a
>> sequence of translation character set elements.
>>
>> *Option 2: *
>>
>> An implementation shall support UTF-8 source files. It may also support
>> an implementation-defined set of other kinds of source files, and, if so,
>> it shall provide an implementation-defined means of designating a file as a
>> UTF-8 source file, independent of the content of that source file. [Note:
>> In other words, recognizing the U+FEFF Byte Order Mark is not sufficient.
>> --end note].
>>
>> If a source file is determined to be a UTF-8 source file, then it shall
>> be a well-formed UTF-8 code unit sequence and its content is decoded to
>> produce a sequence of UCS scalar values that constitutes the sequence of
>> elements of the translation character set.
>>
>> For any other kind of source file, characters are mapped, in an
>> implementation-defined manner, to a sequence of translation character set
>> elements.
>> _______________________________________________
>> Core mailing list
>> Core_at_[hidden]
>> Subscription: https://lists.isocpp.org/mailman/listinfo.cgi/core
>> Link to this post: http://lists.isocpp.org/core/2022/06/12669.php
>>
>