On 1/29/21 4:27 AM, Corentin wrote:
On Fri, Jan 29, 2021 at 9:39 AM Jens Maurer <Jens.Maurer@gmx.net> wrote:
On 29/01/2021 08.20, Tom Honermann wrote:
> On 1/28/21 1:57 PM, Jens Maurer via SG16 wrote:
>> On 28/01/2021 19.37, Corentin via SG16 wrote:
>>> On Thu, Jan 28, 2021 at 7:22 PM Peter Brett <pbrett@cadence.com <mailto:pbrett@cadence.com>> wrote:
>>>
>>> I think the big problem here is trying to make it a template.____
>>>
>>> __ __
>>>
>>> Make it named. It’s literally not possible to use this correctly in generic code.
>>>
>>>
>>> Question then is do we want to solve the issue for wchar_t?
>>> Because having the name of the encoding in the function kinda precludes that - the sizeof(wchar_t) being platform dependant
>> You only get away with char* -> char8_t* because "char" has special
>> aliasing exceptions.
>>
>> You'll get the full set of aliasing concerns for
>> wchar_t* -> char16_t* or char32_t*
>
> I think what we're looking for is a portable solution for this ICU hack <https://github.com/unicode-org/icu/blob/master/icu4c/source/common/unicode/char16ptr.h#L30-L36> (generalized to make it work for [unsigned] char* conversion to char8_t*); the goal being to enable some form of explicit restricted pointer interconvertibility between same sized/aligned types.
Yes.
> I don't understand the ICU hack sufficiently well to relate it to a memory or object model. I'm also not sure that it actually works (though it may suffice for the scenarios that are encountered in practice).
>
> Perhaps something like this would suffice.
>
> template<typename To, typename From>
> requires requires {
> requires std::is_trivial_v<To>;
> requires std::is_trivial_v<From>;
> requires sizeof(To) == sizeof(From);
> requires alignof(To) == alignof(From);
> }
> To* alias_barrier_cast(From *p) {
> asm volatile("" : : "rm"(p) : "memory");
> return reinterpret_cast<To*>(p);
> }
From a C++ memory model perspective, there is no difference between,
say, char16_t and short or int: They form their own aliasing domain.
Converting the pointer with reinterpret_cast or something is NOT
the problem; the problem is accessing the data before and after.
The solution I'm thinking of is that the data can only beaccessed through the returned pointer after the function call.Do you think that is more workable?I was thinking in terms of a similar model as well, but if a returned pointer is required, then existing interfaces that don't return such a pointer won't be usable.
I've been trying to construct a test case that demonstrates that reinterpret_cast doesn't suffice, but I have so far been unable to coerce an optimizer into taking advantage of anti-aliasing rules. I've tried gcc and clang with -O3 and -fstrict-aliasing, but it looks like if the optimizer's escape analysis finds that an address escapes, regardless of by which type, then it assumes the object may have been modified (unless declared const). I've been trying with various types, not just ones that involve the aliasing friendly types.
If anyone can suggest or identify a test that demonstrates that reinterpret_cast doesn't suffice, that might be helpful for testing other solutions.
There have been papers in the past that attempted to bless
regions of memory with a different data type (e.g. to deal with
mmapped file data); I think such a direction might be worthwhile
to investigate.
I certainly don't want to deal with a "solution" that covers
char8_t / char16_t / char32_t only, if the underlying concerns
are also applicable elsewhere.
Even if there was a use case for a generalized solution, we need to do something specific for this as we have additionalpreconditions, namely that the input is a well-formed sequence of utf code units.I'm not sure where this requirement comes from. I think the only solution we need is one that permits access to a [unsigned] char object via a char8_t lvalue (in limited cases) without provoking UB. Whether the data is well-formed UTF-8 is an orthogonal concern that is more in the domain of contracts.
Tom.