Date: Sat, 02 Aug 2025 17:34:23 -0700
> On Aug 2, 2025, at 5:02 PM, Thiago Macieira <thiago_at_[hidden]> wrote:
>
> On Saturday, 2 August 2025 15:27:07 Pacific Daylight Time Oliver Hunt wrote:
>> I was also unsure about whether the info you get from that source allows you
>> to distinguish “mapped” from “mapped and readable”, “mapped and writable”?
>> It sounds like you have more familiarity with doing such queries (for my
>> work I only ever need the darwin “is this permanently readonly” check for
>> root of trust checks).
>
> On Linux, /proc/self/maps does provide the protection flags of the memory in
<snip>
> I also forgot one more detail: mapped memory does not mean *committed* memory.
> You may still end up in an OOM Kill because the OS can't allocate the memory
> you need.
I think this is almost equivalent to your example of memory mapped IO: the existence
of a virtual address mapping does not guarantee that there’s anything to actually back
it when you try to use it.
>> It is also possible to have an access be valid on one thread but not on
>> another, so if you tried to use/cache a query on one thread and then reuse
>> it on another it may fail, even if you are doing it immediately.
>
> That's not supposed to happen. Unless the application is using some exotic OS
> primitives, the memory protections apply application-wide to all threads.
All of Apple’s arm systems have per thread page permissions[1], which are arguably controlled
via exotic OS primitives (mmap flags and a function that lets the thread rapidly change the
permissions from userspace), but for most applications this is behind the scenes in the OS.
That said the only way user code would hit such OS constructed mappings (as opposed to
them deliberately constructing the mappings themselves) would be if they were trying to access
arbitrary chunks of memory they were not involved in allocating, etc. But the context of this
thread is a developer trying to find a random inaccessible region in the address space, so it
seems reasonable (in context, the decision of a developer to do this is not reasonable :) )
to consider the case where they examine the address space and in doing it wrong they choose
to try to use one of these OS internal mappings as pages a developer might randomly encounter.
> It's similar to asking for non-standard memory/cache behaviour, such as
> uncachable or write-combining on x86, or enabling alignment-protection flags in
> processors that support them, if the compiler isn't prepared to generate code
> obeying those rules.
Back when I was writing an x87 implementation I really did find myself wondering if _anyone_
used the various mode change flags because the entire rest of the process would get extremely
confused/wrong.
—Oliver
[1] Though obviously the vast majority of mappings are identical across all threads, just not all of them,
and the only non-OS internal way to do this is either via the (possibly deprecated?) pthread_jit_write_protect_np()
which changes page permissions explicitly, or the preferred pthread_jit_write_with_callback_np function
(which hides the page permission modification, and makes it harder to mess up).
>
> On Saturday, 2 August 2025 15:27:07 Pacific Daylight Time Oliver Hunt wrote:
>> I was also unsure about whether the info you get from that source allows you
>> to distinguish “mapped” from “mapped and readable”, “mapped and writable”?
>> It sounds like you have more familiarity with doing such queries (for my
>> work I only ever need the darwin “is this permanently readonly” check for
>> root of trust checks).
>
> On Linux, /proc/self/maps does provide the protection flags of the memory in
<snip>
> I also forgot one more detail: mapped memory does not mean *committed* memory.
> You may still end up in an OOM Kill because the OS can't allocate the memory
> you need.
I think this is almost equivalent to your example of memory mapped IO: the existence
of a virtual address mapping does not guarantee that there’s anything to actually back
it when you try to use it.
>> It is also possible to have an access be valid on one thread but not on
>> another, so if you tried to use/cache a query on one thread and then reuse
>> it on another it may fail, even if you are doing it immediately.
>
> That's not supposed to happen. Unless the application is using some exotic OS
> primitives, the memory protections apply application-wide to all threads.
All of Apple’s arm systems have per thread page permissions[1], which are arguably controlled
via exotic OS primitives (mmap flags and a function that lets the thread rapidly change the
permissions from userspace), but for most applications this is behind the scenes in the OS.
That said the only way user code would hit such OS constructed mappings (as opposed to
them deliberately constructing the mappings themselves) would be if they were trying to access
arbitrary chunks of memory they were not involved in allocating, etc. But the context of this
thread is a developer trying to find a random inaccessible region in the address space, so it
seems reasonable (in context, the decision of a developer to do this is not reasonable :) )
to consider the case where they examine the address space and in doing it wrong they choose
to try to use one of these OS internal mappings as pages a developer might randomly encounter.
> It's similar to asking for non-standard memory/cache behaviour, such as
> uncachable or write-combining on x86, or enabling alignment-protection flags in
> processors that support them, if the compiler isn't prepared to generate code
> obeying those rules.
Back when I was writing an x87 implementation I really did find myself wondering if _anyone_
used the various mode change flags because the entire rest of the process would get extremely
confused/wrong.
—Oliver
[1] Though obviously the vast majority of mappings are identical across all threads, just not all of them,
and the only non-OS internal way to do this is either via the (possibly deprecated?) pthread_jit_write_protect_np()
which changes page permissions explicitly, or the preferred pthread_jit_write_with_callback_np function
(which hides the page permission modification, and makes it harder to mess up).
Received on 2025-08-03 00:34:40