Date: Sun, 12 Feb 2023 11:43:10 -0500
On 2/12/23 11:28, Jason McKesson via Std-Proposals wrote:
> On Sun, Feb 12, 2023 at 9:55 AM Phil Bouchard via Std-Proposals
> <std-proposals_at_[hidden]> wrote:
>>
>>
>>
>> On 2/12/23 09:05, Sebastian Wittmeier via Std-Proposals wrote:
>>> Hi Phil,
>>>
>>> can you ellaborate on the reliability issues of RtlAllocateHeap()?
>>>
>>> Are you talking about out-of-memory conditions or heap corruption with
>>> HEAP_NO_SERIALIZE in multi-threading scenarios or about performance in
>>> those cases?
>>>
>>> Are there reliability issues even without HEAP_NO_SERIALZE? Then
>>> RtlAllocateHeap() violates the Windows API specification.
>>
>> No I do not use HEAP_NO_SERIALZE so it should be "thread-safe".
>>
>>> Or you want C++ to provide facilities to provide ways to avoid/replace
>>> RtlAllocateHeap() for reliability or compliance reasons for the whole
>>> application, even for static and dynamic libraries, which have been
>>> directly linked to it?
>>
>> In brave new world that's exactly what I would hope for. A memory pool
>> is not very complicated to implement using the following possible
>> characteristics:
>> - thread-local for faster and safer accesses;
>> - type-oriented to group block sizes together and to create less
>> fragmentation;
>> - allocation rate to extrapolate future memory page allocation size;
>> - possible "reallocation";
>> - possible destruction of heap pages with or without calling destructors;
>> - ...
>>
>> So anyway there's a lot of factors that can be considered and I would
>> have hoped to replace low-level memory pools with custom-ones. But the
>> only way to achieve this is by standardizing linkers.
>
> I don't see how "standardizing linkers" helps anything with regard to
> the problem you're talking about.
>
> What you want is to be able to globally replace what the default
> `operator new` and `operator delete` do, even outside of the current
> translation unit. That's not a linker problem; that's a problem in how
> `operator new/delete` *work*.
>
> In order to be able to replace them as you desire, these functions
> have to stop being static function calls and become *dynamic* function
> calls. That is, the static code needs to look for a registered
> function pointer and call that. That is the *only way* to make this
> work across translation units that did not see the global override.
> That's not a linker issue; that's an issue with how code works.
Yeah or "virtual" function calls that can be overriden whatever the
client wants to use them for.
So I guess what I am trying to suggest are: virtual operator new () and
delete() function calls.
> And it doesn't even fix the problem universally because code compiled
> before this change *cannot* be made to use the new dynamic function.
> There's no way to modify the compiled binary to find where they call
> `operator new` because such code may have been (and likely was)
> inlined. And it may even be an ABI break, but I'm not an expert on
> that.
>
> Furthermore... they could just call `RtlAllocateHeap` themselves
> manually. You cannot *make* someone not call some function.
> *Especially* across DLL boundaries.
Yeah that's fine because Windows at least upgrades their compiler
quickly regarding the latest standards; I give them that.
> What you want is simply not viable.
>
>> If you search engine search on Rtl*Heap() functions then this problem
>> lasted for a long time and all related bug reports were simply
>> dismissed. I think it's time to turn the page on those issues.
>
> That's not how the burden of proof works. You don't get to say that
> there's a problem with a thing and we need a workaround, and when
> asked for evidence of that problem then say go find it yourself.
>
> I mean you *can* say that, since you just did. But it's not exactly a
> reasonable response.
>
> And for what it's worth, I did spend a few minutes Googling the issue.
> And the only incidents I found where any actual investigation was done
> (ie: something more than "a program crashed and the stack trace named
> this function") all resolved down to application-caused heap
> corruption. Which... is not a problem even your suggestion could
> actually fix.
My goal here is to find solutions to problems; not to find problems to
solutions.
Memory allocations are a huge cybersecurity problem (70%) and needs to
be fixed at all layers, starting with core allocations routines. That's
an undeniable fact.
Thanks,
> On Sun, Feb 12, 2023 at 9:55 AM Phil Bouchard via Std-Proposals
> <std-proposals_at_[hidden]> wrote:
>>
>>
>>
>> On 2/12/23 09:05, Sebastian Wittmeier via Std-Proposals wrote:
>>> Hi Phil,
>>>
>>> can you ellaborate on the reliability issues of RtlAllocateHeap()?
>>>
>>> Are you talking about out-of-memory conditions or heap corruption with
>>> HEAP_NO_SERIALIZE in multi-threading scenarios or about performance in
>>> those cases?
>>>
>>> Are there reliability issues even without HEAP_NO_SERIALZE? Then
>>> RtlAllocateHeap() violates the Windows API specification.
>>
>> No I do not use HEAP_NO_SERIALZE so it should be "thread-safe".
>>
>>> Or you want C++ to provide facilities to provide ways to avoid/replace
>>> RtlAllocateHeap() for reliability or compliance reasons for the whole
>>> application, even for static and dynamic libraries, which have been
>>> directly linked to it?
>>
>> In brave new world that's exactly what I would hope for. A memory pool
>> is not very complicated to implement using the following possible
>> characteristics:
>> - thread-local for faster and safer accesses;
>> - type-oriented to group block sizes together and to create less
>> fragmentation;
>> - allocation rate to extrapolate future memory page allocation size;
>> - possible "reallocation";
>> - possible destruction of heap pages with or without calling destructors;
>> - ...
>>
>> So anyway there's a lot of factors that can be considered and I would
>> have hoped to replace low-level memory pools with custom-ones. But the
>> only way to achieve this is by standardizing linkers.
>
> I don't see how "standardizing linkers" helps anything with regard to
> the problem you're talking about.
>
> What you want is to be able to globally replace what the default
> `operator new` and `operator delete` do, even outside of the current
> translation unit. That's not a linker problem; that's a problem in how
> `operator new/delete` *work*.
>
> In order to be able to replace them as you desire, these functions
> have to stop being static function calls and become *dynamic* function
> calls. That is, the static code needs to look for a registered
> function pointer and call that. That is the *only way* to make this
> work across translation units that did not see the global override.
> That's not a linker issue; that's an issue with how code works.
Yeah or "virtual" function calls that can be overriden whatever the
client wants to use them for.
So I guess what I am trying to suggest are: virtual operator new () and
delete() function calls.
> And it doesn't even fix the problem universally because code compiled
> before this change *cannot* be made to use the new dynamic function.
> There's no way to modify the compiled binary to find where they call
> `operator new` because such code may have been (and likely was)
> inlined. And it may even be an ABI break, but I'm not an expert on
> that.
>
> Furthermore... they could just call `RtlAllocateHeap` themselves
> manually. You cannot *make* someone not call some function.
> *Especially* across DLL boundaries.
Yeah that's fine because Windows at least upgrades their compiler
quickly regarding the latest standards; I give them that.
> What you want is simply not viable.
>
>> If you search engine search on Rtl*Heap() functions then this problem
>> lasted for a long time and all related bug reports were simply
>> dismissed. I think it's time to turn the page on those issues.
>
> That's not how the burden of proof works. You don't get to say that
> there's a problem with a thing and we need a workaround, and when
> asked for evidence of that problem then say go find it yourself.
>
> I mean you *can* say that, since you just did. But it's not exactly a
> reasonable response.
>
> And for what it's worth, I did spend a few minutes Googling the issue.
> And the only incidents I found where any actual investigation was done
> (ie: something more than "a program crashed and the stack trace named
> this function") all resolved down to application-caused heap
> corruption. Which... is not a problem even your suggestion could
> actually fix.
My goal here is to find solutions to problems; not to find problems to
solutions.
Memory allocations are a huge cybersecurity problem (70%) and needs to
be fixed at all layers, starting with core allocations routines. That's
an undeniable fact.
Thanks,
-- Logo <https://www.fornux.com/> *Phil Bouchard* facebook icon <https://www.linkedin.com/in/phil-bouchard-5723a910/> CTO T: (819) 328-4743 E: phil_at_[hidden]| www.fornux.com <http://www.fornux.com> 8 rue de la Baie| Gatineau (Qc), J8T 3H3 Canada Banner <https://goglobalawards.org/> Le message ci-dessus, ainsi que les documents l'accompagnant, sont destinés uniquement aux personnes identifiées et peuvent contenir des informations privilégiées, confidentielles ou ne pouvant être divulguées. Si vous avez reçu ce message par erreur, veuillez le détruire. This communication (and/or the attachments) is intended for named recipients only and may contain privileged or confidential information which is not to be disclosed. If you received this communication by mistake please destroy all copies.
Received on 2023-02-12 16:43:11