On Tuesday, 18 July 2023 01:08:44 PDT Lénárd Szolnoki wrote:
> How does one replace operator new and other replaceable functions
> within a DLL? Is it possible with only the possession of the DLL
> itself, or does this require recompiling and/or relinking the library?

It requires relinking.

The way this normally works is that those functions are *not* marked
__declspec(dllimport), so the linker replaces the near call to the local copy
with a stub that loads the address from the IAT and jumps there. This is very
similar to how PLTs are implemented in ELF systems.

But when the set of object files being linked together provides this symbol,
the linker does not insert a stub and instead resolves the call to that
symbol. However, once linked, you can't change this. The call either resolves
to the IAT, which is a two-level lookup and therefore specifies explicitly
which DLL the symbol must come from, or it resolves locally to a function.

Do note that Mach-O also provides two-level lookup, but the problem of
operator new and delete is solved by marking those symbols weak in libc++.

> AFAIK one of the design goals of contracts is to allow companies to
> still only release two builds: one debug and one release build, and
> allow users the replace the contract violation handler for the debug
> library if they want to do so. I don't know if this is possible with
> replaceable functions within DLLs on Windows.

So long as they provide sources or at least the .obj files, that can be done.

-- 
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
   Software Architect - Intel DCAI Cloud Engineering