Hi all,

I am answering some of the main points here in a single email, as
briefly as possible, to hopefully make it easy for all of you to
read/handle.

If I forgot any important point, please let me know!



# Jens Maurer

> In order to solve these issues satisfactorily, we probably need a research effort approximately on the same level as improving our memory model for multi-threaded operations. This was a multi-year effort involving top-notch experts in the trade.

Perhaps (I don't disagree with you on this, as you know), but the
multi-threaded memory model is a good example of how something can
work in practice, given programs using it were a thing decades before
the memory model was formalized into the standard. Of course, it was
not in the standard, but `memset_s` provides the example of something
useful in the standard without being fully formal yet.

> I suspect there are different engineering views at work here.

Very true.

> From a standardization perspective, ISO requires us to clearly state the criteria to judge whether an implementation conforms to the standard.

That is a good point Jens -- but being hand-wavy (and/or giving
flexibility) and being conforming seem alright together. For instance,
`memset_s` itself in ISO C or the optionality of some of the Annexes.
I agree that it is better to try to find the most formal solution, of
course, but not to the point of stopping standardization if it is not
found (as long as it is good enough and/or intent is clear; please see
next point and my answer to Niall's below).

> I'm arguing that, under the status quo of the abstract machine, an implementation replacing secure_clear with memset is conforming.

If the intention in the standard is clear, I would argue it is not
conforming "in spirit", i.e. maliciously conforming (which no
implementor wants to be since they are trying to deliver a useful
product).

Further, since being conforming "in spirit" is easy for implementors
(in this case), there is a very high chance that all major ones will
be, I'd argue.

> Then, maybe something like this would focus the attention a bit: [...]

All those pointers you wrote are very useful, thank you! I will see
what I can cook up...



# Nevin Liber

> The burden is on you to either defend why hand-waving is enough or to propose changes to the abstract machine.

I was not arguing about defending hand-waving or not, just asking why
(in particular, "how much" here) hand-waving is a showstopper (noting
that this is not the first thing in the standard like that).

Otherwise, we could reject any proposal by claiming it does not have
property X, without the recourse to ask why property X is required!
;-)

> Please don't bring straw man arguments into this. In particular, note that neither Jens nor anyone besides you mentioned subjective claims like "purity", "beauty", etc.

This is not a new discussion -- there have been related claims in past
ones (with a lot of "color" from all sides! ;-)

I brought it up preemptively in hopes that we would focus on
discussing ideas on how this could be solved in the best possible way
(which is my reading of JFB's questions in the OP), rather than
rehearse and/or shoot down the proposal right away.

> Also, if memset_s were always available, why is it not sufficient to mitigate the threat?

It can be, and the paper itself explains that an alternative is simply
making `memset_s` mandatory. But given we are touching this, it seemed
a good idea to improve it. If people cannot agree, then my proposal
will be reduced to that which I hope people can agree to.

> Why do indeterminate values (instead of, say, zeros or some other bit pattern) need to be written?

We discussed this in a previous meeting. There is the approach of
having a `memset_s`-like function, or to provide a more "meaningful"
function. For the latter, nobody should care about the actual value
written (or if anything is written at all, in fact). So it seems more
flexible to leave the implementation to do whatever they think is best
for their platform (since this should not have a dramatic impact in
e.g. performance to make it unsuitable for some users).

> More importantly, the threat this is supposed to mitigate needs to be described in much greater detail.

A WG21 paper does not seem the right place for that. The use case in
the field is already documented with extensive references in the paper
and a summary is given as an introduction.

> Right now, it looks like the only threat it mitigates is one on a single core machine with a von Neumann architecture and a not too aggressive compiler, and that to me is not a solution worth standardizing.

None of that seems true to me:

  - A typical multi-core system with layered caches and a complex
invalidation protocol will have more chances of leaking stuff in many
ways, but that does not mean memory is not the prime target anymore
(and it is what users have cared about the most so far).

  - Optimizing compilers are the problematic ones as discussed in the
paper, not the opposite.

  - Whether a uarch is VN/H is orthogonal to the issue (the problem
happens in both).

Further, the proposal explains that this does not preclude
standardizing more things in the future as they appear to be used in
the field, but we have to wait until that happens. We should not be
inventing here.

> Now, if we can guarantee that all copies of the data that the program can see are obliterated, across all cores, registers and caches, that could be a solution worth standardizing.

Even assuming ISO C++ would cover hardware details, we cannot
guarantee that since no compiler that I am aware of does anything
close to that. So there is no chance we could standardize that.

However, I agree that we could leave the flexibility for implementors
to do the best they can. The standard would only make them guarantee
only the basic part (clearing the memory).

Previous revisions were more in line with that. The problem is that,
if we do allow flexibility, then we risk implementors diverging and
then projects not being able to use it everywhere for e.g. performance
reasons. Since there is no one that implements anything further at the
moment, it seems risky to provide flexibility and therefore it seems
the best is to standardize the simplest operation for the moment.



# Niall Douglas

> I will apologise in advance Miguel if the following comes across as patronising. However I think it would aid you in writing in the appropriate terminology

No apologies required, and thanks a lot for any help! :-) I knew about
the points you are discussing, although I am no expert, of course.

> Bringing things back to secure_clear()/always_clear(), if Core don't like a hand waving magical function like memset_s() is in the C standard, and they don't like a wording based around intrinsic directives about escape analysis to the optimiser (which was my attempt at this), then that leaves one remaining option: you need to go tweak the language in the abstract machine around volatile objects.

Good summary! I wouldn't divide it into Core liking it or not, though.
My approach has always been trying to get this into the standard in
the best form as possible. That is why, so far, the proposal shows the
simplest solution I found: basically making it the same as `memset_s`
which is already in ISO C.

> I wish you good luck in your endeavours, and for the record, thank you for investing your effort to date as this is a problem which is holding up a ton of other stuff, far far wider than bit clearing (think shared memory, object serialisation to mapped memory, etc).

Thanks a lot! I will certainly need it, it seems I chose a hairy topic
as my first proposal :-)



# Hubert Tong

> I believe Richard is very correct here. P1315 proposes a new mechanism insofar as it suggests that a compiler may (and is encouraged to) avoid involving actual memory for operating on the "secrets" in the first place and thereby gain permission to omit the clearing operation. This suggests a compiler built-in, which is almost the complete opposite of any solutions that depend on the definition of the clearing functions being unknown to the compiler.

I see your concern now. I intended the bit about avoiding memory as an
additional freedom compiler may want to take, but not an obligation
("unless" etc. in the paper). We may leave it as a non-normative
footnote or simply remove it if nobody is convinced.

> In the worst case, the standardized functionality is implemented in a compliant manner that offers less security than the "workarounds" used by the projects. This may be caused by "invention" on the part of the paper or implementers of such in the form of a new compiler built-in.

I addressed the "invention" part in the previous part -- to be clear:
it is not my intention to invent here at all. In fact, the biggest
argument for the proposal is the fact that there are a lot of users
out there.

(In fact, we already removed `secure_val` (a RAII facility using
`secure_clear`, basically) from a previous revision since it was
already "too new".)

> Although, the paper should probably suggest why users either should not want to clear non-trivially-copyable types [...] also make some mention of whether there are meaningful semantics for the function in constant expression evaluation

Both good points -- I will consider how to address them in the paper, thank you!



# JF Bastien

> I don't want to argue this, I'm trying to state facts on various perceptions, and the effects that come out of this. Maybe the paper can discuss this dichotomy in more details to save everyone some time?

I can definitely add more details on this. I hope making the paper
even longer is not a hindrance for people reading it fully ;-)

> That's an interesting approach. Miguel, would you be able to do this?

I will try -- just please be patient. It will take quite a lot of time
to study this better and try to produce some useful wording in my
spare time. But the input I have been receiving the past few months is
very useful to get started.

As usual, thanks for helping me reach experts within the different subgroups.



Cheers,
Miguel