Date: Sun, 25 Oct 2020 13:12:25 -0400
> On Oct 25, 2020, at 11:19 AM, Andrew Sutton <asutton.list_at_[hidden]> wrote:
>
>> Good metaprogramming is semantic, not based on what a string happens to mean when interpreted in a particular context.
>
>
> Is it? Programs are written in text. The user expresses their desired semantics via textual syntax. (It’s even a bit arbitrary, frankly; e.g. musical instructions aren’t written in text.) If you give users the ability to manipulate the textual input sent to the compiler, during constant evaluation in such a way they can depend on the already-compiled information of their program while doing so, there is certainly no conceivable metaprogramming task they cannot do. (Note here too that arbitrary recursion works as well: `constexpr { __queue_metaparse("constexpr { __queue_metaparse(\"int i = 42;\"); }"); }`).
>
> Yes. The semantics of a program are derived directly from the text you write. For a well-formed and correct program there is (ideally) a one-to-one correspondence between the text you write and the behavior. I think the language works hard to preserve that property.
>
> That isn't the case with token-based or string-based metaprogramming. In that model, you write what you hope is a well-formed and correct program, but you can't know if you've succeeded until it's been injected.
>
>
> The same cannot be said of fragment injection — because they don’t operate on text, they can never be said to step fully outside the program. I.e. there is almost certainly a tradeoff between power and safety. Fragments choose safety - but when asked to push the limits with heavy duty higher order metaprogramming, will they be found wanting?
>
> Are fragments also not text? I'm pretty sure they are. It's just that the semantics of a fragment are determined like any other part of the program, although specifically more like templates than other parts.
>
> I don't believe that tradeoff is real. Fragments support both concatenation (you can inject into a fragment) and interpolation (fragments can be nested, and we explicitly support interpolation). Is there some other compositional capability that strings offer?
>
> That said, I think there are things that are harder to write with fragments, and I'd like to find more expressive ways of addressing them. But I don't believe that fragments are less powerful simply because they are safer.
>
>
> On the other hand: might it be true that anything string injection can do, but fragment injection cannot, is necessarily unhygienic? That it can’t possibly be the best or only way to achieve something? Maybe. I could be convinced. But if there is any doubt, then perhaps C++ should err on the side of giving users too much rope, not too little. (Why should C++ leave any room for a higher level language either?)
>
> That's a good question. You can certainly write non-code in strings, but not fragments. Code that appears correct but turns out not to be is probably unhygienic.
>
> I don't think that extra rope can be put to effective use.
>
To summarize my position: I support expanding `|# ... #|` (which also has the problem that the generated identifier may not be determined to be semantically valid until instantiation, or am I missing something?) to allow string injections in as many contexts as possible, to help the user to specify precisely what is dependent in their metafunctions, and precisely what is not, up to and including full statements and declarations should it ever be necessary.
Along these lines I also provided in that old implem a means (however unpolished) of producing expressions of guaranteed types from string literals: `__metaparse_expr("3", int)`, which provided some interesting capabilities, in my case to implement constexpr containers, before that sort of thing was standardized. (Here’s a sample constexpr vector implem: https://github.com/drec357/clang-meta/blob/master/examples/include/ce/vector.hpp, here’s examples: https://github.com/drec357/clang-meta/blob/master/examples/4_constexpr_containers.hpp).
The user should always endeavor to leave as much information *out* of |#…#| as possible, to allow the compiler to verify as much of the semantics as possible prior to instantiation.
But should the user decide that there is no other way to implement what they need than to put whole chunks of declarations/statements in quotations and defer parsing them until instantiation, introducing the possibility of parse errors during instantiation, I think we should give them the rope, unless it has been definitively proven that this can absolutely never be necessary, even under examples that really push the limits.
I am content the case has been made and will leave it to you all — thanks again for your hard work, and thank you Andrew for making your implementation widely available, and *in particular* for documenting it so well that I was able to base that experimental implementation atop it. (And Richard, likewise of course.)
Dave
>
>> Good metaprogramming is semantic, not based on what a string happens to mean when interpreted in a particular context.
>
>
> Is it? Programs are written in text. The user expresses their desired semantics via textual syntax. (It’s even a bit arbitrary, frankly; e.g. musical instructions aren’t written in text.) If you give users the ability to manipulate the textual input sent to the compiler, during constant evaluation in such a way they can depend on the already-compiled information of their program while doing so, there is certainly no conceivable metaprogramming task they cannot do. (Note here too that arbitrary recursion works as well: `constexpr { __queue_metaparse("constexpr { __queue_metaparse(\"int i = 42;\"); }"); }`).
>
> Yes. The semantics of a program are derived directly from the text you write. For a well-formed and correct program there is (ideally) a one-to-one correspondence between the text you write and the behavior. I think the language works hard to preserve that property.
>
> That isn't the case with token-based or string-based metaprogramming. In that model, you write what you hope is a well-formed and correct program, but you can't know if you've succeeded until it's been injected.
>
>
> The same cannot be said of fragment injection — because they don’t operate on text, they can never be said to step fully outside the program. I.e. there is almost certainly a tradeoff between power and safety. Fragments choose safety - but when asked to push the limits with heavy duty higher order metaprogramming, will they be found wanting?
>
> Are fragments also not text? I'm pretty sure they are. It's just that the semantics of a fragment are determined like any other part of the program, although specifically more like templates than other parts.
>
> I don't believe that tradeoff is real. Fragments support both concatenation (you can inject into a fragment) and interpolation (fragments can be nested, and we explicitly support interpolation). Is there some other compositional capability that strings offer?
>
> That said, I think there are things that are harder to write with fragments, and I'd like to find more expressive ways of addressing them. But I don't believe that fragments are less powerful simply because they are safer.
>
>
> On the other hand: might it be true that anything string injection can do, but fragment injection cannot, is necessarily unhygienic? That it can’t possibly be the best or only way to achieve something? Maybe. I could be convinced. But if there is any doubt, then perhaps C++ should err on the side of giving users too much rope, not too little. (Why should C++ leave any room for a higher level language either?)
>
> That's a good question. You can certainly write non-code in strings, but not fragments. Code that appears correct but turns out not to be is probably unhygienic.
>
> I don't think that extra rope can be put to effective use.
>
To summarize my position: I support expanding `|# ... #|` (which also has the problem that the generated identifier may not be determined to be semantically valid until instantiation, or am I missing something?) to allow string injections in as many contexts as possible, to help the user to specify precisely what is dependent in their metafunctions, and precisely what is not, up to and including full statements and declarations should it ever be necessary.
Along these lines I also provided in that old implem a means (however unpolished) of producing expressions of guaranteed types from string literals: `__metaparse_expr("3", int)`, which provided some interesting capabilities, in my case to implement constexpr containers, before that sort of thing was standardized. (Here’s a sample constexpr vector implem: https://github.com/drec357/clang-meta/blob/master/examples/include/ce/vector.hpp, here’s examples: https://github.com/drec357/clang-meta/blob/master/examples/4_constexpr_containers.hpp).
The user should always endeavor to leave as much information *out* of |#…#| as possible, to allow the compiler to verify as much of the semantics as possible prior to instantiation.
But should the user decide that there is no other way to implement what they need than to put whole chunks of declarations/statements in quotations and defer parsing them until instantiation, introducing the possibility of parse errors during instantiation, I think we should give them the rope, unless it has been definitively proven that this can absolutely never be necessary, even under examples that really push the limits.
I am content the case has been made and will leave it to you all — thanks again for your hard work, and thank you Andrew for making your implementation widely available, and *in particular* for documenting it so well that I was able to base that experimental implementation atop it. (And Richard, likewise of course.)
Dave
Received on 2020-10-25 12:12:30