Here is my suggestion for how to implement JIT. Virtual types.

The basic idea:

 Virtual context switch - intended to be implemented as jit recompilation
 Generic virtual context - a context where a virtual value is changed that results in the value being non-jit and instead the context is treated similar to type erasure.

A variable of constexpr-compatible or template type with at least one value-type parameter may be declared virtual by declaring the virtual keyword before the type. The header <jit> must be included before declaring any virtual variables or templates.

Example:

 virtual int i = 0;

A a virtual value may occur in an expression that would otherwise require constexpr values provided it is prefixed with the virtual keyword, but the keyword virtual must be prefixed if any of the inputs are non-virtual or generic virtual:

 foo<virtual i> bar; // here "i" is a virtual value, virtual context switch not required
 virtual foo<virtual i> bar; // here "i" is a non-virtual value, virtual context switch required.

Upon entry to a scope which a virtual value or template is declared, the abstract machine enters a "virtual execution context". The abstract machine can switch to a different "virtual execution context" by reassigning the value of a virtual variable. Values of virtual type may only be reassigned in a generic virtual context. A generic virtual context is either an expression in a virtual execution statement of the form "virtual" "(" <expression> ")" or a collection of statements contained within a virtual block "virtual" "{" <statements> "}". The implementation should remain in the "generic virtual context" until the end of the virtual expression or virtual statement block.

A reference to a variable may also be declared virtual (the virtual qualifier). A virtual value may only be mutated by virtual references. The virtual reference is implicitly convertible to const unless the virtual type contains mutable members.

Examples:

 virtual int foo = 0;
 ...
 virtual(foo = 5); // virtual context switch

 virtual int a = 0;
 virtual int b = 0;
 ...
 virtual {
  // entry into generic virtual context
  a = c();
  // no virtual context switch here
  b = c()+2;
 }
 // the function has entered a new virtual context here


A virtual variable may be declared that does not cause a virtual context switch using the virtual(const) pseudo keyword:

 virtual(const) int a = 0;
 virtual(const) int b = 0;

A reference to a value may be virtual:

 int a = ...;
 virtual int & b = a; // OK, virtual context switch
 int c & = b; // error, discarding virtual qualifier
 int const & d = b; // OK
 virtual(const) int & e = b; // OK, generic virtual context, e is not const.

The virtual(const) keyword may also appear in a virtual expression:

 virtual int a = 0;
 virtual(const)(a = 5);

In which case the function enters a limited generic context in which case virtual variables referred to in a non-const context are treated as appearing in a generic virtual context until the next virtual context switch. Note: A virtual template cannot appear in a generic virtual context unless the template value parameters are declared virtual, in which case the program is ill formed if any possible value would result in an ill-formed parameter. Any such declaration results in a virtual context switch. End note.

A modification of a virtual variable does not affect the type of templates already instantiated with it.

Instantiation of a virtual template or a template with a non-generic virtual parameter may throw an exception of type std::bad_jit.
e.g.

 template <std::size_t Sz>
 class foo
 {
   int a[Sz];
 public:
   ...
 };

 int i = -5;
 virtual foo<virtual i> f; // throws std::bad_jit

 virtual int i = -5;
 virtual std::array<int, virtual i> bar; // throws std::bad_jit
 
 int i = 5; // OK
 virtual std::array<int, virtual i> baz; // OK, virtual context switch

 virtual int i = 5; // OK, virtual context switch
 std::array<int, virtual i> baz; // OK, already in a virtual context
 virtual(i = 3); // OK, does not affect the type of baz

Why is this better than [[jit]]? First, attributes are not supposed to be required, they are supposed to be ignorable. The paper's example test_jit_sz<type, size>(repeat); definitely does not meet this criteria. I think that (ab)using the virtual keyword might provide a way to annotate the intention with a syntax that does not break any existing valid code whilst making it very clear to the reader that something funny is going on.

Formalizing where virtual context switches (jit recompilations) occur also has other advantages.

Perhaps we could also allow variables of type "virtual(typename)" to allow type-as-value expressions for use in jit compilation?

Some questions, should it be allowed to throw a virtual type? How to treat virtual subobjects and heap allocations? We could say that virtual objects can only exist in automatic storage and are equivalent to non-virtual ones outside that context?

-----Original Message-----
From: Std-Proposals <std-proposals-bounces_at_lists.isocpp.org> On Behalf Of Avi Kivity via Std-Proposals
Sent: Saturday, September 21, 2019 2:37 PM
To: c std-proposal <std-proposals_at_lists.isocpp.org>; hfinkel_at_anl.gov
Cc: Avi Kivity <avi_at_scylladb.com>
Subject: [std-proposals] Feedback on P1609R1, C++ Should Support Just-in-Time Compilation

*External Message* - Use caution before opening links or attachments

https://urldefense.com/v3/__http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1609r1.html__;!9cTk8lEYrHI!9-lXib0LHqU9r3AsCglrfHjN6n-DsoLgVoVKvOqAYD-CfYIjt776wfDGatTQKSk$


First, this would be very useful to me. There is a class of servers, which includes databases, that need to serve a wide variety of queries.
There queries involve processing an interpreted language, and so have a high abstraction cost: copying, comparators, arithmetic operators all involve virtual calls, and all must be prepared for the most complex data types. It is not feasible to specialize these queries ahead of time as every user has their own schema and their own queries.


My feedback:


1. Although it is implied, I would like it to be explicit that non-type template parameters of class type are legitimate run-time template parameters for jit. For example, I'd like an expression class that defines some expression to be evaluated, and then


    template <expression e>

    [[jit]]

    value evaluate(std::vector<value> inputs) {

          ...

    }


would allow me to specialize evaluation at runtime. `expression` would be a tree-type object including nested expression objects, so it would need the full power of C++2a constexpr containers.


2. The proposal asks whether the jit indicator should be in the declaration, or point of use. I believe it should be at the point of use. This would allow using a type-erased version of the template function compiled ahead of time, while some other thread invokes the jit compiler and waits for compilation to complete. Or perhaps, the server would only compile the function after the type-erased version has been executed a sufficient amount of times to warrant jit. This also allows limiting server memory allocated to jit functions.


3. Some mechanism should be allocated to ask how much memory a jit function uses, and to allow unlinking and deallocating a jit function after it is no longer used.


Thanks for the proposal! I hope it makes it into C++23 or even some earlier technical specification.


--
Std-Proposals mailing list
Std-Proposals_at_lists.isocpp.org
https://urldefense.com/v3/__https://lists.isocpp.org/mailman/listinfo.cgi/std-proposals__;!9cTk8lEYrHI!9-lXib0LHqU9r3AsCglrfHjN6n-DsoLgVoVKvOqAYD-CfYIjt776wfDGVS6jTMY$