C++ Logo

std-proposals

Advanced search

Re: [std-proposals] Function overload set type information loss

From: Sebastian Wittmeier <wittmeier_at_[hidden]>
Date: Wed, 31 Jul 2024 09:37:51 +0200
For function pointers to instantiated templated functions you can use the uniqueness of the address to find out the template parameters.   While this may theoretically work for function pointers - I give you that - it won't work for your other examples like int or arrays.   Best, Sebastian   -----Ursprüngliche Nachricht----- Von:organicoman via Std-Proposals <std-proposals_at_[hidden]> Gesendet:Mi 31.07.2024 00:39 Betreff:Re: [std-proposals] Function overload set type information loss An:Tiago Freire <tmiguelf_at_[hidden]>; std-proposals_at_[hidden]; Jonathan Wakely <cxx_at_[hidden]>; CC:organicoman <organicoman_at_[hidden]>;     auto operator(int) is not responsible of reserving bytes on the stack for the returned value. The caller is the one taking care of that, and depending on the calling convention , it takes care of pushing the arguments into the stack also.   It's responsible for putting bytes on the stack. And the caller doesn't know it either.   Really... long long callee(); void caller() {    int retVal = callee(); } Do you reserve on the stack sizeof(long long) Or sizeof(retVal) ? The type in these examples maybe only be something that is known when the code is running. But the code needs to be compiled before that.  I understand what you are saying, but it is not relevant to what I'm proposing. It is all done in the compiler backstage, no evidence of any crime will be left behind. :-) > That's called template type deduction. Which happens at compile time. Key detail, compile time. Only the compiler knows the type. The compiler deduces what is meant at compile time based on the type it knows at compile time. There's no such thing as runtime template type deduction.  I never said such thing, maybe you miss understand what i wrote.   There's no such thing as time traveling compilation. Your application can't all of a sudden have different instructions being issued depending on some intangible spirit data.   Please read the implementation of overload resolution, then extend it to fit this proposal. ‐------------- My intention is to say that the compiler as our tool at the moment is streched beyond its capabilities. If you inspect the implementation of std::function, in stdlibc++, for example, you will feel empathy for the implementer. Seriously, union inside union, with union manager, then short buffer optimization then heap allocation... definitely it crys out loud : the tool is incapable to express that feature elegantly,  thus a new way of think is mandatory. So the best approach is to exploit what we have already, and the overload resolution mechanism seems under exploited. That's all. -------------------------------- From: organicoman <organicoman_at_[hidden]> Sent: Tuesday, July 30, 2024 11:11:12 PM To: Tiago Freire <tmiguelf_at_[hidden]>; std-proposals_at_[hidden] <std-proposals_at_[hidden]>; Jonathan Wakely <cxx_at_[hidden]> Subject: RE: [std-proposals] Function overload set type information loss     Type erasure doesn’t need an “answer”, it’s a feature.  I meant : it is useful for type erasure. The more I look at it the less it makes sense. First you want to add unrelated template types to this let’s call it “magic-type”, then you do this:   template<typename T> std::any prototype(int);   struct A{   using eff_type = effective_decltype(prototype);   Now you want to extract a type  from a thing that is not even a type. “prototype” has no type is a template to a function. In current standard you can prove without doubt that: \-/ T,  typeof(prototype) = std::any(*)(int); But the value of the address is generated when that templated function is instantiated. And it is a constexpr value. “prototype<anything>” has a type “std::any (*)(int)”, “prototype” is a template. But let’s pretend that that this is what you meant.  You pretented correctly. Let’s move on to this section of the code:      std::vector<A> vec    vec.push_back(A<&bar>{});    vec.push_back(A<&foo<double>>{});    vec.push_back(A<&foo<someObject>>{});    vec.push_back(A<&lmd>{});    for(const aut& elem: vec)        ActBasedOnType(elem(0));     And let’s put some meat on the bones: And let’s say that ActBasedOnType is this:   template<typename T> void AcfBasedOnType(T t) {                 std::cout << typeid(t).name() << std::endl; }     And correct me if I’m wrong, I assume that what you want to be printed is:   char const[6] double someObject someObject  You are correct.   I think what you are expecting is that the following function does the magic.      auto operator()(int)    {        using Ret = std::extract_1st_templ_type_t<eff_type>; // some type traits acting on effective types         std::any_cast<Ret>( m_memFun() );     }     Let’s analyze it line by line: using Ret = std::extract_1st_templ_type_t<eff_type>;  It was a mistake, i meant using Ret = std::exextract_1st_templ_type_t<effective_decltype(m_memFun)> Because at this stage, m_memFun is well constructed and the template type is resolved  What the hell is Ret? Well its “std::extract_1st_templ_type_t” of “eff_type” but what is “eff_type”? using eff_type = effective_decltype(prototype); oh… yeah, that thing that has no type…. did you mean “std::any (*)(int)”? Or maybe you meant “double” when “foo<double>”? Yes, that's it.   I assume that is what you meant based on your second line (You forgot you return there, let me fix that for you) [return] std::any_cast<Ret>( m_memFun() );  Thank you. But how the hell is it supposed to figure that out from looking at “eff_type”? Maybe you meant to look at “m_memFun”? Yes, it looks like you understand the concept very well !!   Maybe because of the constructor  template<auto F>   A() : m_memFun (F){}   You would expect “m_memFun” to be imbued with some form of “magic type” information.  Let say meta data. But then what happens if you had that?     eff_type m_memFun;   eff_type m_memFun2;     template<auto F>   A()     : m_memFun (F)   , m_memFun (bar)   {}  I guess you meant  m_memFun (F), m_memFun2(bar) I don’t know at this point, but lets just assume that this convoluted mess all works.  It will ... because eff_type is tagged with the template parameter which will be resolved at constuction. So for m_memFun, the eff_type will be tagged with meta data resolved from the type of F, if necessary, and m_memFun2 is just std::any(*)(int) with empty meta data. But remember C++ is a compiled language, so we have to build some machine code at some point and let’s try to do that:  That's correct Ok we have a function: auto operator()(int);   How many bytes should I reserve on the stack? Well, I guess sometimes is 1 when it’s “someObject”, sometimes it’s 8 when it’s a “double”.   But there can only be one “auto operator()(int)”, so it must return both 1 byte and 8 bytes, and the type changes? auto operator(int) is not responsible of reserving bytes on the stack for the returned value. The caller is the one taking care of that, and depending on the calling convention , it takes care of pushing the arguments into the stack also. What about “AcfBasedOnType”?   “AcfBasedOnType<double>” “AcfBasedOnType<char const[6]>” “AcfBasedOnType<someObject>”   Are different function, and the compiler needs create different instructions for those. That's called template type deduction. Followed by implicit function template instantiation. Then return value optimization. But what is going to be on the vector is not even known until runtime, and that that point there’s no source code to figure out what any of these things meant. The vector is of type  std::vector<A>; //well defined The constructor of A is templated on a non type template : auto F That's what I'm pushing back into the vector: A<func_ptr>{ } How would the compiler know to generate “AcfBasedOnType<someObject>”? Is your application supposed to travel back in time and tell the compiler to issue new instructions and reset reality? Is this a quantum computing thing where you can have overlapping realities?  See answer above about how compiler deduces template types If C++ was an interpreted language like Python, sure, you could get this information… in Python… because it is interpreted and it needs to track the type of the objects being passed around, and it is directly working from the source file to re-interpret what these things mean.     This is the most charitable interpretation that I could make of your proposal. It’s a convoluted mess. This does not work. Sometimes i feel you understand it perfectly, actually you were correcting it as i should have wrote it !! And sometimes i feel you don't want to believe, which makes you refute it. I’m 100% sure you have absolutely no idea what you want to do either.  I will untangle your questions one by one, slowly... don't worry What about this? template<int Val> int globalVar1 = Val; template<int Val> int globalVar2 = Val + 1; is “effect_type(globalVar1<37>)” equal to “int<37>”? is “effect_type(globalVar2<37>)” also equal to “int<37>”?  Yes, and yes, and it is already the case with the current standard. std::is_same_v(decltype(globalVar1<37>, decltype(globalVar2<37>)> // Yields true This is such a convoluted mess, that I personally believe that you generated these examples out of some highly hallucinating ChatGPT or some Gemini or something. Seriously ... I understand the words, but none of this makes any sense Try now with the clarification above   has programmed professionally for a sufficient amount of time.  No doubt , i read sometimes your answers in other threads and most of the time they are very constructive and valuable,i saved some of them for me to go back to it. The general observation, is that you lose your tamper quickly... honest observation.    Any other ambiguity? From: organicoman <organicoman_at_[hidden]> Sent: Tuesday, July 30, 2024 17:59 To: Tiago Freire <tmiguelf_at_[hidden]>; std-proposals_at_[hidden]; Jonathan Wakely <cxx_at_[hidden]> Subject: Re: [std-proposals] Function overload set type information loss         And I'm telling you it's not a thing that makes sense.   There's no such thing as "int instantiated for double", all "int"s are just "int". This is not information about the type that gets discarded, it was never part of the type to begin with. Nor it should ever be.   What kind of meaningful information about the type could this be useful for? Good question. Answer Type erasure And type introspection  Look at this imaginary example PS: this is not valid C++ syntax, but if the idea is implemented we can write it this way    template<typename T> std::any prototype(int);   struct A{   using eff_type = effective_decltype(prototype);     eff_type m_memFun;     template<auto F>   A()      : m_memFun (F)   {}      auto operator()(int)    {        using Ret = std::extract_1st_templ_type_t<eff_type>; // some type traits acting on effective types         std::any_cast<Ret>( m_memFun() );     } };   std::any bar(int) { return "hello"; } // plain function   template<typename T> std::any foo(int) { return T{}; } // function template with effective_ type != apparent type.   struct someObject {};   template<typename T> void AcfBasedOnType(T t) { // do something }   int main() {    constexpr auto lmd =[](int)->std::any { return someObject{}};    std::vector<A> vec    vec.push_back(A<&bar>{});    vec.push_back(A<&foo<double>>{});    vec.push_back(A<&foo<someObject>>{});    vec.push_back(A<&lmd>{});    for(const aut& elem: vec)        ActBasedOnType(elem(0)); }   As you can see above, i can hide the template type of foo inside its effective type, and recall it back in the calloperator of A through effective_decltype type alias. Yet the vector is holding the same type A. No information lost , all recycled.   Plus for any program relying on the apparent type of m_memFun,  it will see the type of m_memFun as void(*)(int)  That's why i can create A with the address of lmd and bar. I hope itit's illustrative enough        --------------------------------   From: organicoman <organicoman_at_[hidden] <mailto:organicoman_at_[hidden]> > Sent: Tuesday, July 30, 2024 4:16:35 PM To: Tiago Freire <tmiguelf_at_[hidden] <mailto:tmiguelf_at_[hidden]> >; std-proposals_at_[hidden] <mailto:std-proposals_at_[hidden]> <std-proposals_at_[hidden] <mailto:std-proposals_at_[hidden]> >; Jonathan Wakely <cxx_at_[hidden] <mailto:cxx_at_[hidden]> > Subject: RE: [std-proposals] Function overload set type information loss   Tiago, I seriously appreciate your contributions very much. But please try to read the thread from beginning and accumulate the idea's details then try to understand it as whole picture.   Also I'm not discussing the current tools available with the current standard,  I'm just using them to illustrate a point, and invent other tools to explain the idea. So, when i write, (int)<double>, i know it is not correct C++, but it is just merely a way to express a variable template of type int instantiated for type double. That's all.       Sent from my Galaxy     -------- Original message -------- From: Tiago Freire <tmiguelf_at_[hidden] <mailto:tmiguelf_at_[hidden]> > Date: 7/30/24 5:55 PM (GMT+04:00) To: std-proposals_at_[hidden] <mailto:std-proposals_at_[hidden]> , Jonathan Wakely <cxx_at_[hidden] <mailto:cxx_at_[hidden]> > Cc: organicoman <organicoman_at_[hidden] <mailto:organicoman_at_[hidden]> > Subject: RE: [std-proposals] Function overload set type information loss   > template<typename T> > int globalVar = 123; > In theory the entity:  glabalVar<int> > Is not the same as : globalVar<double> > Otherwise why we specialize based on template type at all.!!?? We can just write globalVar   Nope. They are all “int”. The type of “globalVar<double>” is an int, the name is “globalVar<double>”. There’s no “globalVar” whose type is “int<double>”.           From: Std-Proposals <std-proposals-bounces_at_[hidden] <mailto:std-proposals-bounces_at_[hidden]> > On Behalf Of organicoman via Std-Proposals Sent: Tuesday, July 30, 2024 15:49 To: Jonathan Wakely <cxx_at_[hidden] <mailto:cxx_at_[hidden]> > Cc: organicoman <organicoman_at_[hidden] <mailto:organicoman_at_[hidden]> >; std-proposals_at_[hidden] <mailto:std-proposals_at_[hidden]> Subject: Re: [std-proposals] Function overload set type information loss   Thank you Jonathan for your discussion,  I just want to draw your attention that what I'm talking about is not available within the current state of the standard.  To understand this idea you must untie your mind from the rules set by the current state of C++. Be abstract.    Let me explain more: In the current C++ standard we have 3 objects that lose their type information by design. Namely: 1- functions  2- arrays 3- variable template. At the compilation stage, we preserve the whole type signature of the above objects and we manipulate it,  but as soon as we change the block scope we lose part of that information. e.g template<typename T> int globalVar = 123;   In theory the entity:  glabalVar<int> Is not the same as : globalVar<double> Otherwise why we specialize based on template type at all.!!?? We can just write globalVar   Value wise they are both equal globalVar<double> == globalVar<int> // true   But type wise they should not. Yet the current standard says that: std::is_same_v<decltype(globalVar<int>), decltype(globalVar<double>)> // true.   This is incorrect in type's theory point of view. The rule says that, if you specialize any type based on a template, the two types are different, even if their values are equal.   Given the above observations, and the current state of C++, If we try to change the standard to implement the rule above for the 3 categories above, we will break a lot of code.   So what is the solution? My solution is to divide the type of an object from the 3 categories above, into 2 parts: 1- an apparent type, which is implemented currently in the standard  2- an effective type, that reflects the real type at the time of its instantiation.   To implement the 2nd I suggested an operator called: effective_decltype ();   How it works: When you pass one object from the 3 listed above, it will return its full signature.   e.g. decltype(globalVar<double>) // is int , but   effective_decltype(globalVar<double>) // should return (int)<double>, to say that globalVar is a variable template of apparent type int instantiated for double   Take this concept and generalize it to arrays and functions.   So one can ask, how to implement this?   Remember that when you call, for example a function let say: // declaration of function  void useGlobalVar(int v); // usage of function  useGlobalVar(globalVar<double>);   The type is right there at the call site, you can capture it inside the function body using the suggested operator: effective_decltype()   Even if you do: auto var = globalVar<double>;   The auto key word behavior should be changed to capture the effective type of right hand side of the assignment. So when we call   useGlobalVar(var);   The effective type is preserved.   One exception though is when you write: int var = globalVar<double>;   Here you are explicitly casting the effective type of var to be a plain int.   In conclusion: I know it is a liitle bit twisted abstract math, but if you squint just a little bit, you can catch it. Also the implications of this approach is very very beneficial.     Sent from my Galaxy     -------- Original message -------- From: Jonathan Wakely <cxx_at_[hidden] <mailto:cxx_at_[hidden]> > Date: 7/30/24 4:04 PM (GMT+04:00) To: organicoman <organicoman_at_[hidden] <mailto:organicoman_at_[hidden]> > Cc: std-proposals_at_[hidden] <mailto:std-proposals_at_[hidden]> Subject: Re: [std-proposals] Function overload set type information loss       On Tue, 30 Jul 2024 at 11:09, organicoman <organicoman_at_[hidden] <mailto:organicoman_at_[hidden]> > wrote:     This technique of apparent and effective type, could be generalized to C style arrays and variable templates. For example:   size_t foo(const char* arr) {  //inside the function arr type is just a pointer  // but it's real type has an array extent N.       using eff_type = effective_decltype(arr);     return extract_array_size<eff_type>::value;   How would this be possible if 'foo' is separately compiled? How can the value returned by 'foo' depend on information only available at the call site, not in the function body? How would you avoid ODR violations, and how would you even implement this?   The same way you implement  void foo(auto arg); // c++20   OK, that's creates a function template, equivalent to:   template<typename T> void foo(T arg);   So the equivalent for your example would be a function template too, which you can already write in C++ today (and all the way back to C++98):   template<std::size_t N> size_t foo(const char (&)[N]) { return N; }   And this already exists as std::size.   There's no need for your effective_decltype and extract_array_size magic, which are not implementable. For a non-template function, what you want is not possible. For a template function, you can do it already.   The reason you don't generally want to do that for more complicated functions is that you get a different foo<N> specialization for every different string length:   foo("1"); foo("12"); foo("123");   Each of these creates a separate specialization, so generates more code and increases your binary size.         }   template<typename T, typename V > int gInteger = 1234;   int main() {     char arr[]="cool_idea";     foo(arr); // prints 9   Why not 10? It's a const char[10] because it's a null terminated string.            auto i = gInteger<double,char>;     // type of i is integer     std::is_same_v<                 effective_decltype(i),                 (int)<double, char>)>; //<- this is a new notation to express a variable template  }   The whole idea is revolving around keeping the real type of each object, even if it decays to pointer ( ex. Functions and Arrays), or it loses the full type signature ( like variable templates,  and function templates).   This sounds like magic. No,  When you are writing a program you are explicitly passing variable names of fixed types between functions and objects, but as soon as you read data from IO, you will rely on a chosen type(picked by yourself) to interpret that data. So, when the compiler is crunching the source code, it keeps record of all declared types,   We can do that today with templates. You're trying to make non-templates behave like templates, and what you want is impossible in separately compiled code.   Just the current standard chose to omit and drop the effective type of arrays and functions to be of type pointer (decaying). I guess its a C legacy! But now we have the tools to rectify this loss of information.    No you don't. If your 'foo' function is defined in a separate source file there is no way for it to get the information you want. It's impossible.       After all, a C++ source code, is just a series of inspections and manipulation of types. So the more we keep information about types the more it is useful.   What you probably want is reflection, which would not allow you to write your size_t foo(const char*) function, but if you write different functions, reflection would allow you to inspect and manipulate more information about your program.   The proposed idea will help in implementing reflection.   It's not needed.   Take this example:   template<typename T> size_t    type_hash_value = typeid(T).hash_code();   void bar() {     /* we will use the apparent type of a variable to instantiate a vector. And its effect type to record the type*/     /* remember the new notation of a variable template mentioned in example above: (type_id)<Ts...> */       std::vector< (size_t)<T> > vec;     vec.push_back(type_hash_value<int>);     vec.push_back(type_hash_value<double>);     vec.push_back(type_hash_value<someT>);       size_t val;     std::cin >> val;     // IO input     vec.push_back(val);    /*apparent type of val is the same as its effective type => val is not a variable template yet it is still a valid entry to the vector*/       for(const auto& elem: vec)        if ( std::is_same_v<effective_decltype(elem),                             (size_t)<double>>            )            return elem; }   The elements of the vector above have two types of information: compile time and runtime. The size and alignment of the elements is the same as std::size_t, that's for runtime. The effective type of the elements is available at compile time only.   With this facility you can implement a program wide type_id hash table, and you can go back and forth between type<->value.   It is worth mentioning that, if the effective type and the apparent type are the same, this means that the inspected object is not templated, nor decayed.   I'm not 100% sure how this would be implemented, i need more research, but i guess compiler wizards out there have the necessary knowledge.  For now it is just a floating idea.     See "Reflection for C++26" https://wg21.link/p2996 for the active work in this space.   Yes, I'm already following its progress closely.        Hello Gents, Let: O(f) be the Set of 'function overloads' of the function  named "f".   Let: "Ret" be the return type of function "f".   The elements of O(f) can have 2 forms:   1-> Ret f ( Vargs )    2-> Ret f <Ts...> ( Fargs )    Where: 1-> "Vargs"is a variadic list of arguments types, participating or not in template type arguments deduction if "f" is a template. e.g:  template<typename T, typename V> void f(T, V, double, int) Vargs = {T, V, double, int}   2-> "Fargs" is a fixed list of arguments types, not related to any template type parameter. e.g:  template <typename T, typename V> void f(double, int) Fargs= {double, int} // T,V are not in the list.   "Ts" is just a variadic list of template parameters types.   The type of any function of the 1st form is: decltype (f) = Ret(*)(Vargs),  which keeps information about the function's type template parameters participating in the function argument's list.   But the type of the 2nd form is: decltype (f) = Ret(*)(Fargs); No mater what the template parameters are, the type of the 2nd form always decays to: Ret(*)(Fargs)   And always lose any type information about the function's type template parameters   Yet, when we want to get the address of such function,  we are obligated to use the template types in the function name. e.g: auto select_f = &f<Ts...>;   Otherwise we get overload ambiguity,  This is a proof that the template arguments participates in the function type.   In my opinion,  the compiler should keep the template arguments type information.   I know that changing the type of "select_f" in the example above will break a lot of code. But i have a suggestion.  If the compiler can keep record of : * an apparent function type ; (the usual one)    decltype (&f<Ts...>) = Ret(*)(Fargs) * and an effective function type which is:    decltype (&f<Ts...>) = Ret(*)<Ts...>(Fargs)    This would fix the problem without breaking any neck.   Why is this useful? Take this example:   struct Erased{ std::any (*m_fun) (void);   template<auto Func> constexpr Erased()   : m_fun(Func) { }   auto operator ()() {     using f_type = effective_decltype (m_fun);     // imagin we have a type traits that     // extracts template types.    using T = extract_1st_template_type<f_type>;         return std::any_cast<T>( m_fun(void) ); }   };   template <typename Ret> std::any foo() { return Ret{}; }   int main() {   std::vector<Erased> vec;   vec.push_back(Erased<&foo<int>>{});   vec.push_back(Erased<&foo<double>>{});   vec.push_back(Erased<&foo<some_type>>{});   for(const auto& elem: vec)  DoSomethingBasedOnReturnType(elem()); }   Using this technique we can store the template type, then recall it back. I guess it will make type erasure more effecient.   Any thoughts?   Regards Nadir     Sent from my Galaxy   -- Std-Proposals mailing list Std-Proposals_at_[hidden] <mailto:Std-Proposals_at_[hidden]> https://lists.isocpp.org/mailman/listinfo.cgi/std-proposals   -- Std-Proposals mailing list Std-Proposals_at_[hidden] https://lists.isocpp.org/mailman/listinfo.cgi/std-proposals

Received on 2024-07-31 07:37:57