Date: Sat, 12 May 2018 12:32:26 +0200
Indeed, I’ve thought of that after I sent the mail. Your embed is analogous to std::fread(). I even tried to find the place where MSVCs STL invokes std::fread() under the hood, but I could not find the place. There is a std::basic_streambuf::xsgetn(){ …; _Traits::copy(); } function over the input tellg() pointer in std::streambuf it won’t let me step into that should be doing the magic. I’m not sure if std::embed() fundamentally differs from std::fread(). If it does (and should), by all means, we do need it.
I deal with a bunch of GPU APIs, graphics and compute alike, and loading shaders, kernels in source or binary is a daily task. Some APIs take of this under the hood (C++AMP), some allow implementations to outsource it to the build system (ComputeCpp, a SYCL implementation in their CMake and MSBuild integration), others provide mechanisms to do it both run-time and compile-time (OpenCL clCreateProgramWithSource vs. clCreateProgramWithBinary)… the spectrum is wide and I do want something like this. (Although my primary motivation is configuring TMP and the ABI of my code through a proper JSON file that has a schema and can be validated. Also, having libraries like C++/WinRT and XAML be implementable inside the C++ compiler alone and have IntelliSense light up without any external tooling; although the latter required static reflection.) I only aked „Could you import your data easily in a run-time manner?” was only to prove the point that everyone is already familiar with one workflow and syntax from C++ 101, and there’s nothing in that method that inherently would outrule it from working in a constexpr context.
Don’t get me wrong, I WANT this to happen. I just want to propose in trying to get the right syntax, something that automagically lights up other parts of the language, or at least allows them to. I’de like to avoid duplicates like std::span and std::string_view. The fact that actually std::string_view is not just an alias or specialisation of std::array_view<char> is a failure in abstraction. A view of a const char* and a view of const float* share a great deal of common functionality. Strings do have some useful extra, but the commonality should reflect in the type system. We are heading towards totally different names with totally different semantics.
Let’s not do the same with std::embed, that’s all I vote for. If it’s not going to share any commonality with current file IO, I’ll just put it among the rest of things I don’t agree with and use it nonetheless.
Cheers,
Máté
Feladó: Corentin
Elküldve: 2018. május 12., szombat 10:26
Címzett: WG21 Tooling Study Group SG15
Tárgy: Re: [Tooling] [ std::embed ] Getting Data into C++ at Build Time
They are orthogonal idea.
This paper puts a chunk of data in the program data section, then you can, at run time, use iostream on it to read the data, if you want.
Embedding resources in an application is very useful and there has been a number of tool to do that ( rcc, rc, xrc, etc etc).
It's not always desirable to open the file at runtime ( think of applications shipped as a single binary, like a windows installer) - or even feasible (on embedded targets there may not be a file system at all)
Le sam. 12 mai 2018 à 09:59, Nagy-Egri Máté Ferenc <nagy-egri.mate_at_[hidden]> a écrit :
Hi!
First of all, thank you for the mental sider to my Saturday morning coffee. 😊 I do enjoy reading technical stuff to get my brain going.
However, I would consider taking a totally alternate route for a reason you also mention along the lines of std::span, namely to avoid „type spam”. My rhetorical question would be:
Could you import your data easily in a run-time manner? (Of course you could.)
Why introduce an entirely new way to process input, when all you intend is to be able to read files at compile time? We already have a versatile way to do that at run-time through iostreams, std::ifstream to be precise. I would much sooner vote std::ifstream and related types to gain constexpr support for many reasons:
1. Easier to teach. Everyone starts with std::cout << „Hello, World!” << std::endl; and related operations. There’s really no reason why custom type IO operators (std::istream& operator>>(std::istream& is, MyType& rhs) {…}) should not translate into this brave new world. The only thing needed is std::ifstream to work in a constexpr context.
2. You mention the troubles of how to specify file-system paths portably, which you say could be specified via embed_options. Don’t we have a way to deal with those? Oh yeah, std::filesystem, and it just happens to be that std::ifstream has matching CTORs.
AFAIK constexpr std::allocator is already in the works, which is needed for std::basic_streambuf to work in a constexpr context. I do understand that iostreams and formatting my possibly throw, which renders it unable to be used in constexpr contexts. However, avoiding „type spam” and multiple ways to get the same job done is enough incentive to consider making exceptions cx-friendly. (When I first heard that exceptions cannot be used in cx contexts, I was puzzled… Why not emit a #error or any compiler error on unhandled constexpr exceptions? It really is an error in the compilation process.) Alternatively, we could introduce a std::basic_cxios that does not have an exceptions() member and can only report errors through error_codes.
I do agree that this feature would greatly simplify tooling around the language, but I think the proposals current direction is harmful. I think it’s bloating the language (STL) in ways that are not necessary. Everyone has muscle memory in how to deal with iostreams (may that be good or bad) and we should leverage as much of that as possible for simplifying things in our heads. Having constexpr std::ofstream is a totally different pothole that can wreck havoc (translation-unit crosstalk via constexpr file output Will create dependencies of source files possibly on an intra build target level), so I’m not advocating for that, even though it might have value and C++ programmers are free to shoot themselves in the foot if they see fit. Even now there are gazillions of ways to do that. If someone believes there’s enough value down that road, they should propose.
What are your thoughts? I don’t think I’m „bikeshedding” here, so I’m curious to what you think. Is this madness?
Cheers,
Máté
Feladó: ThePhD
Elküldve: 2018. május 12., szombat 1:47
Címzett: tooling_at_[hidden]
Tárgy: [Tooling] [ std::embed ] Getting Data into C++ at Build Time
Dear SG15,
Titus Winters at C++Now invited people to join the SG15 mailing list (which I didn't actually know existed!) and share their ideas. I wrote a paper about getting resource data from files into C++ in a cross-platform, compile-time way, under the name "std::embed" -- http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p1040r0.html.
I expect the paper to be changed quite a bit as I go to my first C++ Meeting in Rapperswil, but before I do I would like to submit the paper here. It seems like something that would be of benefit to this Study Group, as it can replace many build steps currently used today to handle the same sort of issue.
If anyone could give me feedback about the paper, what it might meant to them, or if it even is something that truly concerns Tooling, I would greatly appreciate it!
_______________________________________________
Tooling mailing list
Tooling_at_[hidden]
http://www.open-std.org/mailman/listinfo/tooling
I deal with a bunch of GPU APIs, graphics and compute alike, and loading shaders, kernels in source or binary is a daily task. Some APIs take of this under the hood (C++AMP), some allow implementations to outsource it to the build system (ComputeCpp, a SYCL implementation in their CMake and MSBuild integration), others provide mechanisms to do it both run-time and compile-time (OpenCL clCreateProgramWithSource vs. clCreateProgramWithBinary)… the spectrum is wide and I do want something like this. (Although my primary motivation is configuring TMP and the ABI of my code through a proper JSON file that has a schema and can be validated. Also, having libraries like C++/WinRT and XAML be implementable inside the C++ compiler alone and have IntelliSense light up without any external tooling; although the latter required static reflection.) I only aked „Could you import your data easily in a run-time manner?” was only to prove the point that everyone is already familiar with one workflow and syntax from C++ 101, and there’s nothing in that method that inherently would outrule it from working in a constexpr context.
Don’t get me wrong, I WANT this to happen. I just want to propose in trying to get the right syntax, something that automagically lights up other parts of the language, or at least allows them to. I’de like to avoid duplicates like std::span and std::string_view. The fact that actually std::string_view is not just an alias or specialisation of std::array_view<char> is a failure in abstraction. A view of a const char* and a view of const float* share a great deal of common functionality. Strings do have some useful extra, but the commonality should reflect in the type system. We are heading towards totally different names with totally different semantics.
Let’s not do the same with std::embed, that’s all I vote for. If it’s not going to share any commonality with current file IO, I’ll just put it among the rest of things I don’t agree with and use it nonetheless.
Cheers,
Máté
Feladó: Corentin
Elküldve: 2018. május 12., szombat 10:26
Címzett: WG21 Tooling Study Group SG15
Tárgy: Re: [Tooling] [ std::embed ] Getting Data into C++ at Build Time
They are orthogonal idea.
This paper puts a chunk of data in the program data section, then you can, at run time, use iostream on it to read the data, if you want.
Embedding resources in an application is very useful and there has been a number of tool to do that ( rcc, rc, xrc, etc etc).
It's not always desirable to open the file at runtime ( think of applications shipped as a single binary, like a windows installer) - or even feasible (on embedded targets there may not be a file system at all)
Le sam. 12 mai 2018 à 09:59, Nagy-Egri Máté Ferenc <nagy-egri.mate_at_[hidden]> a écrit :
Hi!
First of all, thank you for the mental sider to my Saturday morning coffee. 😊 I do enjoy reading technical stuff to get my brain going.
However, I would consider taking a totally alternate route for a reason you also mention along the lines of std::span, namely to avoid „type spam”. My rhetorical question would be:
Could you import your data easily in a run-time manner? (Of course you could.)
Why introduce an entirely new way to process input, when all you intend is to be able to read files at compile time? We already have a versatile way to do that at run-time through iostreams, std::ifstream to be precise. I would much sooner vote std::ifstream and related types to gain constexpr support for many reasons:
1. Easier to teach. Everyone starts with std::cout << „Hello, World!” << std::endl; and related operations. There’s really no reason why custom type IO operators (std::istream& operator>>(std::istream& is, MyType& rhs) {…}) should not translate into this brave new world. The only thing needed is std::ifstream to work in a constexpr context.
2. You mention the troubles of how to specify file-system paths portably, which you say could be specified via embed_options. Don’t we have a way to deal with those? Oh yeah, std::filesystem, and it just happens to be that std::ifstream has matching CTORs.
AFAIK constexpr std::allocator is already in the works, which is needed for std::basic_streambuf to work in a constexpr context. I do understand that iostreams and formatting my possibly throw, which renders it unable to be used in constexpr contexts. However, avoiding „type spam” and multiple ways to get the same job done is enough incentive to consider making exceptions cx-friendly. (When I first heard that exceptions cannot be used in cx contexts, I was puzzled… Why not emit a #error or any compiler error on unhandled constexpr exceptions? It really is an error in the compilation process.) Alternatively, we could introduce a std::basic_cxios that does not have an exceptions() member and can only report errors through error_codes.
I do agree that this feature would greatly simplify tooling around the language, but I think the proposals current direction is harmful. I think it’s bloating the language (STL) in ways that are not necessary. Everyone has muscle memory in how to deal with iostreams (may that be good or bad) and we should leverage as much of that as possible for simplifying things in our heads. Having constexpr std::ofstream is a totally different pothole that can wreck havoc (translation-unit crosstalk via constexpr file output Will create dependencies of source files possibly on an intra build target level), so I’m not advocating for that, even though it might have value and C++ programmers are free to shoot themselves in the foot if they see fit. Even now there are gazillions of ways to do that. If someone believes there’s enough value down that road, they should propose.
What are your thoughts? I don’t think I’m „bikeshedding” here, so I’m curious to what you think. Is this madness?
Cheers,
Máté
Feladó: ThePhD
Elküldve: 2018. május 12., szombat 1:47
Címzett: tooling_at_[hidden]
Tárgy: [Tooling] [ std::embed ] Getting Data into C++ at Build Time
Dear SG15,
Titus Winters at C++Now invited people to join the SG15 mailing list (which I didn't actually know existed!) and share their ideas. I wrote a paper about getting resource data from files into C++ in a cross-platform, compile-time way, under the name "std::embed" -- http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p1040r0.html.
I expect the paper to be changed quite a bit as I go to my first C++ Meeting in Rapperswil, but before I do I would like to submit the paper here. It seems like something that would be of benefit to this Study Group, as it can replace many build steps currently used today to handle the same sort of issue.
If anyone could give me feedback about the paper, what it might meant to them, or if it even is something that truly concerns Tooling, I would greatly appreciate it!
_______________________________________________
Tooling mailing list
Tooling_at_[hidden]
http://www.open-std.org/mailman/listinfo/tooling
Received on 2018-05-12 12:32:32