Hello,
I am a bit concerned with the direction std::embed is going and I'll like
to see if we can agree on a few things.

A preprocessor based approach does not offer sufficient benefits

I understand that JeanHeyd has been given enough contradictory guidances
that he might be tempted to go with the preprocessor #embed solution.
I am concerned that this would not solve anything.

The entire value of std::embed is to improve compile time. Anything that
would create a node per byte would have disastrous
compile time performance. My observation is that parsing is not the
bottleneck.
Sure that might be solvable by an attribute which would change how arrays
are represented by the compiler.
Even then, the compiler would have to store the entire file's content in
memory whereas std::embed can be designed to be backed by a memory mapped
file.
And so a preprocessor based approach would have little value over generated
source files and suffer many of the same issue.
That it can be pushed through the committee faster should not be a reason
to pursue that direction.

Security concerns
It was always my understanding that embedable resources would be found by a
mechanism similar to include paths
and as such giving full filesystem access was never really on the table ?
Is such a flag insufficient? It would require inspecting what the build
system does, rather than individual file.
I would be sympathetic to a per-file mechanism to identify resources that
can be open but would like decoupled from std::embed

ie:
#pragma resource "foo.txt"
[...]
std::embed("foo.txt")
We might also want to make all paths relatives and implementations can
support a white/blacklist of resource path.
I think it's important to support both file and directories - but
supporting only files as a first approach seems reasonable.

Tooling
A mechanism decoupled of std::embed as in the paragraph above would support
the needs of tooling.
I don't think tooling support should stop std::embed though.
Having to specify dependencies on resources manually is reasonable given
that resources should be few

Modules
To properly supports std::embed and modules, BMIs should keep track of
resources path when they are compiled relative to the BMIs, or the source
files
The idea is that an importing module should have access to the same
resources as the imported modules.
Alternatively it can be the responsibility of the build system to deal with
that.

More General Solution ?
It is unclear to me that std::embed is not already the general solution.
It returns a span, which is mean it can be manipulated with any algorithm
or view adapter offered by the standard library, most of which are
constexpr already.
This is strictly more versatile than file which we have fewer tools to
handle.
In fact i suspect a primary use case for memory mapped file will be to wrap
them in span and use them with algorithm.

Moreover, the concerns that std::embed has security concerns but is not
general enough seem antithetic.
I don't think we want write capabilities, or i/o on fd that are not file on
disks - more for reproducibility than security reasons.
I am also not convinced that the use cases for runtime i/o and compile
time/io are the same.

Neverther less, i see several solutions:

   - Making file and mapped_file as proposed by P1031[1] constexpr
   (partially, we need a small subset), which I believe Niall has been working
   towards
   - Continue with std::embed which does not preclude a constexpr
   stream-like interface on top of a span in the future


Trying to blend compile-time and runtime io, seems to me to be only
seductive on the surface, but in reality the use cases will be different,
and I believe std::embed serves all the use cases people actually care
about in practice.

Anyway, I wanted to start a discussion as the problem that std::embed
strive to solve has currently no solution at all.

Corentin



[1] P1031R2: [SG1, SG14, LEWGI, Library Evolution] Low level file i/o
library <https://wg21.link/p1031r2>