Thank you for the feedback.

On Thu, Aug 29, 2019 at 10:59 AM Niall Douglas <s_sourceforge_at_[hidden]>
wrote:

> > 1) Submit pull requests to JeanHeyd for features that you need. His
> > target is C++23 of course and I at least don't want to see him get
> > distracted with supporting earlier standards. If we need that to
> > evaluate the design, so be it, but please try and assist where
> > possible. I see this work as SG16's #1 priority for C++23.
>
> If he breaks the requisite parts out into a standalone git repo, I'll be
> happy to submit issue requests to that.
>
> In my opinion, every WG21 library proposal should come with a reference
> implementation in a standalone, highly reusable, submodularisable git
> repo. I know that I don't *quite* achieve that myself, but I think it
> particularly important to avoid bundling reference implementation in
> with lots of other unrelated stuff.
>
> People won't lock in dependencies on bundles, too ripe for breaking
> change. They like self contained, single purpose, orthogonal git repos.
>

The implementation needs a bit more work. Itprovides a bit more than what
the paper says, but needs more work before its ready to use. I plan to
spend the next 4 months working on it, if Grant Proposals and University
Class Proposals work out. Otherwise, it will be the usual free time work
improved and be hopefully production ready near midsummer 2020.

That being said, it is going to be by no way shippable with something like
LLFIO in the next month or so. Apologies.


> > 2) Write papers proposing changes to the interface that you would need
> > (e.g., deterministic exceptions). Papers will help us more quickly
> > evaluate and consider changes. Any such papers should include
> > motivation, examples, and proposed changes (preferably wording if
> > applicable as well). But you know all that of course ;)
>
> Alas my plate is overflowing. I have four new papers in progress, such
> is my overload that none of those four will make Belfast.
>

Same.


> >> 3. Anywhere where you might throw an exception, you need to use a
> >> deterministic alternative like error_code, Outcome, etc.
> >
> > I think this needs a paper.
>
> I gotta be blunt here, if text reencoding throws exceptions, it has the
> wrong design.
>

Error handling in the text API is done with your choice of error handler
(the default uses the replacement character to encode errors into the text
itself). You never throw unless you are:
A) working with a dynamic container (e.g., std::basic_string/std::vector)
B) you ask for it by injecting the throw_error_handler directly

We can make a `herbception_throw_handler` as a test drive when Herb's
implementation makes progress. The base decoding objects are
constexpr-capable as well. Unfortunately, this means that intrinsic use on
MSVC is out of the question, because they have not implemented
std::is_constant_evaluated, nor have any internal compiler tricks for it.
GCC and clang should be okay here. (In MSVC, that just means I need a
TEXT_LIB_CONSTEXPR macro, which defaults to nothingness in VC++. I will
prefer this approach heavily because WideCharToMultiByte is the easy way to
get fast performance on Windows and that's not constexpr-blessed.)


> >> 5. cmake build system suitable for reuse from a cmake superproject (i.e.
> >> targets only modern cmake, exports the targets consumed by the
> >> superproject)
> >
> > This seems like something that you or others could help out with.
>
> I can give advice.
>

The text implementation is currently buried in my "bag of literally
everything" library (ThePhD/phd) where I put incubating ideas. When they're
ready, they get moved out into standalone repositories (e.g.,
https://github.com/ThePhD/out_ptr, https://github.com/ThePhD/itsy_bitsy).
Text is going to be moved out into its own library soon, with its own CMake
file that can be add_subdirectory'd in with little to no hassle. I haven't
quite gotten INSTALL targets perfectly right on either of those 2
libraries, so I'll need to learn how to make the CMake loop easy.


> >> 6. Any public headers should not include anything "heavy" like Ranges,
> >> Variant etc as some of my user base like to include LLFIO headers in
> >> global headers. String is acceptable, as LLFIO includes <filesystem>.
> >
> > This library is fundamentally ranges based. I don't see any way to
> > avoid dependencies on ranges in the interface; that is something we
> > explicitly desire.
>
> Unfortunately Ranges-based libraries would be a showstopper for me.
>

It is for most people. I use ranges mostly to avoid redefining concept
checks and for ranges::begin/ranges::end and other ADL-based shenanigans.
Personally, lowering the requirement on the API is going to be better than
concept checking and more; I'm going to be manually implementing the ADL
extension points I need and probably including a `text/span` polyfill to
erase the fact that most implementations don't have a span. (Or letting the
user provide their own, but every minute spent trying to do polyfill
garbage is less time spent writing good Unicode abstractions.) The goal
will likely be C++17 to start, then down to C++14 later on if I can do so
without compromising the performance and algorithmic capabilities for most
encodings.


> The problem is their current hefty impact on build times. Most of my
> clients, and consumers of my libraries, therefore ban Ranges in their
> codebases.
>
> Even long term, I can see outright bans on Ranges in header files
> continuing until compilers bake Ranges into the language. So expect a
> substantial subset of C++ users refusing to use *any* Ranges based
> library, including standard libraries, for at least a decade or more to
> come.
>
> Once again, being blunt here, I can't see why text reencoding needs
> anything more than span<T> and basic_string_view<T>. Ranges
> auto-understands both of those. Again, if the design really needs
> Ranges, it has to be the wrong design.
>

     Iterators (and their child, Ranges) have been the backbone of
libraries communicating with one another since Stepanov got it right.
Hardcoding span/string_view as the communication medium is the
fundamentally wrong choice, because it fundamentally excludes a wide range
of use cases, including Boost.Text's unencoded_rope, libstdc++'s and SGI's
__gnu_cxx::rope<T> storage, gap_buffer implementations, ring_buffer
implementations on the network, and more.

    My implementation of this library will most certainly specialize when
it detects you have handed it contiguous iterators/ranges and do "the fast
thing" when it can, but by no means should pointers be the layer of
abstraction that text encoding is built on. At the least,
RandomAccessIterators are useful, but most of the unicode algorithms "base
implementation" can be done with Forward or Bidirectional Iterators. We
need to have an implementation that works for most things, and lean on our
library implementers to do QoI. Not that it will be left entirely to
Standard Library maintainers: a critical part of the paper is allowing
users to specialize encode, decode and transcode calls with their own
(potentially ADL, though I really loathe ADL) customization points to be
far more tailored to their encodings and use cases.

     This does not necessitate std::ranges or range-v3: most of what is
necessary are simple adl_(c)begin, adl_(c)end, and maybe adl_size free
functions. Checks like "is this a contiguous iterator" can just be
backported hardcodings of what is speified as concepts and requirements in
the C++20 Working Draft. It's work-around-able,

but every workaround is another day of mucking around playing "how many
features did this compiler not implement under /std:c++14 | -std=c++14".

Sincerely,
JeanHeyd