C++ Logo

SG16

Advanced search

Subject: Re: [SG16-Unicode] Replacement for codecvt
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2019-08-29 14:11:03


> That being said, it is going to be by no way shippable with something
> like LLFIO in the next month or so. Apologies.

Ok. Thanks for being honest.

> Once again, being blunt here, I can't see why text reencoding needs
> anything more than span<T> and basic_string_view<T>. Ranges
> auto-understands both of those. Again, if the design really needs
> Ranges, it has to be the wrong design.
>
>
>      Iterators (and their child, Ranges) have been the backbone of
> libraries communicating with one another since Stepanov got it right.
> Hardcoding span/string_view as the communication medium is the
> fundamentally wrong choice, because it fundamentally excludes a wide
> range of use cases, including Boost.Text's unencoded_rope, libstdc++'s
> and SGI's __gnu_cxx::rope<T> storage, gap_buffer implementations,
> ring_buffer implementations on the network, and more.

The reason why I really like span<T> for this is because reencoding
benefits greatly from pipelining tricks only possible when input is
guaranteed contiguous.

I also agree that support for discontiguous input (and output) is also
very important.

But I'd tend to approach this as span-of-spans i.e. scatter-gather
buffers with resumability. Or, precisely what LLFIO's read()/write()
already implements.

That can be simplified down into a loop over a list of spans if you have
resumability. So that's where I'd start my API design, personally. Hard
coded function overloads all working with span<CharT>, that way we can
hide implementation details behind an ABI boundary, and keep build fast.
Just like the rest of LLFIO.

For sure, if someone wants to build on top of that API a fancy
Ranges-based convenience API, I'm all for that. LLFIO was designed to be
the backend for Ranges I/O and Coroutine I/O after all. But it's not
something I would be able to personally use any time in the next decade,
so my interest level is correspondingly low.

Unfortunately I am now at the end of my dev time, I must start paper
writing next week. So I'll kick this stuff until after Belfast when I
next get a dev window. Thanks for your feedback.

Niall


SG16 list run by sg16-owner@lists.isocpp.org