> On 21 Feb 2025, at 12:38, Jonathan Wakely <cxx_at_[hidden]> wrote:
>
>> On Fri, 21 Feb 2025 at 11:25, Hans Åberg <haberg_1_at_[hidden]> wrote:
>>
>> > On 21 Feb 2025, at 11:34, Jonathan Wakely <cxx_at_[hidden]> wrote:
>> >
>> >> On Fri, 21 Feb 2025 at 10:09, Hans Åberg <haberg_1_at_[hidden]> wrote:
>> >>
>> >> > On 21 Feb 2025, at 10:53, Jonathan Wakely <cxx_at_[hidden]> wrote:
>> >> >
>> >> > On Fri, 21 Feb 2025 at 09:08, Hans Åberg <haberg_1_at_[hidden]> wrote:
>> >> >
>> >> > > On 21 Feb 2025, at 00:51, Jonathan Wakely via Std-Proposals <std-proposals_at_[hidden]> wrote:
>> >> > >
>> >> > > On Thu, 20 Feb 2025, 21:39 Phil Bouchard, <boost_at_[hidden]> wrote:
>> >> > >
>> >> > > On 2/20/25 16:19, Jonathan Wakely wrote:
>> >> > > >
>> >> > > >
>> >> > > > On Thu, 20 Feb 2025 at 20:41, Phil Bouchard via Std-Proposals
>> >> > > > <std-proposals_at_[hidden] <mailto:std-proposals_at_[hidden]>>
>> >> > > > wrote:
>> >> > > >
>> >> > > > regex_match would get 1 more character on the need basis using in.get()
>> >> > > > quite simply. If it fails then it would rewind the read pointer to
>> >> > > > where
>> >> > > > it was.
>> >> > > >
>> >> > > >
>> >> > > > How? iostream putback is extremely limited.
>> >> > >
>> >> > > Using seekg().
>> >> > >
>> >> > > That might work on an ifstream or istringstream but not on an arbitrary istream.
>> >> >
>> >> > It is as necessary to have a buffer of an arbitrarily large size for regexes, as the underlying theory for regular expressions just tells whether a string is in the language or not, and cannot tell when to stop. Examples are expressions like a|a*b, where on a string a… followed by something else than b, all but the first ‘a’ must be put back into the buffer. (Or some similar idea.)
>> >> >
>> >> > So when starting with parsers, the one character put back rule is no longer useful:
>> >> >
>> >> > One other example is when reading UTF-32 characters from a UTF-8 stream: Then one in general cannot put back the UTF-32 character, as it will in general occupy mora than one byte.
>> >> >
>> >> >
>> >> > Which is why you don't want to build it on top of a single-pass range, like an istream.
>> >>
>> >> One can have a single-pass buffered input stream, only that the buffer is larger than one character, which is what Flex does. Or like the other C++ formatted input already present, for different types, int's, floats, and strings, where you can't put back characters.
>> >
>> > But if you require a specific kind of buffered input stream (or even just a specific kind of streambuf with an arbitrary size putback area) then it's not a generic std::istream and so you don't want a operator>> overload that works with arbitrary std::istream objects.
>>
>> The std::istream objects might be extended to have arbitrary size buffers with only the first character in the synchronized C stream, and if putting back more than one character, the synchronization is broken. It will simplify the implementation of regexes, and also formatted input can be put back.
>>
>> They are indeed in effect a new type of istream objects on top of those are now.
>>
> So ... not istreams.

As they are now, but it might be useful to add aribtrary size buffers:

When parsing included files, then the normal is that each file has its own buffer, in a stack. If the buffers are in the istream objects, that would not be needed.

> Surely what we want is a regex that can match an input_range, which could then be used with your new "buffered input stream" thing, and other ranges defined by input iterators. Rewinding/resetting the range on failed matches should be a separate behaviour that is optionally supported by whatever range you're using, not related to the regex matching.

Then it seems that the regex buffers should be in the std::isteam objects, as one then can try a sequence of regexes separately, not having to worry about failed matches.