Hi Ivan,
Not sure that it would be possible to make it to C++26. You can try to pre-allocate, but the question is how much?
For that it would require a 2-pass solution, the first pass figures out the required size, the allocation would happen in the middle, the second pass would fulfill the conversion keeping the promise of the first pass.
But there's currently no formatting function in the standard that can give you the size, you would need to write that, for everything. Then you would have to make the formatter compatible without breaking ABI.
That is challenging change and not a simple one to push through in time.
I believe that progress in this area would be worth it. I am however skeptical of the time frame.


-----Original Message-----
From: SG16 <sg16-bounces_at_[hidden].org> On Behalf Of Ivan Solovev via SG16
Sent: Monday, July 1, 2024 10:38
To: sg16_at_[hidden]
Cc: Ivan Solovev <ivan.solovev_at_qt.io>
Subject: Re: [isocpp-sg16] std::format and charN_t

Hi Thiago,

> I spent some time looking into the libstdc++ implementation for the
> feasibility of the above and I think the answer is the same for all of them:
> Qt provides a qFormat() function, which wraps std::format_to.
>
> The trick is that we provide our own output iterator. When
> std::formatter for our own types detects this output iterator type
> from the context, we can implement the space reservation and writing
> into the buffer properly. That can also therefore implement the
> conversion from UTF-8 for std::string and std::string_view, and a direct memcpy for std::u16string.

I do agree that this makes sense for formatting into QString, as the standard does not provide any char16_t formatters anyway. That's something that we can further discuss in our Qt Development ML.
However, I'm not sure that we need such approach for anything else. Users should be able to use std::format with Qt types, not some fancy wrapper function.

So, in my opinion the fact that std::formatter is missing the APIs to pre-allocate a buffer, and to write directly into it is still a problem that should be fixed for C++26. Not only Qt, but everyone writing formatters for custom types would benefit from it.

Best regards,
Ivan

------------------------------

Ivan Solovev
Senior Software Engineer

The Qt Company GmbH
Erich-Thilo-Str. 10
12489 Berlin, Germany
ivan.solovev_at_qt.io
www.qt.io

Geschäftsführer: Mika Pälsi,
Juha Varelius, Jouni Lintunen
Sitz der Gesellschaft: Berlin,
Registergericht: Amtsgericht
Charlottenburg, HRB 144331 B

________________________________________
From: SG16 <sg16-bounces_at_[hidden]> on behalf of Thiago Macieira via SG16 <sg16_at_lists.isocpp.org>
Sent: Saturday, June 29, 2024 4:29 AM
To: sg16_at_[hidden]rg
Cc: Thiago Macieira
Subject: Re: [isocpp-sg16] std::format and charN_t

On Friday 28 June 2024 15:53:21 GMT-7 Thiago Macieira via SG16 wrote:
> The trick is that we provide our own output iterator. When
> std::formatter for our own types detects this output iterator type
> from the context, we can implement the space reservation and writing into the buffer properly.
> That can also therefore implement the conversion from UTF-8 for
> std::string and std::string_view, and a direct memcpy for
> std::u16string. We don't need to implement std::u32string and I quite
> frankly don't care about wchar_t -- I think it was a mistake for
> std::format to have that instead of char16_t and char32_t.

Actually, the for the majority of those types (if not all of them), we'll probably just have an entire hand-rolled implementation anyway, compiled out- of-line and built into QtCore. With that, we can probably take extra liberties with buffers that wouldn't be allowed in inline code being emitted into user code, with generic formatters.

After all, the vast majority of strings being formatted are not known at compile time. They come from translation files installed by the application.

That means we won't simply wrap std::format_to like std::format does. We'll need to provide a function that extracts arguments and separates the ones for which we have a built-in formatter, and thus only need a reference capture, and those for which we don't and thus must capture a type-erased formatter.

I'm hoping we can keep the format parser.


--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
  Principal Engineer - Intel DCAI Platform & System Engineering



--
SG16 mailing list
SG16_at_[hidden]
https://lists.isocpp.org/mailman/listinfo.cgi/sg16
--
SG16 mailing list
SG16_at_[hidden]
https://lists.isocpp.org/mailman/listinfo.cgi/sg16