C++ Logo

std-proposals

Advanced search

Re: Modern std::byte stream IO for C++

From: Lyberta <lyberta_at_[hidden]>
Date: Fri, 6 Mar 2020 16:15:38 +0300
Maciej Cencora via Std-Proposals:
> That leads to duplication, since you need to write custom read/write for
> every type

No. That's what templates are for.

> Your solution:
> struct MyStruct { int a; int b;}
> void writeMidiFile(const MyStruct& obj, output_stream& os)
> {
> std::io::write(MidiInt(obj.a), os);
> std::io::write(MidiInt(obj.b), os);
> }

No. My actual code is like this:

class NoteOnEvent final : public /* ... */
{
public:
 NoteOnEvent(const StatusByte& status_byte, std::io::input_stream auto& stream);

 void Write(std::io::output_stream auto& stream) const;
private:
 Music::Pitch m_note; ///< Note of an event.
 DataByte m_velocity; ///< Velocity of the note.
};

NoteOnEvent::NoteOnEvent(const StatusByte& status_byte,
 IO::InputStream auto& stream)
 : BasicChannelEvent{status_byte.GetChannel()}
{
 std::io::read(m_note, stream);
 std::io::read(m_velocity, stream);
}

void NoteOnEvent::Write(std::io::output_stream auto& stream) const
{
 std::uint8_t byte = NoteOnEventCode + this->GetChannel().GetValue();
 std::io::write_raw(byte, stream);
 std::io::write(m_note, stream);
 std::io::write(m_velocity, stream);
}

Notice that I never expose fundamental types publicly. Music::Pitch is an enum class with custom deserialization code, DataByte
is a class with its own invariant and custom [de]serialization code. If you look closely, there is also a Channel class with,
you guessed it, its own invariant.

The moment you expose raw "dumb" types, you lose all the semantics and your code starts to be hard to read and debug and
compiler no longer can help you. See, I call Channel::GetValue() only in the lowest private level of my code where I can't do
with strong type anymore. Here, the reason is that some events encode channel information differently but, again, I can put
"unsafe" code in the implementation that is private.

A struct with two ints would never pass my code review.

> and write such function for each and every user defined struct that you
> want to serialize (for each serialization format).

There's rarely more than 1 format. MIDI 2.0 will require very different types. For WebAssembly I solved all the integers with 1
template:

namespace LEB128
{

template <std::integral T>
class Integer final
{
public:
 constexpr Integer(T value = {}) noexcept;
 Integer(std::io::input_stream auto& stream);
 void Read(std::io::input_stream auto& stream);
 void Write(std::io::output_stream auto& stream) const;
 constexpr operator T() const noexcept;
private:
 T m_value; ///< Raw value.
};

}

Then I just instantiate with any integral type I want:

namespace WebAssembly
{

using Int32 = LEB128::Integer<std::int32_t>;
using Int64 = LEB128::Integer<std::int64_t>;
using UInt32 = LEB128::Integer<std::uint32_t>;

}

And then I always use those types instead of fundamental ones. Yes, I cheated a bit with implicit conversion, it would go in the
future. Also, when I encounter other format that uses LEB128, I would modify LEB128::Integer to accept the "tag struct" so the
types would be different and there would be no implicit conversions between them. This is exactly how std::chrono::time_point
works. Time points with different clocks are different, incompatible types even if they have the same "dumb" type under the hood.

> Instead parametrize by codec:
> void write(const MyStruct& obj, output_stream& os, Codec auto& cdc)
> {
> std::io::write(cdc.encode(obj.a), os);
> std::io::write(cdc.encode(obj.b), os);
> }
> And you have only one 'write' function per user defined type, for all
> serialization formats.

I would like to see a real case when there is more than 1 different binary format for the same semantic types. But even then,
there is already a solution in the paper:

void write(const MyStruct& object, output_stream auto& stream, CustomInfo info)
{
 /* Serialize differently based on what is in CustomInfo */
}

Then you just call it like this:

std::io::write(my_object, some_stream, my_custom_info);


Received on 2020-03-06 07:18:32