3

I am writing a network wrapper around boost::asio and was wondering what is a good and simple way to serialize my messages. I have a message factory which can take care of dispatching the data to the correct builder, but I want to know if there are any established solutions for getting the binary data on the sender side and consequently passing the data for deserialization on the receiver end.

Some options I've explored are: passing a pointer to a char[] to the serialize/deserialize functions (for serialize to write to, and deserialize to read from), but it's difficult to enforce buffer size this way; building on that, I decided to have the serialize function return a boost::asio::mutable_buffer, however ownership of the memory gets blurred between multiple classes, as the network wrapper needs to clean up the memory allocated by the message builder.

I have also seen solutions involving streambuf's and stringstream's, but manipulating binary data in terms of its string representation is something I want to avoid. Is there some sort of binary stream I can use instead? What I am looking for is a solution (preferrably using boost libs) that lets the message builder dictate the amount of memory allocated during serialization and what that would look like in terms of passing the data around between the wrapper and message factory/message builders.

PS. Messages contain almost exclusively built-in types and PODs and form a shallow but wide hierarchy for the sake of going through a factory.

Note: a link to examples of using boost::serialization for something like this would be appreciated as I'm having difficulties figuring out the relation between it and buffers.

ChrisF
  • 38,878
  • 11
  • 125
  • 168
mtsvetkov
  • 151
  • 4
  • 1
    if you are passing pointer you should also pass the length of the remaining memory available to be used (otherwise it's a bufferoverflow waiting to happen) – ratchet freak Sep 24 '12 at 11:58
  • I am doing that, but was looking for a way to invert things - instead of giving the serializer memory to work with, have it give me the memory it allocated, since it knows how much it needs. Also, if I just use one buffer it'll need protecting for threads and eventually become contested. If the wrapper allocates memory to give to the message builder, it's the same as the builder doing it itself. What's the best way to have the builder manage its memory is my question – mtsvetkov Sep 24 '12 at 12:02
  • @mtscetkov if you do that you'll get a lot of allocations, most C++ implementations will pass in the buffer and return it (I'm thinking of the cstreams operator<< as bad as the choice of operator is, not allocating the strings to copy into the output buffer is a good idea) – ratchet freak Sep 24 '12 at 12:48
  • `std::string` is an arbitrary array of bytes and can contain NUL bytes and even more so `std::stringstream` is simply a memory-based stream. Using it for binary data is normal and expected. – Jan Hudec Sep 25 '12 at 12:55

1 Answers1

2

Since you can't have your cake and eat it (see comment thread for the question), I've decided on the following solution:

The network wrapper will keep a boost::asio::streambuf, while serialization/deserialization will happen over ostream/istream. This way the message builders don't have to deal with memory management, can use as much/as little as they need and can choose whether they want to serialize as binary or text. As an added benefit, I can also easily serialize messages to/deserialize from a file for record/replay during debugging.

mtsvetkov
  • 151
  • 4