How can we extend a packet in the case of an overflow?

Question

I have asked this question in stackOverflow, but I guess that maybe softwareEngineering is more appropriate. (here is the question in stackOverflow)

I have frames sized with a fixed size of 7 values (t_i, i in {1..7}). Each value is stored on 2 bytes (the max value is then = 2^(16) -1 = 65 535) and is divided by a resolution factor of 10000, which makes 6.5535ms as the maximum value (0xFFFF). Beyond those values, the 2 bytes are no longer enough and the counter overflows.

In my test cases, I often encounter overflow especially when I use quite low frequencies

eg: frequency = 400Hz and t_1 = 1ms => t_7 = 1 + 6 * 1/400 = 16 ms > 6.5535 ms

To handle this overflow, what has been done until now was that that we allowed the overflow as it was detectable during the decoding.

To detect this overflow, the time delta between 2 consecutive (t_i) has been calculated to see if the result was lower than the max value (0xFFFF equivalent to 6.5535 ms) in which case, there is an overflow (counter and growing by definition, a decay shows an anomaly ).

I am thinking about extending the frame (x2) in case of overflow. How can I proceed to do so?

Existing lines to bypass the counter's overflow problem :

    if (timeCounterInMs > 2000000000) { // 2000000000 is the value to be substracted to  the counter to transform long into double
                timeCounterInMs -= 2000000000;
                isCounterOverFlow = true;
    } else {
                isCounterOverFlow = false;
            }

     currentIndexInBytes += 4;
     body[index++] = timeCounterInMs; // body of the frame
            
     for (int j = 0; j < 7; j++) { 
                long timeInTimeRatio = /*transform bytes to long*/;

                currentIndexInBytes += 2;
                body[index++] = timeInTimeRatio; 

                if (timeInTimeRatio == 0 && j != 0) {
                    // the end of the packet
                } else {
                    
                    if (timeInTimeRatio <= theLastValueInPacket && j != 0) { // If the time counter declay => overflow and start to zero

                        timeInTimeRatio = timeInTimeRatio + 65536; // 0xFFFF = 65536
                    }
}

Thank you so much

Answer 1

Variable-length encodings are much harder to deal with than fixed length encodings, because you need to decode all preceding elements before you can even tell where t_5 starts.

Because of this, if those low frequencies that now cause trouble in your tests are representative of a real-world situation, I would advise you to 32 bits (4 bytes). This allows you to store a time of 429496.7296 ms or a frequency in the order of 1e-5 Hz.

If 4 byte elements are too much for your throughput, you should consider going to 24-bits (3 bytes) before going to variable length. 24 bits still gives you a range of 1677.7216 ms and is only slightly more complicated to convert to/from a long (and actually the same if your network protocol uses a different byte order than Java does).