What can I do to decrease the latency from these serial ports which are attached to a PC via a Serial to USB adapter?

Question

I think I have accidentally discovered a need in my life for embedded systems. Which is great! And kind of scary. And I need help.

Background: I got hired to build a GUI application which takes scans from two SICK LMS-291s and integrates them with a sub-inch accuracy GPS, so you know where each scan occurred. As the naive web programmer I am, I understood that timing would be important, but didn't realize it would also be hard! If you don't know when each GPS point and each scan occurred, you can't figure out where the scans occur. Oops.

They had specified windows 7 as the platform, as well as bought a SeaLevel RS422 to USB box to hook up the sensors and GPS to, and in short order I discovered my folly. Somewhere between the sensors and my computer program, something was keeping the scans from arriving in a timely manner. The LMS spits out 75 scans per second, or at 13.32 ms/scan. My program doesn't get them in a timely manner. It gets them every 100 or so milliseconds, in groups of 7 or 8 or 10 or something. Also sometimes not enough scans show up, or they're mangled. Either this SeaPort adapter is only sending ten times a second (is that possible? I don't know how USB works) or Windows isn't checking the buffer (there must be a buffer somewhere, right?) nearly often enough.

Present Day: This leads to some inaccuracies that the client is basically okay with. I'm not, though, and since I've got a chance to do similar work for the client (integrating more sensors inputs!), I'd like to figure out how to do it right, e.g. given the accuracy of the GPS, be able to give guarantees about the precision and accuracy of the scan locations.

What does that look like? I need a UI, and to be able to check input from these three devices every 13.32 milliseconds. If I used FreeRTOS with, say, Nano-X for the GUI, run on a laptop they provide, would that sound like a sane solution? Is it possible that the RS-422 to USB adapter is causing these delays, and using Windows is actually just fine for this purpose?

Answer 1

The problem is almost certainly in the USB buffering related to the USB-RS422 converter. USB has variable and fairly high latency.

The easiest solution would just be a better RS422 interface, ideally something PCI/PCI-e based. That would solve the latency issues.

You may also be able to modify the USB polling rate, though this is fairly dependent on the host OS (what platform are you on?).

---

For what it's worth, I looked about on sealevel system's website, and it is MASSIVELY infected with marketing bullshit. They literally spend like 10 pages and multiple white-papers on saying "we use a USB hub internally, rather then doing MCU multiplexing".

Hey SeaLevel! That's what the cheap-ass $50 4-port USB-serial interface I bought on e-bay from china does too! You're not special, even if you write half a dozen vapid whitepapers trying to make it sound like you are!

Have you tried to force those things to use FTDI drivers? I'd put money on it they're just using bog-standard FTDI FT232 or similar. Can you pop the box on one of them open, and take pictures?

---

If you really want to spin your own hardware, for fun or educational opportunities, I'd strongly suggest you not try to do everything in the hardware. Since you need to simply time-correlate all three signals, all you really need is something that can listen on three serial lines (two RS422, one RS232 (the GPS)), time-stamps the data, and forwards it to the main computer.

Once the data is time-stamped, you're free to have all the buffer-latency you want, since you can always just look at the time-stamps.

Realistically, if you have no base in hardware, designing something with enough crunch to draw a nice GUI is quite the undertaking.

Personally, I'd probably throw a fairly chunky ARM MCU at the buffering issue, and be done with it. Despite the fact that it's an Arduino, the Arduino Due has plenty of SRAM, and is more then fast enough for what you need (and there is lots of support, which is always nice).
Alternatively, the STM32 series has similar performance, and is more intended for "advanced" users (read, there are fewer, or no examples to reference). ST makes lots of quite nice, extremely inexpensive eval boards as well.

With the Due, you do get a native USB port, for which you could roll your own CDC driver if you wanted. Some of the STM32 boards have native USB as well.

Answer 2

If I understand the question, what this comes down to is taking time/place messages from GPS over one serial line, and correlating range data messages received over other serial lines. Ideally, the range finder messages would have their own time stamp, and if you could synchronize all the clocks, you'd be able to pin down the location where the range was taken by interpolating the range time stamp between the two gps messages with the closest matching time stamps. But of course, you don't have that.

A couple of solutions come to mind, all along the lines of using a microcontroller to do the real-time data correlation, and pumping the output of that into the PPC for display purposes. Basically the microcontroller is there to deal with the tight timing issues.

So for a simpler solution, all you'd need is some way to collect all the messages from the several different serial lines, and combine them into a single stream, in the order they were received in, and pump that into the PC. You can infer that any range finder message between two GPS messages occurred sometime between those two messages.

This gives you a degree of uncertainty of course, dependent on the frequency of the GPS messages. The trade-off is that as you increase the GPS message frequency (to get more accurate correlation), you increase the demands on the microcontroller, and the demands on the serial link into the PC. At an extreme, the serial link is saturated with GPS messages with an occasional range message thrown in. Obviously most of those GPS messages are not needed. The advantage of this solution though, is that the micro has very little to do, from a software standpoint. You could do it all in a simple loop of assembly language, no OS required.

For a more complex solution, you can form a local clock in the micro, and use the GPS to synchronize that clock, so that when you get a range message, you can obtain a time stamp using the micro's clock. Using a micro with a crystal timebase, you could probably get by with 1Hz GPS messages and still get much better than millisecond accuracy times stamped on the range messages. A competent embedded systems person could probably also pull this one off in assembler on a lower end micro, but you mentioned you're just starting out. You could look at a more powerful micro that can run linux, and probably find a pre-existing solution for synchronizing the linux clock to the GPS.

Answer 3

What I would probably do is multiplex the serial data into a new serial datastream at higer rate, with a static interleave. Then feed the datastream through USB-UART to the computer. That way you would know that the first byte is from device 1, the second byte from device 2, the third byte from device 3 and in this case a fourth byte as CRC or sequence number. Throw in a couple bytes start of frame marker to sync on the datastream, padding bytes if no data available and you're done. You may not know the absolute time, but you do know that the relative time between the various data chunks.

Answer 4

I'd also go with a microcontroller to receive the RS data, timestamp it, and forward it to the PC. You can't do any I/O timing critical work with a Windows PC (except with a sound card, perhaps), because the software and the drivers are changing all the time as drivers are updated behind your back.

But the first thing I would do is to get a USB logic analyzer, you can buy one for less than USD100, grab some messages from the GPS receiver and check out exactly when each byte is received. Use that info to try out / calculate exactly how accurate the timing of the data is in the first place: e.g. exactly what and when the GPS box transmits. Then you can you can work out what accuracy is achievable, and how much effort you would be willing to have in order to get to a certain level of accuracu.

Above, a picture of some (Saleae?) USB logic analyzer.

---

[Edit] Haha, using the sound card could be a funny way to make it actually work; you could connect the UART data to the sound card input (via some resistors and capacitors) and write software to detect each edge in the serial stream, giving you beautiful timing accuracy!. Ok, I'm not really suggesting this seriously, just for some laughs!