How can I troubleshoot stuck/flipped bits in a serial connection?

Question

I'm working on reverse engineering some scrap vaccuum fluorescent displays from Noritake that lived their previous lives inside a payphone. No datasheet exists for it since it's a custom part; Noritake themselves couldn't help out. Some sibling parts are available but they all use different interfaces.

I figured out that the VFD boards use RS-232 thanks to a DS14C232CM level shifter right next to the main connector. Some prodding with a continuity tester later and I came up with this schematic and testing setup:

The second pair of lines looks to be used for flow control.

An extremely fortunate find was a dip switch on the reverse that triggers the module into a debug mode where it lists baud rate, parity, packet size (based on the other dip switch settings) as well as the contents of some registers; a printout of every character in ROM; and a demo of markup support (e.g. scrolling/flashing); it even has an RTC. This tells me the boards are functional and how to talk to it.

The interface is a standard RS232 terminal with some display control done via escape codes. I tried using a USB-RS232 adapter to send some characters to it using RealTerm, but here's my issue: some characters display reliably, while others show consistent flipped bit errors. Here's an example, I'm sending lots of '2's to the screen:

'2' is ASCII 0x32 or 0b00110010. ':' is 0x3A or 0b0011*1*010. Bit 3 is flipped. Similarly when I send 1, I almost exclusively get 9; same bit 3 error. Printing : or 9 displays reliably. This is what printing the alphabet looks like:

I swapped in a second screen and it too has similar problem. Printing f results in an occasional 'v' or 'd'. Other characters have other issues.

How do I troubleshoot this? Is this indicative of some sort of timing error or failed hardware? There is no obvious damage to the boards that would make me suspect the latter.

I've verified the USB-RS232 adapter is functional with a Tektronix Logic Analyzer.

Answer 1

Suggest checking whether the problem recurs at differing bps ("baud") rates:

• If the problem goes away at low bps rates then it seems an "analog" problem such as grounding continuity issue (maybe the chip has multiple ground pins and one's open?) or ringing ('scope it out, though it seems unlikely at RS-232 rates ) or charge accumulation leaked from the high voltage needed for driving the VFD

• If the problem occurs regardless of the bps rate, then perhaps there's a logic, framing (wants 1-and-half or 2 stop bits, computer set for one?) or clocking issue issue.

Other things to look for:

• Where does the clock source for the UART come from? Is there an external clock or crystal? Perhaps the board was designed with an intentionally off-frequency clocking which happened to be convenient for the phone's CPU's clock divider, allowing a very cheap, but non-standard serial bps rate via "software UART"

• Presumably the board has a crystal for its clock. Try padding it with a few pF in parallel to see if such varying of the chip's clock rate straightens up the framing.

• When the rational fails, try the moderately irrational:

  o Another computer? Another USB cable?

  o If you're ready to give up, presuming the chip has an external crystal, driving one or the other crystal pins through a high series impedance (to avoid chip damage) such as perhaps a 100K ohm resistor from a variable frequency sine wave generator at various frequencies slightly above and below the crystal's frequency. The idea here is to override the crystal's frequency and take control of the chip's clock, varying to see if this straightens up the framing and bit decoding.

• Might the circuit supply a clock-out somewhere for serial rate clocking?

Answer 2

I have found a similar problem where the 6th bit is sometimes flipped, e.g.:
0x50 -> 0x70, or 0b01010000 -> 0b01110000

This did not happen consistently, only for values greater than 0x40. I noticed that this was the values which represented the upper case ASCII characters. As such:
0x50 = 'P', and 0x70 = 'p'

In my case I was using C on Linux and set the input flags for the serial port incorrectly, e.g.
ty.c_iflag = ~(IGNBRK | BRKINT);
instead of
ty.c_iflag &= ~(IGNBRK | BRKINT);

(If you don't the error, it set all bits, except the specified bits, instead of clearing the bits)

This error inadvertently also set the IUCLC input flag, which does exactly what I saw; it converts uppercase ASCII to lowercase ASCII. This is a legacy TTY (terminal) feature from *NIX systems.