Is my VNA broken or miscalibrated, or are these weird results normal?

Question

My company got a new vector network analyzer recently and I'm using it to try to learn more about microwave circuits, especially the resonant cavities that we use in our measurement systems. I've seen some unexpected behavior in simple situations and I'd like to know whether this is a gap in my understanding or if the VNA could be broken or miscalibrated.

The VNA is a Rigol RSA5065N (DS), a combination spectrum/network analyzer with a bandwidth of 6.5 GHz. Our systems operate in the 4.0 - 4.5 GHz range. It's not a full two-port analyzer; it only measures S11 and S21. (I'm mainly interested in S11.) We have an AppliedEM 6 GHz calibration kit (DS) and a Times Microwave CLU18-SMNM-03.00F N to SMA cable (DS).

The analyzer lets me set coefficients for each standard -- C0/1/2/3 and an offset length for the open, L0/1/2/3 and an offset length for the short, a resistance for the load, and an offset length and an attenuation for the thru. The cal kit comes with the C and L coefficients but provides offset delays instead of lengths. I wasn't sure how to translate those so I adjusted them until the post-calibration measurements matched the values calculated from the C and L coefficients at the center frequency of 4.25 GHz, giving 0mm for the open and 0.54mm for the short. I'm not sure if that was the right thing to do or if I should be adding the offset delay to those values to adjust the phase.

Below, I've provided screenshots of various situations along with what I expected to see on the Smith chart. Some of them make sense and some of them don't. I'm hoping someone can explain the mismatch between the two. Most of these show frequencies between 4.0 - 4.5 GHz, but some show the full span of 100 kHz - 6.5 GHz.

S11: After system power-up but before calibration; cable left unterminated
What I expect: A 0dB trace spread around the unit circle.
What I see: A trace of -20dB/varying phase below the center frequency and 0dB/0 degrees above the center frequency. Well, it's not calibrated, so I guess the error cancellation can do weird things.

S11: After calibration, cable terminated with calibration open
What I expect: A trace along the unit circle with capacitance values matching those given by the C coefficients.
What I see: Exactly that.

S11: After calibration, cable terminated with calibration short What I expect: A trace spread along the unit circle with inductance values matching those given by the L coefficients.
What I see: Exactly that, although I lost the screenshot for it.

S11: After calibration, cable terminated with calibration load
What I expect: A trace that's basically a dot in the middle of the Smith chart with return loss >36dB.
What I see: Exactly that.

S11: After calibration, cable unterminated
What I expect: An arc very near the unit circle but with a different phase from the open standard.
What I see: A short wiggly line moving inward and capacitive starting near the short position. Return loss increases a bit with frequency, but is still less than ~3dB. Could this be radiation loss? I know it's hard to make a good open circuit at microwave frequencies.

S11 and S21: Cable terminated with a 22mm-long 12.4GHz F-F SMA adapter (Amphenol 901-9986-RFX, DS)
What I expect: Something similar to the above but with a different phase.
What I see: A consistent 5dB return loss with about a 45-degree phase shift. An S21 measurement of this same adapter showed losses between +/-0.5dB with a 45-degree phase shift across this frequency range. I suppose if there are radiation losses too this makes sense. I also tried a shorter ~1cm Rosenberger F-F adapter (32K101-K00L5, DS) and it looked very similar to the calibration open.

S11: Cable terminated with a 6in-long 6GHz SMA-F to MMCX-M adapter cable (Cinch Connectivity 415-0071-006, DS)
What I expect: Something similar to the above two opens.
What I see: The trace departs the Smith chart entirely, showing an |S11| of more than +20dB at the center frequency. I didn't think this was physically possible.

S11 full-span: Cable left unterminated
What I expect: A trace that goes around the unit circle.
What I see: A trace that starts at the open point and immediately departs the Smith chart, eventually arriving back inside near where the unterminated cable was at 4.25 GHz. |S11| shows a peak of around +7.5dB near 2.5 GHz.

I tried adding port extension delays to the above. They dramatically change the shape of the trace without affecting the magnitude at all. The delay produces a variable phase shift, so I think it makes sense, but it also makes me think my calibration offset settings aren't the problem. Here's a 100-picosecond delay, for instance:

Ultimately, what I want to do is understand the impedance characteristics of our cavity resonators. But sometimes when I look at a resonator I get something like this, and I have no idea how to interpret it:

Can anyone tell me what's going on?

Answer 1

It sounds like you have a good network analyser, top quality cables and a solid calibration kit. I think the coefficients are wrong.

In my experience, a 4 GHz impedance measurement with a 3 foot cable should be fairly easy, at least down to the +-0.5 dB level - I get this at 4 GHz with a NanoVNA and some $2 SMA cables from Amazon (just don't bend them). |S11| should never be >> 1.0 unless you remove some cables or adapters after calibration.

---

Your first graph looks very odd. -20 dB is too low for S11 through such a short cable at this frequency, it should be -3 dB. The step in the middle is terrible. I can only assume there is some old stored calibration applied for 4.0-4.2 and then it runs out, creating the step, but the Cor icon is grey. No idea.

The next two graphs are what you'd expect but they prove nothing. Remember a VNA will regurgitate whatever you typed in for the standard, while that standard is connected - that's what you asked for. To test a set of cal standards, (or their coefficients), you need another known standard.

The open cable graph makes me think you mixed up open and short, or used some very large delay errors. Open is on the right and removing a few mm of adapter shouldn't make it a short.

S11+S21 graph - shows some bad calibration in S11. If the ports are connected together, you expect S11 to be small, basically 50 ohms, because it sees port 2 which is a fairly good load. That |S11| is still -10 dB is a problem. The S21 shows about the right magnitude for the loss of two cables, -5 dB, so this looks correct (unless you also calibrated Thru, then S21 should show 0 dB...). Adapters do contribute phase but their loss will be < 0.1 dB at these frequencies, and it's mostly by reflection from the discontinuities, not absorption, definitely not radiation.

---

Ways forward:

Try reverse-engineering the cal kit ?
Is the "factory" cal of the VNA fairly good (or can you reset it)? You should see a reasonable open circuit with a small tail, with nothing connected and all calibration turned off.
Try using that to get some independent data for your standards. Use just one N-SMA adapter and a port extension to compensate for it.

Try using zero coefficients?
At 3 or 4 GHz I got pretty good results with three cheap SMA F connectors on a keyring, one shorted, one cut flat as an open (in the same plane) and one load made with two 100 ohm SMD resistors, all coefficients set to zero. So (some of) the coefficients you've entered are at least 10 times away from their true values.

Beware when converting Length to Delay, there may be a factor of 2 lurking in the definitions - is it one way or there-and-back? You might need to ask both Rigol and AppliedEM to clarify, or figure it out yourself. Also to 1st order you don't really need to have the reference plane correct on each standard, just the delta-delay between Short and Open, and you will measure that easily.

Finally, when you are calibrated to the end of the measurement cable, use my knife-short-circuit trick to move the measuring plane to where you want it at the DUT.