For audio I can use just sound card, plus OS api to record any signal from 20 hz (below that's blocked via capacitor) to 96khz(that's half of max samplerate).
But what if I just want to go higher? As high as it possible!
How far could I go and what PC hardware should I use, in order to sample that signals?
For example, I want to record broadband signal, and do FFT, to see "what's happenin here?"
how this was even recorded? 
If your goal is to generate the plot shown in your question, the JRE is correct, you would use a spectrum analyzer (SA). In fact, with a spectrum analyzer, the data you'd get would be essentially the plot you showed, and you wouldn't need to do any post-processing to get it.
But, you'll notice that the plot doesn't show any phase information about the signal at different frequencies. And an SA does not typically store phase information. So if there is other processing you want to do that requires the phase information, or requires reconstructing the time-domain waveform of the signal you measured, then an SA will not do what you want.
You can buy a digital storage oscilloscope with bandwidth up to over 100 GHz (and sampling rate over 250 GHz). This would allow you to do what you specifically asked for, Take the FFT of the data to get a spectrogram, and to do it for signals up to ~100 GHz. However you must be prepared to spend quite a lot of money to get this capability. AFAIK these scopes retail in the ~$1 million range at the high end.
You would use an RF spectrum analyser.
You can also use a software defined radio module and something like GNU Radio to capture and analyse the signals.
The GNU Radio Wiki lists compatible hardware. Much of it is USB-based, some network-based.
To see what is happening, you'll need to record the I+Q information of each signal.
This lets you examine the modulation (amplitude, phase, frequency, frequency sweeping, phase-ramps, pseudo-random-behaviors) of each signal. For such as 256-QAM (a rather busy signal), you will need much more information than just the time of zero-crossings. To detect (or make observable) various circuit flaws, such as intra-pulse ringing that the antenna itself did not filter out, you'll need lots of bandwidth. This is in conflict with the broadband thermal noise of a 3GHz unfiltered input.
Additionally, to handle 50dB dynamic range (-12dBxx to -62dBx) you'll need another 50/6 = 8 bits of ADC resolution.
Fundamentally, the broadband ADC noise floors (even if 50 ohm system) do not permit digitizing AT THE ANTENNA, and you will use PreAmplifiers. Since DC_3GHz amplifiers with low noise figure (the noise density, factoring in the Zsource) are expensive and prone to intermodulation between the weak RF and the strong RF, people tend to implement narrow-band monitoring systems for RF.
Friend of mine, who provided unknown services to the USNAVY but knew a lot about signals, reminded me the Vacuum Tubes operating with 30 volt grid-cathode potential were much more robust in the presence of close-in blockers and adjacent-channel energy. Thus the IP2 and the IP3 of tubes was/is crucial for robust communications in the presence of jamming. To achieve these numbers using transistors, you need large resistors in the emitters of the transistors with volts of degeneration voltage ----- as far as I know.
The data you show was almost certainly generated by a spectrum analyzer. However, spectrum analyzers only measure power. They can only tell you that power was present at certain frequencies, not what information was transmitted. To see the waveform, you would need to capture the phase information as well.
The hardware that would allow you to capture the amplitude and phase information of an RF signal is usually some kind of radio receiver. A receiver would convert the signal to a frequency where you can capture it. If you connected the output of AM radio to your computer's mic port, you are capturing the amplitude and phase information encoded into the RF signal. However, this approach is limited to the bandwidth of your audio card and the bandwidth of your AM receiver. If you're adventurous you could hack a Mini-Circuits LNA and mixer to your radio input to extend its frequency range.
If you must collect wideband measurements at the highest frequencies possible, then you are talking about serious lab grade equipment. Tektronix sells this mixed domain scope that will show you waveform data at up to 6 GHz. Rhode and Schwarz or Keysight sales reps could set you up with similar equipment at up to 85 GHz. I recommend renting and not buying if possible.
You say that you want to see the absolute highest frequency signals possible. Receivers have been demonstrated as high in frequency as 850 GHz. However, there are unlikely to be interesting transmissions in those bands unless you yourself are generating them centimeters away. Also, the receiver described in the paper uses a 40 GHz intermediate frequency, so additional down conversion will be necessary.
