Questions tagged [resolution]
98 questions
15
votes
5 answers
Why isn't there any 9.5 digit multimeter or higher?
Why can't you buy a 91⁄2-digit multimeter?
Isn't there any need for it? Is a 81⁄2-digit multimeter the latest high end you can buy? I have tried Keysight, Keithley, and Fluke, but there is nothing higher than 81⁄2 digits.

voltage_drop
- 167
- 1
- 3
12
votes
6 answers
Would reducing the reference voltage of an ADC have any effect on accuracy?
Regarding the following information taken from a paper:
The paragraph is telling about what happens when the ADC Vref is reduced.
There is the statement from the above quote:
Note that if you reduce the reference voltage to 0.8V, the LSB would
…

user1245
- 3,955
- 4
- 37
- 87
11
votes
3 answers
Does higher resolution of LCD/LED display mean higher energy consumption for the very same matrix size
If two mobile devices being compared have exactly the same LCD/LED screen (matrix) size and exactly the same battery, will the one with higher resolution have higher battery consumption and thus will require to be charged more often?
For example --…

trejder
- 286
- 7
- 17
7
votes
3 answers
resolution bandwidth
I have the following trivial question:
I have acquired a noise spectrum with a resolution bandwidth of \$124\mathrm{Hz}\$.
I would like to convert it to \$\mathrm{V/\sqrt{Hz}}\$. Shall I simply multiply or divide the signal by…

user222401
- 71
- 1
5
votes
1 answer
MCP3008 ADC , Reading 16 bits instead of 10 bits
I was playing around with some MCP3008 ADCs from Microchip with an Arduino. The playground has an example for the MCP3208 (which seems to be just a 12 bit version of the MCP3008)
Well, when I tried the example, I forgot to only read 10 bits…

Michael Styln
- 145
- 5
5
votes
2 answers
What does 0.5lsb and 1.5lsb mean in ADC
Quoted from the ebook Introduction to Microcontroller by Gunther Gridling (pg. 43):
The class width of most classes corresponds to 1 lsb, with the
exceptions of the first class (0.5 lsb) and the last class (1.5 lsb).
This asymmetry stems from the…

KMC
- 1,380
- 2
- 18
- 39
5
votes
3 answers
Ambiguity in ADC conversion of analog voltage to digital value
I have question about the common formula used to turn analog input voltage of ADC into an integer binary number that is as bellow :
I think that in the equation above N is not integer but floating-point value. And also we can not consider [] as…

A.R.S.D.
- 53
- 4
5
votes
3 answers
Resolution of potentiometer
I'm a bit confused about the resolution of a potentiometer as a voltage divider.
What's the relation between voltage and number of turns in a pot?
For example In below circuit:
What the resolution of the multi-turn potentiometer would be if we…

ElectronSurf
- 2,185
- 3
- 24
- 61
5
votes
2 answers
How to distinguish between voltage resolution and bit resolution
One might way that an ADC has a resolution of 12-bits. One could also say that it has a resolution of 1.22mV (5v/4096).
Is one of these uses of the word 'resolution' incorrect?
Is there a word to replace one of these uses of the word…

Rocketmagnet
- 26,933
- 17
- 92
- 177
4
votes
2 answers
Time domain reflectometry and resolution
Regarding the temporal (and consequently spatial) resolution of a TDR system, I've read from different sources that the maximum resolution achievable is half of the rising time of the input step response, which is usually in the ns range.
How is…

Anice41
- 95
- 5
4
votes
1 answer
Voltmeter resolution increase by having several in series
The problem
I am trying to measure quite precisely a voltage around 7.2V on a line of a PCB with a voltmeter. In this voltage range, my voltmeter only displays 3 digits. So the reading is something like 7.26V. However, I would like to have 4 digits…

Tulmandil
- 51
- 4
4
votes
4 answers
Confused with the effect number of decimals at the output of transducer
There is an ultrasonic device which measures and outputs both wind speed and direction and has a very long manual. But to keep it short and summarize, the accuracy and the resolution for the wind direction is given in the following table:
As…

ty_1917
- 949
- 1
- 5
- 20
4
votes
4 answers
What sets the resolution of an analog resistive sensor?
When I read about film or silicon strain gauges their data-sheets mentions about their resolution.
But in analog world what sets the resolution? For example if you vary a potentiometer the output voltage changes accordingly so one can say there is…

pnatk
- 1,389
- 12
- 33
3
votes
2 answers
What does DAC resolution mean?
I am reading the specs of a DAC (this one), and it says the resolution is 10 bit. Does it mean it can generate 1024 voltage levels at the sample rate which is 12 GSa/s? It sounds pretty surreal. Can it really do communications at 120 Gb/s or what…

LWZ
- 173
- 1
- 8
3
votes
2 answers
Custom VGA output : what resolutions are expected to work?
I am thinking of creating a device which displays graphics and animation to generic computer display with 16:9 aspect ratio via analog interface (D-sub).
The question is which resolutions are guaranteed to work on most monitors?
Obviously, standard…

BarsMonster
- 3,267
- 4
- 45
- 79