Electrical Engineering Stack Exchange
Electrical Engineering Stack Exchange

Questions tagged [accuracy]

137 questions
29
votes
3 answers

Why are resistors tolerances relative instead of absolute?

Every resistor has a tolerance, this provides the user with an idea of the accuracy of the product. This tolerance is represented by a percentage. This means: a big value resistor will be less accurate than a small resistor with the same…
M.Ferru
  • 3,926
  • 3
  • 22
  • 48
27
votes
3 answers

What does it mean when multimeter accuracy is marked as: ±0,03%+10Digit?

I have a digital multimeter and its accuracy for VDC is marked like this: ±0,03%+10Digit This multimeter has maximum display of 80000. So in the 80 V range it can show for example 79.999V. 0.03% of 80V is 0.024V - that is clear for me. But what…
ria
  • 1,965
  • 9
  • 25
  • 37
12
votes
3 answers

How much does temperature affect the 555 IC?

I read that one of the disadvantages of the 555 timer is that it varies in timing accuracy depending on temperature. How much can accuracy vary, and why does this happen? Is temperature the only thing that affects timing accuracy? It seems the 555…
Nick Bolton
  • 2,043
  • 8
  • 31
12
votes
5 answers

Why would a part be available as 4.99k and 5k?

I was looking at chip resistors, and came across the MORNTA1001AT5, which comes in many resistance values. The resistor tolerance is 0.1%, so what is the logic of having the part available in both 4.99k and 5.0k versions? Generally, if you needed…
b degnan
  • 3,167
  • 2
  • 17
  • 36
12
votes
6 answers

Would reducing the reference voltage of an ADC have any effect on accuracy?

Regarding the following information taken from a paper: The paragraph is telling about what happens when the ADC Vref is reduced. There is the statement from the above quote: Note that if you reduce the reference voltage to 0.8V, the LSB would …
user1245
  • 3,955
  • 4
  • 37
  • 87
8
votes
3 answers

Accuracy vs Precision

Does it ever make sense to have more precision than accuracy? E.g. if you had a variable resistor which you could control with a resolution of 0.01 ohms but your multimeter only had 1% accuracy, then that extra resolution would be pretty useless…
jrandj
  • 105
  • 3
8
votes
4 answers

Accuracy of a multimeter over 10 years period

Datasheets of multimeters contain accuracy specifications. One parameter is usually accuracy over period of 1 year. I understand that it means multimeter can be off by the specified value. For example Keithley 2000 for 100mV range: 1 year accuracy…
Chupacabras
  • 5,394
  • 2
  • 23
  • 50
7
votes
2 answers

How do the electrical characteristics of an ADC degrade over lifetime?

How do the electrical characteristics of an ADC typically degrade over lifetime? And is there a way to do a somewhat realistic worst-case estimate for that before doing aging tests over a big batch? Example data, probably for begin of life, taken…
Torsten Knodt
  • 273
  • 2
  • 8
7
votes
1 answer

Why is my 10k NTC thermistor on average accurate to 2C

I am trying to figure out what is causing the 2C inaccuracy in my thermistor calibrations, my goal is to get my accuracy down to > .1C. I am using the KS103J2 10K NTC thermistor. To calibrate my thermistor I am using a vernier temperature sensor…
Aditya Kendre
  • 97
  • 1
  • 7
7
votes
2 answers

Precision notation for voltmeters

From the technical details of a multimeter: AC Voltage: - Range and Accuracy: 200mV ± (1.0%+2); 2V ± (0.7%+3); 20 ± (1.0%+3); 200/600V ± (1.2%+3) What does the notation "± (0.7%+3)" mean? Specifically, what does "+3" mean?
user31264
  • 189
  • 4
6
votes
3 answers

Is the error in a "5%" resistor consistent across measurements?

One of my projects involves a number of voltage dividers... but if I'm worried about precision (which I'm really not, but I want to understand how these things actually work) can I realistically use "standard grade" 5% resistors for these dividers…
ljwobker
  • 295
  • 3
  • 11
6
votes
1 answer

Implications of INL on the accuracy and resolution of an ADC

I was trying to understand the meaning of various specs given in a typical ADC datasheet, and I came across this article. Now, if I understand correctly, it seems to me that the accuracy of the ADC in bits can be taken as \$n - \log_2{{e}}\$ bits,…
mayank
  • 183
  • 1
  • 1
  • 6
6
votes
1 answer

GPS L5C Differential? Max Update Rate(bits/s|Hz)?

The differential GPS chips use L1 and L2C to calculate the ionospheric interference. With the new GPS satellites, the new band L5C is being deployed, and I am wondering about the following things: Will L1 and L5C be used as new type of…
Jack Shephard
  • 53
  • 3
6
votes
6 answers

Statistical accuracy of multiple resistors in series or parallel

I'm having a senior moment and would appreciate clarity. I'm working on an inverting amplifier design where I need to add low-pass filtering and a voltage clamp. Without the filtering and clamp, the op-amp configuration would simply be a feedback…
Dwayne Reid
  • 23,390
  • 2
  • 35
  • 67
6
votes
1 answer

What causes inaccuracy over time on a DMM?

Here is a piece of information from a datasheet of a DMM. The accuracy rates seemed interesting for me. I wonder what causes difference in the accuracy, especially the changes after 24 hrs? Is it only shunt related, or another components related…
Angs
  • 3,275
  • 8
  • 33
  • 54
1
2 3
…
9 10