22

What is the best way to explain floating point rounding issue to customers?

I know

http://download.oracle.com/docs/cd/E19957-01/806-3568/ncg_goldberg.html

as well as the entries in the C++ FAQ and various other pages aimed at developers and scientists, but is there a web page, article or explanation, aimed at "regular" customers with limited mathematical or scientific background? (for which the above references fall flat).

If it were maintained or coming from a well-known and well-recognized institution or corporation, all the better, given that, as some of you might have experienced, it can be a little complicated to explain that yourself.

mouviciel
  • 15,473
  • 1
  • 37
  • 64
Eric Grange
  • 221
  • 1
  • 5
  • 1
    I wouldn't bother... – John Shaft May 17 '11 at 07:53
  • How is that going to help? – Aditya P May 17 '11 at 07:56
  • Some customers consider those precision issues as bugs. – Eric Grange May 17 '11 at 08:00
  • Unless they really need to know the detail and they have the aptitude I wouldn't be teaching mathematical concepts to my clients – John Shaft May 17 '11 at 09:37
  • 1
    This is a really fantastic question, +10 if I could. A frequent problem for developers. – Cody Gray - on strike May 17 '11 at 11:21
  • 2
    It's not a problem of detail, it's answering f.i. why does adding what looks at 2 decimal digits precision numbers, you end up with a 5.9999999 and not a 6, and why you have to specify rounding precision when it's "obvious" the result shouldn't have more than two decimal digits. Or why sometimes 2 minus 2 isn't always zero, and not look like a bamboozling fool while doing so. – Eric Grange May 17 '11 at 11:56
  • 9
    @Eric Grange : if your customers consider those precision issues as a bug, then, it *is* a bug and you have to find a way to fix it (maybe by not using float). They don't care about where this precision issue comes from. They don't care about how your software works. They just want it to work. – David May 17 '11 at 12:14
  • well they might wanna care when it comes to banking transactions. thiose places even a 0.00001 "defect" on each transaction could cause the bank to end up knee deep in lawsuits and criminal fraud – Chani May 17 '11 at 12:17
  • @David: Not necessarily. I've seen lots of complaints about floating-point imprecision even when it didn't actually matter. If it does matter (like financial transactions representing the number of dollars in floating-point with cents as the fractions) you do need to implement something else (typically using some sort of integer to represent cents). – David Thornley May 17 '11 at 13:58
  • 1
    @David As David Thornley pointed, they complain even if it doesn't have an effect, and they complain too if you point them to fixed-point solutions (because they are more restrictive, and then they have to specify the precision they need, and they think they shouldn't have to, because, well, it doesn't matter in their case, back to square one). Also fixed-point has worse precision issues once division is involved, so it leads to tailored (expensive) solutions. – Eric Grange May 17 '11 at 14:12
  • 1
    @David Thornley: I would say that the client should be the one to decide whether it actually matters or not. We can help him decide of course. But, I see your point :) – David May 17 '11 at 14:15
  • 2
    What's the context? Why on earth are you discussing implementation details with a customer? – Tom Anderson May 17 '11 at 14:29
  • @Tom Anderson Re-read the question, the whole point is to explain why floating point can exhibit precision issues even in "trivial looking" situations, precisely **without** having to discuss implementation details, if possible by having a simple, official and non-bamboozling looking webpage, reference, etc. – Eric Grange May 17 '11 at 16:05
  • 3
    @Eric: The use of floating-point *is* an implementation detail. I repeat my question, which asks for not something explained in the question, and which i have evidently not made clear (apologies): what's the context, and why are you discussing the use of floating point with a customer? – Tom Anderson May 17 '11 at 16:46
  • 1
    The contexts are varied, as experienced by others, it's a recurring question, and typically precision in those contexts is irrelevant, it is purely a matter of display and rounding (or lack of), and of the customer accepting the need or not to merely having to specify a number digits for rounding. The issue is discussed, because brought up by the customer. Floating point is not an implementation detail, it's "the" industry standard, and alternatives (fixed point, large numbers, etc.) all have costs and constraints attached to them which make them only useful when truly required. – Eric Grange May 18 '11 at 09:01
  • an example that i always use to explain to DSPers what might be a problem with floating-point is the application of an efficient moving-average filter using a delay line and an integrator. you want to make sure that the number you add to the integrator is exactly what will be subtracted. in fixed-point you can be guaranteed of that, but not with floating point. – robert bristow-johnson Aug 30 '15 at 21:47

8 Answers8

8

I find a simple way to explain this is to demonstrate it. Discuss how dividing x by a number, then multiplying by the same number should return you to x again - get the customer to agree that this should always be the case. Then do the old (100 / 3) * 3 on a calculator; show that the value doesn't, as you would expect, return to 100. When most people see apparently simple maths "breaking down", then tend to 'get' the danger of floating point numbers where accuracy is important (although in an intuitive way, rather than to the low level the article you point to goes into).

Unfortunately most half-decent calculators (certainly all the scientific ones I've seen, and more than a few basic ones) nowadays are able to handle this - I presume they're storing extra digits beyond what can be displayed and rounding - so do check how clever your calculator is before you do it in front of your customer.

Scott
  • 2,091
  • 2
  • 18
  • 21
  • Repeating a more complex operation a few times and then the inverse the same number of times is usually enough (eg. apply square root to 2 eight or more times, square the result eight times and then subtract 2; the result is not zero). – Richard May 17 '11 at 07:53
  • 1
    Yes, almost all calculators are storing at least 2 extra digits, so you have to add a few multiplies in the mix, which muddies the explanation, and IME makes them think you're trying to fool them. Square root requires less operations, but square root is already outside the everyday realm of the regular customers. – Eric Grange May 17 '11 at 07:56
  • @Richard Yes, but the more complicated the operation, the more you risk bamboozling those of a non-mathematical bent. You should use the simplest operation you can - `(100 / 3) * 3` works well, because almost everyone can divide and multiply by three - thus, the fact that the calculator "got it wrong" is more immediately obvious, and it makes it clear you're not playing some 'take away the number you first thought of' trick. – Scott May 17 '11 at 07:59
  • @Scott: true, but if such a simple calculation doesn't show the error it doesn't help. In the end you've got to balance users who demand to know why but don't want the complexity needed to understand why. (As Douglas Adams put it "To summarise the summary of the summary: people are a problem".) – Richard May 17 '11 at 08:03
  • 2
    @Scott I tried on a few calculators here, none exhibited issues with (100/3)*3, even (100/3)*3-100 didn't exhibit issues.... Excel gets it right too. – Eric Grange May 17 '11 at 08:05
  • @user25419 I doubt it'll work on anything made in the last decade or so unless it's *really* naff. Foiled by cheap semiconductors yet again! ;-) – Scott May 17 '11 at 08:11
  • Grab a financial calculator and put it into 2, then 4, then 6 decicaml point precision, then explain that the computer has the same problem, just more deciamal points. – mattnz May 17 '11 at 09:17
  • 9
    Take money as an example, that has a perfect limited precision. Explain you divide up one dollar, then every person gets 33 cents and one penny is lost in rounding. Anyone can relate to that. – Inca May 17 '11 at 13:00
  • @Inca Problem is it doesn't take a division, additions is enough for the issue to popup, and money-based examples don't happen with additions. – Eric Grange May 17 '11 at 13:56
  • 4
    Don't bother with the calculator. Divide 1 by 3 on paper, keeping three significant digits. – David Thornley May 17 '11 at 14:00
  • You can make it a multistep problem. Start with the identity x=sqrt(x)^2, then this implies x=(sqrt(sqrt(x))^2)^2. generalize for N repeated square roots and squares. The bigger N the more precision is lost. Plot the error as a function on N. – Omega Centauri May 17 '11 at 14:27
  • @Inca - ooooh, remember in Superman 3, when Richard Pryor stole all that money by taking the rounded bits and putting them in his own account? :) – Scott Whitlock May 17 '11 at 14:38
  • 5
    @omegacentauri if you think that explanation helps, I'm guessing you don't talk to customers often. – jhocking May 17 '11 at 15:00
  • @Eric Grange: I would start with the division, and then explain the same thing because of the way it is internally stored. When working from an analogy they can understand you can explain it a bit more - and no, not fully because if they would like to understand the problem fully they probably be programmers themselves. I'd focus on a simple example of the problem, the scope (how much can it deviate) and ways of handling the differences, and leave it at that. – Inca May 17 '11 at 15:12
5

I don't think there are shortcuts. You have to either:

  • Understand what floating point is and how it behaves.

Or, if that's too much required, the you have to just:

  • Accept that the computer won't give you exact numeric results.

Maybe an example with irrational numbers helps (even though floating point issues apply to rational numbers as well): sqrt(2) ~ 1.414. Then 1.414^2 = 1.999396. No matter how many digits you take, you'll never get quite back to the original 2. Ok, 4 significant digits correct may be acceptable, but then consider what happens when this kind of "rounding errors" accumulate. That's where the real danger is.

Joonas Pulakka
  • 23,534
  • 9
  • 64
  • 93
  • 2
    I personally know and understand, but for some people "floating point" is already an alien term, so you need more than mathematical or scientific explanation to explain that what they can calculate right in their head, their expensive computers and software has trouble getting right ;) Also square root is outside the everyday realm of regular customers. – Eric Grange May 17 '11 at 07:54
5

First, determine what they're complaining about. Financial transactions have to be done precisely, with the right number of decimal places and the right rounding rules. This typically means maintaining integral numbers of currency units and making sure the arithmetic is done right.

Alternatively, they may be complaining about overexact displays, and reducing the number of significant digits output may be all that's necessary.

For numbers in general, you can always try to come up with a three-digit decimal x such that x * 3 is 10. That shows the basic principles.

There are two remaining problems. One is that certain numbers can be expressed exactly in decimal but not binary (3.15, say). That's going to be hard to explain to non-technical people, and your best bet is to try to avoid it by not providing enough significant digits for it to show up. The other is the customer who knows a little bit, enough to know that computer arithmetic isn't always exact and not enough to realize that decimal arithmetic isn't always exact. I have argued with a few of those, and have nothing useful to report.

David Thornley
  • 20,238
  • 2
  • 55
  • 82
2

Floating point numbers in computers use binary, so just like we have a number system with a ones, tens, hundreds, and tenths, hundredths columns, floating point numbers in computers actually have a ones, twos, fours, and halves, quarters, and eighths columns. If the customer is familiar with feet/inches, then remind them of how you typically use base-2 fractions of an inch for measurement.

Now try to store 10 cents as a combination of halves, quarters, eighths of a dollar. It just doesn't work:

.00011001100110011 . . . (repeats infinitely)

It's the same as taking a standard imperial measuring tape and trying to measure one tenth of an inch. You can't do it accurately. There is no representation of 1/10 as X/Y where X and Y are whole numbers and Y is a power of 2.

That's why we have the decimal data types that use 4 bits to store each decimal digit, so we're back to base 10 representation. The trade-off is in space and performance (about a 100% performance hit, from what I've read).

Scott Whitlock
  • 21,874
  • 5
  • 60
  • 88
1

2/3

Ask them to write down the exact answer to two divided by 3.
Since the answer 'goes on for forever' you can point that out.

Using 1/3 would also work but 2/3 is perhaps a slightly better example as rounding gives you (e.g.) .6666667 whereas .3333333 looks like it can just be truncated.

Michael Durrant
  • 13,101
  • 5
  • 34
  • 60
1

Tell them that just like their bank account cannot hold 4.4423425908459032890413... dollars (it's either $4.44 or $4.45, nothing in between), the computer cannot easily store a number with arbitrary precision. Imperfections of storage lead to imperfections of computations.

(It's slightly cheating, but should give them an idea of what the problem is.)

quant_dev
  • 5,117
  • 2
  • 22
  • 26
  • 2
    Alas, that explanation doesn't work, as the precision issue can happen when summing up numbers that all have only two digits of precision to begin with. – Eric Grange May 17 '11 at 08:07
  • 1
    Two *decimal* digits. Yes, I agree, an inquisitive customer will spot holes in it. But then you can hit them with the discussion of binary representation -- they asked for it ;-) – quant_dev May 17 '11 at 08:21
  • Well, already trying to explain floating point they IME immediately start thinking you're trying to bamboozle them, which is something that could be alleviated if it was coming in simple, understandable terms, or from a well-know institution or corporation. :) – Eric Grange May 17 '11 at 08:33
  • 1
    @Eric Math is hard, let's go play baseball :P – quant_dev May 17 '11 at 08:36
  • 1
    Ask whether it's more precise to measure something to the nearest 1/10", or the nearest millimeter. The latter is more precise, but objects which are a precise multiple of 0.1" will not be a precise multiple of 1mm unless they are also a precise multiple of 5" (precisely 127mm). Adding the size of two 2.54mm objects which are measured to the nearest 0.1" will yield a combined size of 0.2"; adding together the sizes rounded to the nearest millimeter will yield 6mm even though the actual size should be 5.08mm. – supercat Jul 31 '12 at 22:59
0

Some calculations are done according to some legal rule. For example, if you want to calculate how much income tax has to be paid on a taxable annual income of €79.245,18 in Germany, there is only one correct answer. You get it right or you get it wrong. If you get it right, you don't need to explain how floating point arithmetic works. If you get it wrong, you don't need to explain how floating point arithmetic works, you have to fix your broken code.

Sometimes you display results that don't look right. For example, if you convert US$ 13,297.46 into UK£ with two decimal digits, and then convert that amount of UK£ back to US$, you might not get US$ 13,297.46 but US$ 13,297.45 or US$ 13,297.47. That has nothing to do with floating-point arithmetic. It's an unavoidable problem and you better be able to explain why it is unavoidable. (You should also know why the problem doesn't happen when you convert from UK£ to US$ and back).

There are other possible results that don't look right. If you convert numbers to percentages the percentages should add up to 100%, but they might not. If you display four percentages with two decimals, the four displayed percentages might add up to 99.99% or 100.01%. Nothing to do with floating-point arithmetic. Still you should be able to explain why.

Next, there are situations where careless use of floating-point arithmetic leads to inappropriate results. For example, a + b + c is usually not the same as b + c + a. If that causes a problem, there is nothing to explain, it's something that you fix.

gnasher729
  • 42,090
  • 4
  • 59
  • 119
0

When doing calculations computers usually use approximations to numbers (like rather than using 1000000.7 they use 1000000) because using approximations is much faster. The problem with that is that when you do calculations with approximations you get approximations back. Usually that works pretty well, but sometimes it leads to unexpected results.

  • I don't really understand what you're saying here. "Because using approximations is much faster"? Sometimes integer arithmetic is at least as fast, and that's precise. Sometimes there's no alternative (as in printing out the square root of 2). – David Thornley May 17 '11 at 14:04
  • Well you try explaining to a marketing guy why computers can't actually represent irrational numbers, or basically any number in the grand scheme of things (oh and then you might want to give him a quick lecture on the irrationality of \pi: something on Fourier series might be cute). Approximate is a word that people can understand. You are approaching this from the point of view of someone who knows that all numbers are not equal. –  May 17 '11 at 19:56