9

Actually, in this question I do not ask about particular language or architecture, however I understand there might be some differences.

In physics / engineering it is usually better to handle greater numbers than smaller ones, because of better absolute accuracy. For example when I need to calculate the area of a circle, having its diameter, I'd use the equation S = d * d / 4 * pi rather than S = pi * (d / 2) * (d / 2) (in both cases calculating from left to right).

How does it look in programming languages?

Is it important to provide "better" order of arguments? Are there any compilers that can optimize calculation for this?

Do such constructions make sense:

// finding a result of a*b/c
if (abs(c)<1){
    result = a / c * b;
} else {
    result = a * b / c;
}

(in the example one should also test values of a and b but let's assume they are large numbers)?

I know that using too large numbers I am risking an overflow. I know there is a difference between integer numbers (which are better in addition/subtraction) and float (which are better in multiplication / division).

There is also an issue with integer division, for example in Pascal there is common * multiplication operator and if both numbers are Integer, the result is Integer too, if one of them is Real (equivalent to float), the result is Real too. For division there are two operators: / which always results with Real number and div which takes only Integers and the result is Integer. So in this language it would be better first calculate multiplications and then divisions because Integer divisions may lead to losing the fractional part, and it is better to lose it later than earlier.

But for float numbers, which are stored with mantissa and exponent order of multiplications/divisions seems to be not necessary.

What I want to achieve is as most exact result as possible (speed is not necessary).

gnat
  • 21,442
  • 29
  • 112
  • 288
Voitcus
  • 201
  • 2
  • 5
  • 3
    For a good discussion including a few examples in the sense of yours, see David Goldberg's famous article [What Every Computer Scientist Should Know About Floating-Point Arithmetic](https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html). Short summary: With floating-point, you can massively shoot yourself in the foot if you don't know what you are doing. – 5gon12eder Sep 18 '15 at 08:06

3 Answers3

10

There are two cases to consider: multiplication (including division) and addition (including subtraction). Because floating point numbers are stored in exponential form (i.e as m*2^e), these operations are performed on the mantissa (m) and exponent (e) as separate values, not on the whole numbers involved.

Multiplication (basically) involves multiplying the mantissae (which are always between 1 and 2) and adding the exponents (which are integers). It follows that unless the integer addition of the exponents overflows, the absolute magnitude of the numbers makes no difference to the precision of the result.

For addition, however, things are different. To add floating point numbers, one of the mantissae is shifted by an appropriate number of bits to make the exponents equal, and then the mantissae are added. This shifting operation loses precision proportionally to the difference in exponents. Thus, addition is more precise when the operands are closer in magnitude. In practice this means that if you are adding many floating point numbers that may vary wildly in magnitude, it is best to start with the smallest (closest to zero) and progress out to add the larger magnitude.numbers at the end. If you don't know the relative magnitudes in advance, put them in an array and sort the array by magnitude before summing it. If you do, use an accumulator variable and add the smaller numbers first.

Jules
  • 17,614
  • 2
  • 33
  • 63
  • Yes, this makes a huge difference in the accuracy of the result summation. – Mandrill Sep 18 '15 at 08:53
  • I couldn't observe compilers doing much evil here. While the language might not forbid it, good compilers are conservative enough not to mess with your floating point expressions assuming you know what you're doing. Did you make different observations? – 5gon12eder Sep 18 '15 at 09:01
  • No observations - I just tend to prefer not to rely on what is basically an implementation detail. – Jules Sep 18 '15 at 09:08
  • 1
    "every language I'm familiar with allows the compiler to reorder the terms for efficiency" – No language *I* am familiar with allows compiler optimizations to change the result of the program. – Jörg W Mittag Sep 18 '15 at 09:15
  • I stand corrected. I just looked up the Java spec, which specifically warns not to reorder floating point expressions unless the result can be proved identical. I assume other languages are similar. – Jules Sep 18 '15 at 09:25
  • For summation don't bother sorting - instead use the asymptotically faster Kahan summation algorithm, which has fewer pathological issues than adding sorted lists. – Reinstate Monica Sep 23 '15 at 18:29
0

In fact I do not have much knowledge or idea whether using large numbers is better or not. But I can share two points that I know...

First is Operator Priority, that is something that you probably know. Some operators have higher priority than others (like * will be executed before + in 2 * 3 + 1 unless you use parenthesis).

For exact results; there are two points too. First is, float to int (or reverse) conversion is specific to programming language and different languages may act differently. Even the different versions of the same language may act differently.

Second point in exact results is the float type, because it is known that Floating Point Math may look Broken, because computers used to calculate using binary (base 2) so the number is represented as the power of two. It is not easy to represent 0.2 by the power of 2's and what you get is 0.001100110011001... or such. Using multiplication or division many times on floating point values causes this difference to get bigger and bigger. David Goldberg's What Every Computer Scientist Should Know About Floating-Point Arithmetic have great information on that topic.

Python have Decimal which can calculate and store exact value of a decimal represented number. Most programming languages have similar data types like that.

Community
  • 1
Mp0int
  • 429
  • 3
  • 12
  • 1
    You link to the thread "Is floating point math broken" but apparently didn't even read the information there. Floating point math is not "broken" is the conclusion of reading that article and the others you linked to. – Brandin Sep 18 '15 at 09:18
  • @Brandin I know it is not broken, it is just a dark humour I like to use since many developers who do not know how `float` got calculated stumbled that thought (You probably did not notice that I use the Question title as the link text). I try to explain it in a few words (since that is already told in those articles and I see no reason to told the whole story while giving the related links) but I guess they would not seem enough for you. – Mp0int Sep 18 '15 at 09:37
  • 1
    The page you link to is a question ("Is floating point math broken"), but when you link to it you've turned it into a statement, as if to support a claim. It didn't seem like humour but who knows. I've got no sense of it. – Brandin Sep 18 '15 at 09:49
  • Ok @Brandin I fixed it to remove confusion and misleading (: – Mp0int Sep 21 '15 at 06:49
0

In physics / engineering it is usually better to handle greater numbers than smaller ones, because of better absolute accuracy.

I don't think this is true at all. There is indeed a point when you add a small number to a big number. But regarding multiplication division there is a risk of overflow when you multiply numbers that are both greater than 1, so for example

99999999999999999999999/9999999999999999999x999999999999999999999999999

is better than

99999999999999999999999x999999999999999999999999999/9999999999999999999 because the later can (in some machines) result in an overflow.

On the other hand if you divide several times an expression by a big number you can get an underflow. So if you do have a good idea of the order of magnitude of the number involved it is a good idea to interleave multiplications and divisions to keep the intermediate results within the range of numbers the computer can represent.

In your area calculation S = pi * (d / 2) * (d / 2) is worse because there is one more operation than S = d * d / 4 * pi.

Mandrill
  • 517
  • 1
  • 3
  • 12
  • With your numeric examples why not use floating point if you're talking about such large numbers? And if they're integers and need the exactitude of integer representations, use a "BigInt"/"BigNum" library. – Brandin Sep 18 '15 at 08:42
  • Bradin, what I said is valid for floating point as well. Assuming a computer can represent 10^300, calculating (10^300)x(10^300)/(10^300) will cause overflow if the computer can't represent 10^600, but (10^300)/(10^300)*(10^300) will be fine. – Mandrill Sep 18 '15 at 08:46