12

Floating point calculation basically involves representing units in a scientific notation and then deciding how many bits to devote to the manitssa and exponent. Therefore, all calculations involving FP numbers involve these two quanities which must be manipulated. This sounds simple enough and is not hard to do on paper.

I have always come across description of floating point hardware design as being difficult and heard/read things like multiplying and dividing a number by 1 may not give the same result. This perhaps has something to do with how numbers are "unrolled" when arithmetic is to be performed.

Shouldn't there be a unified approach to how floating point hardware is designed in hardware? Why is design and verification of such a hardware considered to be difficult and challenging in spite of there being IEEE 754?

Ken Williams
  • 103
  • 4
quantum231
  • 11,218
  • 24
  • 99
  • 192
  • 10
    ieee 754 tells you what the results must be, not how to do it. If you can come up with a quicker way to get exactly the same results, then you could sell your idea to chip manufacturer. The payoff for them is they could reduce their chip area and so improve yield. You know there are several ways to multiply two numbers together, right? One or another might be a better fit for your process. Start with schoolbook, and improve with various clever factorisations and identities. – Neil_UK May 19 '19 at 19:33
  • 2
    I assume this would be a major area of research and people come with new methods from time to time? – quantum231 May 19 '19 at 19:45
  • 3
    I imagine because there are a lot of design tradeoffs and objectives that can be prioritized because no only are you working with logic on paper, you're also working with silicon gates. It's like op-amps...they all do the same thing yet no standard design with thousands of varieties. – DKNguyen May 19 '19 at 19:48
  • 3
    IEEE 754 makes it harder to implement, not easier. Just look at Xilinx offering and their deviation from 754: https://www.xilinx.com/support/documentation/ip_documentation/floating_point/v7_1/pg060-floating-point.pdf There are many corner cases to handle for floating point – Jonathan Drolet May 19 '19 at 19:57
  • Bipolar Integrated Technology (aka BIT) fielded ASIC chips that provided floating point done just as you suggest. That would be prior to 1990, as I was working with one of the engineers from there on a separate project using the MIPS R2000 around 1987-ish. I cannot say how complete they were in terms of implementing IEEE 754, though. I'm pretty sure they didn't implement the full specification. BIT was located in the Beaverton, Oregon area. – jonk May 19 '19 at 20:01
  • The addition is the difficult part. You need to be able to shift the mantissa by an arbitrary amount when you equalize the exponents using a barrel shifter, for example. This is really difficult to pipeline. All the other steps are relatively simple. – user110971 May 19 '19 at 20:05
  • I shall read the Xilinx documentation. However, I am still trying to understand what the corner cases are and, what the tradeoffs are. Wonder how long it shall take to get there. – quantum231 May 19 '19 at 21:28
  • @quantum231 Tradeoffs on a silicon due means area, yield, and cost vs speed. There are probably at least a few dozen ways you can go about doing the same thing on paper through logic, and then each of those ways have several dozen ways it can be implemented on silicon. – DKNguyen May 20 '19 at 01:56

3 Answers3

23

The standard is well designed and there are subtle details that ease implementation, for example, when rounding, the carry from the mantissa can overflow to the exponent. Or integer comparisons can be used for floating point compares...

But, an FPU is a big heap of combinatorial mess; besides adding, multiplying, dividing, there are barrel shifters to align mantissas, leading zeros counters, rounding, flags (imprecise, overflow, ...), NaN and denormals (which need additional hardware for calculations, particularly for mul/div, or at least trigger an exception for software emulation).

And most FPUs also need to do conversions to/from integer and between formats (float, double). That conversion hardware can be mostly implemented through existing floating point hardware, but it incurs additional multiplexers and special cases...

Then, there is pipelining. Depending on the transistor budget and frequency, either add/sub/mul can have the same throughput, or double precision can be slower, which can incur additional complexity in the pipeline. Modern FPUs now have a pipelined multiply-add operator.

For division, it is always iterative, it can be a separate unit or reuse the multiplier-adder for Newton-Raphson or Goldschmidt. And while you are busy making a divider, you look for ways to tweak it for square roots...

Validation is complex because there are many corner cases. There are a few systematic test suites with test patterns for "interesting" cases about all the rounding modes but things like fast multipliers or dividers are too complex to test easily. Iterative dividers can have non obvious bugs (for example the famous Pentium bug in its SRT radix 4 divider), multiplicative (Newton) are difficult to test exact rounding (some bugs in old IBM computers).

Formal methods are now used to prove these parts.

Modern FPUs also implement SIMD hardware, where FP operators are instantiated several times for parallel processing.

There is also the case of the x87 and MC68881/2 FPUs which can calculate decimal conversions, hyperbolic and trigonometric operations. These operations are microcoded and use basic FP operators, they are not directly implemented in hardware.

Glorfindel
  • 1,245
  • 3
  • 15
  • 20
TEMLIB
  • 3,347
  • 1
  • 14
  • 21
  • 1
    The Standard suffers a bit from trying to serve all purposes, and thus being too complicated to serve some while lacking features needed to serve others. For example, questions about the cases when it should guarantee "perfectly" rounded results were based upon whether that would be *possible*, rather than upon whether the costs would be worth the benefits for all applications. For many purposes, a computation that yields a result within two units in the last place would be more useful than one which yields a perfectly-rounded result but takes twice as long,... – supercat May 19 '19 at 23:37
  • 1
    ...and on many implementations, computing a result within two ULP would take less than half as long as computing a perfectly-rounded result (as a simple example, computing `x*(1/1.234567)` will yield a value within a couple ulp of `x/1.234567`, but will be much faster than computing the latter value). There are times when perfect rounding is useful or even necessary, but having a means of specifying when it *isn't* necessary would also have been useful. – supercat May 19 '19 at 23:41
  • @supercat Yes. For things like divisions, inverse square root... they are often implemented for graphics using pipelined Newton-Raphson on the FPU multiply-add hardware. Denormals are often flushed to zero (for example in DSPs, GPUs). The default rounding mode "round to nearest-even" is slighly easier to implement than the other modes (to infinity, to zero), because of rounding, so sometimes it is the only mode available. – TEMLIB May 20 '19 at 01:25
  • The standard was designed by a Mathematician, W. Kahan, and tried to address the shortcomings of previous formats, notably DEC VAX. The first implementation, the 8087, was a rather slow implementation but with extended precision and lots of microcode for a "math library in a chip". – TEMLIB May 20 '19 at 01:33
  • I've read some of Kahan's papers, and in most cases where things have evolved contrary to what Kahan advocated, I think Kahan's approach would have been better, but he did make a few mistakes. I think the Standard would have benefited from having an unsigned zero along with signed infinitesimals; 1/(tiny * tiny) should yield +Inf, but 1/0 should yield NaN, to avoid the asymmetry with zero. Still, parts of the Standard fail to consider that cost/benefit trade-offs are different in different kinds of application. – supercat May 20 '19 at 02:36
  • 6
    This is the kind of answer that makes me furious about SEEE's tendency to close such questions as "opinion based". Bullshit. There might not be a singular answer, but there sure is valuable perspective to be conveyed to the not-a-neophyte who asked the question. – Techydude May 20 '19 at 16:08
6

Having a look on opencores might give some hints e.g.: https://opencores.org/websvn/filedetails?repname=openfpu64&path=%2Fopenfpu64%2Ftrunk%2Ffpu_mul.vhd

The trouble with floating point is the large number of annoying corner cases. Integer operations have no concept of NaN, but it appears a lot in floating point. Numbers must also be normalised and denormalised correctly.

pjc50
  • 46,540
  • 4
  • 64
  • 126
5

Even if you don't handle all the corner cases, floating-point addition or subtraction of two well-formed numbers requires significant logic, because the scale of the mantissa can dramatically change -- consider the problem (in decimal) of the problem 1.9999 - 1.9993 = 0.0007. In floating point the location of the decimal point must be discovered, which isn't trivial, and the mantissa and exponent adjusted. This is even without trying to deal with NaN or denormalized numbers.

All the mention of handling the special cases is quite valid, but even if you put the onus of avoiding special cases on the system designer (which is not uncommon with floating-point IP intended for DSP applications), your floating point arithmetic is still more expensive than equivalent-sized fixed-point arithmetic.

Witness the latest Altera/Intel FPGAs, which have "DSP blocks" that are twinned, and will either do n-bit (I think it's 32-bit, but I'm not sure) fixed-point math in each block, or will do the same-sized floating-point math in one pair of blocks -- so going to floating point not only loses precision (because you only have 25 effective bits of mantissa in an IEEE 32-bit floating point), but uses twice the resources, with very limited handling of corner cases.

TimWescott
  • 44,867
  • 1
  • 41
  • 104
  • We live in age where nm resolution in fabrication has become quite small and logic resource in FPGAs is quite cheap. What difference does it make how much logic is required for a FPU that complies fully with IEEE 754? – quantum231 May 19 '19 at 21:29
  • 3
    Well, again, look at Altera parts. The FPU can be a big part of a processor if it's fully IEEE compliant. In an FPGA that's filled with a DSP algorithm, those "sorta-compliant" blocks allow far more computation in the same amount of silicon than fully compliant ones would -- and FPGA DSP is often limited by the number of blocks you can afford. – TimWescott May 19 '19 at 23:53
  • @quantum231 size *always* matters in the commercial world; either you use the extra space to make it cheaper, or to add more features. – pjc50 May 20 '19 at 10:37