Why do we need nonlinear electronics for computing?

Question

When I read about photonics, I always see that they can be used for linear transformations (just matrix multiplications), and that this is a limitation that makes them unsuitable for building a complete photonic microprocessor. Why are linear transformations insufficient? What kind of computations require nonlinear transformations, and is there a subtopic in EECS that tackles the separation between applications of linear and nonlinear operations?

Answer 1

Most computers use digital logic. Digital circuits are 'restoring' and minimize (eliminate) signal level error propagation. Analog circuits (which are used for linear transformations) generally add noise (errors) and decrease accuracy as signals propagate, making them unsuitable for complex multi-stage calculations.

Basically, if a logic gate has a signal (voltage level) that is high, or 'nearly high', its output will be even closer to 'perfect'. Similarly for low signals. This means that a logic level will propagate through the logic (with some delays), but at each step it doesn't lose quality -- in fact it improves the precision of the signal.

This comes because the digital logic gates are non-linear. For input signals close to the transition point, it has very high gain -- thus the output signal will be further away from the transition point. Since input and output signal levels span the same range, the corollary of this is that as the input level moves away from the transition point, the output saturates -- i.e. asymptotically approaches the ideal level; this implies that in this condition the gain is very low. Thus the circuit is non-linear.

This means that even to implement linear calculations (e.g. matrix multiplication) only the most basic manipulations are practical (i.e. accurate enough) to implement with analog circuits (e.g. opamp gain or integrators). When linear algorithms are implemented digitally (e.g. floating point computation) it is practical to have millions of variables and billions of calculations -- that is not feasible with an analog implementation.

There are a few cases where analog (linear) implementations can be better -- these are ultra low power circuits where a few transistors can implement a calculation that would require 1000's of digital gates; also extremely high speed circuits (multi-GHz) where digital logic isn't fast enough and some noise (inaccuracy) is tolerated. Examples include the front end of RADAR systems where the initial signal processing and filtering is implemented with analog circuits.

to clarify -- even if the computation is linear, it is generally most robustly implemented with digital electronics (e.g. a computer), likely with floating point representations, and computation using that uses non-linear functions (binary logic operations).

Answer 2

I think you're confusing linear mathematics with linear electronics by attempting to view it all through the lens of abstract mathematics. Or perhaps confusing the method (the logic of the math being performed) with the means (the physical representation of that logic and how the calculation is physically carried out).

Our digital computing is built on ones and zeroes because switches that are either conducting or not conducting are easy to construct in real life. That's it. That's the reason we do the things the way we do it. No need to think about math like linearity transformations.

But it happens to be that a switch is a nonlinear device because the frequency spectra of the signal you get out of the switch is not limited to the frequency spectra of the signal you use to control the switch. But that's we don't use switches because we need nonlinear devices in computing. It's the reverse. We use switches for much less abstract reasons and they just happen to be nonlinear devices.

Answer 3

We ask computers to calculate arbitrary functions of their inputs. Most functions are nonlinear. The simplest example is comparison if(a>b) .... A linear computer can't make decisions.

Answer 4

Consider adding two numbers in binary. They can be 1-bit numbers, to start with:

+	0	1
0	00	01
1	01	10

There are actually two circuits/functions here, one for each bit of the output. The least-significant bit of the sum is represented by the function XOR:

XOR	0	1
0	0	1
1	1	0

and the "carry" output is represented by the function AND:

AND	0	1
0	0	0
1	0	1

Both of these are nonlinear. AND is at least monotone: you could represent it using a linear function of its two inputs and then a thresholding operation — but of course thresholding is nonlinear as detailed by jp314. XOR isn't even that nice: you can never draw a neat line between the inputs that get a 0 output and the ones that get a 1 output.

Adding two numbers of more than one bit requires chaining together the carry-outs of one adder to the input of another adder, so the absence or presence of a one bit at a given point in the output is a very nonlinear function of every less-significant bit of both of the inputs.

Now, is this a consequence of our insistence on binary representation? Sort of. What about analog computers? Well, they're called that because the voltages inside the computer are analogous to the values in the computation. An analog computer with only linear elements could only solve linear problems, and would therefore be rather unexciting.