For distance and to avoid noise, also because the data is transmitted without a clock, the data is encoded, for example 8b/10b for every 8 bits of real data 10 bits go on the wire. If you sent many zeros in a row without something like this or one of the other schemes, you might just have no signal, this is bad. So even if you have a lot of zeros or a lot of ones the signal on the wire, pair of wires (differential) is constantly changing. As a result of the signal constantly changing you can easily detect on a receiver if something is there, and get link state and what speed the link is for 10/100/1000 interfaces for example.
This is independent of whether or not a higher protocol level is working. Basically you have a link between the phys on the network cards but then you go to the mac layer and try to talk, then on up through the rest of the network layers. If the phys are not linked though then nothing will happen.