To expand on an answer already here, I wanted to include this graphic, taken from an website about the optical properties of different types of art material:

The vast majority of materials that a lidar system encounters will have reflectivity that looks like item #3 in the above graphic. Even though only a small fraction of the incident light is reflected in a direction that hits the lidar, it's still discernible by the sensors. To do this, a bunch of different signal-to-noise ratio boosting techniques are used. For instance
- The laser transmitter shapes the intensity of its pulses over time so the receiver can more easily distinguish them.
(This also helps the receiver get picosecond resolution on the time-of-flight, even if the sample rate is on the nanosecond level.
- The laser transmitter only transmits photons of a very precise wavelength, so that the receiver can more easily pick up the tiny amounts of reflected light.
- The photo-detectors that listen for the reflected light have a small optical bandwidth (so they'll just react to light in a specific frequency range).
However, as you suspect, a lot of materials don't have this diffuse reflection property, like windows, mirrors, or very polished metal surfaces. Indoor robots do have issues in spaces that have lots of glass or windows. I used to work as an assistant in a university robotics lab, and a grad student in my lab published a paper specifically about how to deal with this issue. All this to say that your intuition about this issue isn't wrong, we just get around it by some sophisticated DSP, and by 'luck' (I guess) in the fact that most materials diffuse light.
I guess humans would be pretty bad at seeing if all of our environments reflected light like mirrors do. (or we'd get pretty good at echolocation lol)