IR receivers work by measuring the leakage current on a photodiode with a high-gain amplifier. Because of the high impedances involved, they can pick up electrical noise from nearby sources - like your finger for example. There are shielded versions of IR sensors which help with this issue. Otherwise, just know to keep your finger away...
Also, some light sources can interfere with IR sensors, such as high-frequency fluorescent light ballasts. A light filter can reduce this.
More: why does it stop when the scope is connected? Your wall-wart DC supply has a 'floating' output that is probably injecting hum on your setup due to leakage from the line to the ungrounded secondary. (If it's a really cheap wall-wart it can be quite a lot.) This is normal and not (usually) hazardous, but it can cause problems with sensitive circuits like this.
In this case the sensor is picking up that floating-ground hum as a difference between itself and your somewhat-grounded body. With the scope attached the power supply secondary is no longer floating, and the hum is shunted away to ground. So with both sides grounded, the difference disappears, and along with it, the sensor misbehavior.
Try this: Touch the board's ground and bring your finger to the sensor. I predict that the problem won't happen then, either. Why? Your body will be at roughly the same potential as that leaky wall-wart ground, no noise will couple to the sensor.
What's this noise then? Try this: Measure it with your scope with the probe ground connected to earth and the board itself ungrounded. That is, have the low-voltage side floating. You will see the floating ground hum / leakage that typically looks like spikes at twice the line frequency. It can be surprisingly large - 60V in some cases, and it may depend on which way you have the wall-wart plugged in. But because it's a weak AC path it's very low current so it's not dangerous.
Where does it come from? The noise is AC coupling from the primary to the secondary, through the transformer that isolates the two domains. And if you’re really unlucky, besides line-frequency noise there may be some switching noise riding on it too, which might be in the bandpass range of the sensor (e.g., 38KHz.)
Finally, why it’s worse in the dark: the sensor AGC function is maxed out so it will be at high gain, making it more sensitive to noise.