The output impedance is the drain impedance in parallel with the resistor. The drain impedance is very large when the MOSFET is off, so the output impedance is dominated by the resistor. The output resistance can be decreased by decreasing the value of the resistor, but this will result in excessive current through the MOSFET when the inverter output is low.
A large output impedance is noisy for a few reasons. First, the resistor contributes thermal noise. The equation for thermal noise \$\sqrt{4kTRB}\$ where \$R\$ is the value of the resistor, so the larger the resistor the more thermal noise.
Second, if the inverter output is connected to another amplifier, that amplifier's input current noise is multiplied by the source resistance (the inverter's output impedance) to create voltage noise at the input of the second amplifier. The larger the source resistance, the greater the voltage noise at the second amplifier's input.
Thirdly, high impedance nodes are more sensitive to capacitive coupling as seen in this question: Why are high impedance circuits more sensitive to noise?