Yes, you generally want the resistor there. It forms a high-pass filter with C1, and you would normally want the time constant short enough that the level at the non-inverting input stabilizes reasonably fast, but long enough that the cutoff frequency of the high-pass filter lets through the desired signal.
The output voltage of the op-amp (with no input and power applied) will stabilize near (1+R3/R2) times the voltage on the lower end of R1. So if you connect R1 to ground, the output voltage should be near zero (ignoring the bias current multiplied by R1 and op-amp input offset voltage). If you connect it to something other than 0V, the output voltage may be different from 0V and may even saturate.
If you omitted R1, and used a very low leakage op-amp and a low-leakage capacitor then it might appear to work, perhaps even for quite some time, but eventually the output average voltage will likely drift towards one rail or the other. If you used a relatively leaky capacitor such as a bipolar electrolytic, then it might work fine, though under some conditions the capacitor could find itself charged (apply a voltage beyond +/-Vs to the input, for example) and might take many minutes to recover.
If the gain is high, you might want to add another capacitor in series with R2, which will reduce the maximum output offset voltage since the input offset voltage will no longer be amplified.