if a comparator has delay of x for one transition, will it be x for
every transition?
Delay may vary depending on the amount of 'overdrive' (voltage beyond the comparison threshold). Here's a graph showing response of the LM397 to various overdrives:-

In this case the difference between 5mV and 50mV is 300ns, which corresponds to 0.03% of a 1kHz cycle.
If the signal amplitude is constant then the delay doesn't matter because it will be consistent from one cycle to the next. Delay is generally smaller with greater overdrive, so the effect of amplitude variation on delay can be minimized by making the signal much larger than necessary for detection.
Your 3mV signal is too low for reliable detection by most comparators, so you should amplify it to at least 100mV. If the amplitude varies greatly over time (eg. after plucking a guitar string) then you might have to use an AGC (Automatic Gain Control) circuit to maintain relatively constant amplitude at the comparator, though if the signal is clean and pure just applying more amplification may be enough.
To prevent noise and harmonics from causing 'false' zero crossings you should apply some hysteresis to the comparator, which makes it ignore small variations in the signal. Bandpass filtering the signal before detection will also help to remove noise.
After detection you have to decide how to get the 'several decimal places' of precision that you want. The simplest way is to just count pulses over a fixed time period long enough to get the desired precision. However at low audio frequencies the required 'gate' time is very long.
Measuring the time between cycles is much faster, but makes the result sensitive to variation between cycles. The simplest solution is to measure the time of several cycles and then divide by the number of cycles, which averages them. If your lowest frequency is eg. 50Hz then you can measure and average the periods of at least 50 cycles in 1 second.
At higher frequencies the resolution of the timer may become significant. If so then just switch to pulse counting mode. At 1kHz you have 3 digits past the decimal point with a 1 second 'gate' time.