A capacitor can economically replace a battery for only short period of time due to its limited energy storage. Using your numbers, a 0.023 amp load at 1.2 volts is equivalent to a 487 ohm load (1.2 volts / 0.023 amps). If we assume that the capacitor must keep the voltage within 5% of the 1.2 volts (i.e. no less than 1.14 volts) for say a 10 day period, we can compute the required value of capacitor.
The formula to use is:
Capacitance = -t/(R*ln(Vend/Vstart))
Where capacitance is in Farads, t is in seconds, and R is in Ohms. So if we plug in your numbers, we find that we need a capacitor > 34,000 Farads. Note that is Farads, not microFarads. A capacitor bank this size would cost several thousand dollars and occupy a volume greater than a server class PC. Not at all practical for the stated conditions.
But now you have the formula so you can plug in your own values to find the required capacitance.