There should be no big problem to replace internal battery with external DC source. In my former place of work the software people routinely (and without any reservations!) used various benchtop variable PSU to feed phones and tablets under development directly to battery connector, because the internal charger management firmware/software was broken at the time.
From your level, the module like "Mini LM2596s 3A DC to DC Buck Converter Power Supply Step Down Module" will do the job. You can set the output voltage to something in the range between 3.6 to 4.2 V, although the phone might still be confused regarding "battery charge status". If the confusion won't be gone after several power-on-off cycles, then you are out of luck, since phones use pretty sophisticated methods to check battery health status.
Also, if the battery has a built-in thermal sensor (usually as a third contact on battery case) , you might need to simulate "good thermal status" by connecting a 10k resistor to ground. Your battery (BL-53YH) does have two additional contacts, but in recent phones they are used for stick-on near-field antenna, so no worry about them.
However, it might have more economical sense to get a cheap replacement battery from e-Bay or AliExpress, they seem to be in $10 range in qty.1. At least the risk will be much less than from developing a regulator in battery fromfactor on your own, with AC-DC wall supply (or carrying a USB cable), or other ugly stuff.