Tom Carpenter already did a great job explaining the main question, so I'll answer the side question:
For example, if I were to power a push button by connecting it to the
board's 5V and ground pins, and then connect another leg of the button
to one of the board's digital I/O pins, would this be allowed? Or
would this burn the circuit because 5V is too high?
Many push buttons have 4 legs (or maybe even more). These legs are usually all disconnected from each other, until you press the button, then all the legs are shorted together.
So if you connect 5V, GND and the microcontroller's input pin to the same button and press the button, you will short 5V to GND, which is very bad.
Depending on your power source, you are now looking at anything between no power on the board (e.g. if you power it from a low-power alcaline battery), a broken power supply (when you power it from an LDO or other simple power source) or a fire (e.g. if you power it from a high-power lipo battery).
Whatever happens, the 5V rail will get pulled close to the GND rail, so your microcontroller will probably brown out (means, there isn't enough voltage to run the controller, so it temporarily stops working). If it doesn't, you will probably be reading garbage from the IO pin (can be either HIGH or LOW).
So whatever you do, don't short 5V and GND in any way, not even across a button.