Usually, if you can stand the voltage drop, you use a diode from each supply to the circuit. This way the supplies won't back feed into each other.
If you can't stand the voltage drop, then you would use FETs to switch them into the circuit, using a little logic to turn on whichever FET you want based on which combination of supplies is attached.
But if you're just starting out in electronics, and are not sure of power supply design then go for the diodes.

Each diode will have a voltage drop, a current rating, and a wattage for dissipation. Since you're drawing from a USB port, your current isn't going to be over 500mA, and that current with the voltage drop is less than a watt in power dissipation, so as long as you choose power diodes you should be fine.
The voltage drop for a standard 1N4001 diode ($0.30 each at Digikey) is about 1 V at 1 A, so the output to your circuit would be 4V. If you wanted to get a lower voltage drop you would move to Schottky diode, such as a 1N5817 which has a drop closer to 450mV at 1 A, so the voltage on the output side would be 4.55V. That diode is only a few cents more.
If, however, you require exactly 5 volts to the circuit, you'll have to look at other switching methods, such as FETs. Keep in mind, however, that it'll be more expensive, and nearly every other simple option still includes a voltage drop.
Most 5V devices and circuits will run fine at 4.5V, though, just check their specifications carefully and do some testing.