'Hot-swappable' is a combination of both hardware and software that allows you to connect and remove devices without powering the machine down, without damaging them or losing data, and use them once connected.
The hardware part typically consists of making the connector pins connect in a specific order, usually first ground, then power, then data. Additionally careful design of the device interface is required to limit inrush currents, so that plugging the device in doesn't cause a large voltage dip on the host system.
The software part requires that the host machine can see the new device, and initialise its driver, without requiring a power cycle. This isn't difficult, but has to be designed for. In the bad old days, machines would enumerate attached devices at powerup, and a device arriving later could not be accommodated. It may also need an orderly shutdown of the device before it can be removed, flushing buffers to permanent storage for instance.