As you noticed there's generally a capacitive connection between the DC output and the high voltage side of the power supply. The given comment directs you to an explanation which tells it reduces radio interference. It unfortunately in ungrounded systems bring mains AC voltage onto your hands through capacitors.
The capacitive impedance is high enough to prevent painful shocks, but you can easily feel it. If you wipe with your fingers on a very thinly insulated, say anodized aluminium surface the electric field can be so strong (volts per meter) that it changes the friction (via changing the force) of the wiping and you feel 100Hz roughness which do not depend on how fast you move your finger.
If you have one device in your system properly grounded the effect vanishes because also the synth is grounded and that shorts the leaked AC. But that is also a danger. The grounding happens via signal cables. If you connect or disconnect an audio cable so that the signal terminal connects first or last the audio circuit can enjoy the leaked mains AC and get destroyed. The hot tip of a 1/4 inch audio plugs virtually every time touches metal (=gnd) first when plugging in and last when disconnecting.
The danger grows by a magnitude if you have devices which have grounded mains AC plugs but someone has inserted tape or otherwise broke the grounding to reduce ground loops which catch unwanted noise to the audio system. They say "it's grounded via signal cables, it's safe". Maybe it's grounded but only as long as the signal cables are connected. I cannot accept that thinking, because power supplies which are designed to have grounded AC supply leak easily 1000% more than your power supply. If the grounding doesn't happen very painful shocks are probable. In addition removed grounding make the system illegal, it's an intentional safety code violation. The ground remover is treated as a criminal in case something happens, say a fault which causes damages.
What to do: have one device in the system as grounded but be sure nobody connects or disconnects inter-device cables before the mains AC cables of the system are 100% unplugged from the mains AC outlets. I have seen how music bands have only a single mains AC plug to keep things under control.