YES! The ground wire in the outlet is security feature by design. To fulfill ESD specification, everything, including you, shall be on the same, grounded, potential by design.
If you connect yourself to a ground wire, it should be safe.
If you connect yourself to a neutral wire, it should be safe.
If you connect yourself to a hot wire it depends how you are connected, what your health status is and how lucky you are...
If you touch the wire while on floating potential, ie. not connected to anything else, it is safe. If you are grounded, you can get a kick, suffer form burns or die (from burns or hearth attack). Kids, do not try it at home.
Anecdotally; Resistance of five people is high enough that when they hold each other's hand one can hold the ground wire and fifth can touch the hot wire. They all get small shock. The number of five was found out during experiments on students during physics course. :) Professor was the one, who touches the hot wire.
Regarding the questions burried inside the text:
The case is here just to hold all the parts in the position. You can build wooden case, glass case, whatever you want. If the case is conductive it must be connected to the ground wire of the outlet. As any other electrical device. When you get kicked when touching the case there is something terribly wrong with the outlet or the wiring in the house.
If the device has no ground wire, in must be sealed so you cannot reach working parts at all.
If you don't have proper qualification you should't be able to touch hot wire/part at all. Unless there is malfunction (insulation got broken) or you are darwin prize nominee (you are fool enough to open the device and play around with the wires inside).
Regarding the computer design: Everything inside should have the common ground (usually the case). If you want to have insulating case, you should connect the components to different common ground - thick copper wire. If the case is conductive it is "the stuff thet holds it together" and common ground simultaneously.
Regarding your plan: The power supply case (the box with female screws) is already grounded via outlet. Anytime you touch the screw in the washing machine, fridge etc, you get connected to the case which is grounded via the outlet.
As I looked for some ESD setups, the point of static discharge suppression is in keeping everything on same potential, the ground.
If you try to build ESD workplace without any connection to the outlet your cross-wiring the case, yourself, desktop, floor etc. will equalize the potential to the same voltage but uknown voltage to the ground. There would be a risk of lighting a discharge between any part of your ESD setup and the actual ground.
If you connect everything to a single ground wire in single outlet, you will get no potential difference betwee the connected parts (you, the case, desktop,...) and the ground. There is no risk of discharge at all.
If you want to be connected during working IN the computer, for gods' sake, plug it off the outlet first and don't be conected to anything at all.