The ICSP pins are for programming the Arduino directly with an ISP/ICSP programmer. This lets you do things like mess with the bootloader/etc. Basically, while the Arduino bootloader understands UART (the Rx/Tx business), the AVR microcontroller doesn't, so if you want to program it directly, you have to use the ICSP pins.
See here and here for more details.

The set of 6 pins near the AREF pin are the ICSP for the USB system. The USB interface has its own microcontroller (AFAICT it is the ATmega16u2 in the Uno Rev 3). Apparently folks use this to make it keyboard- or mouse-compatible.
Regarding that lone pin -- it's disconnected. Looking at the schematic, one can see that IOREF is nothing but a +5V voltage reference pin, and that pin 1 (the unknown pin) is disconnected.
