This is not trivial.
LED's typically work like this:

simulate this circuit – Schematic created using CircuitLab
The [R]esistor is important because it controls how much current goes through the LED. The brightness of an LED is controlled by current; the LED's voltage itself usually doesn't change much. (So if you hook up an LED directly to a power supply, it could easily blow up, even if the supply's voltage is close to the LED's voltage. Your strips avoid this by including the resistor internally.) The current is the difference between the supply voltage and the LED voltage, divided by the resistance in ohms: I = (V - Vled) / R
So, to change the brightness, you need to vary the current, and you want to do this using a pot. There are two ways you can do this:

simulate this circuit
where A is the position of the pot as referenced from one end of its travel, times the total resistance of the pot.
In the first example, you effectively modify the supply voltage as [V * A], and at the same time you add [(A in parallel with (1 - A)) * total_pot] to the existing resistance R. Feed that into the current calculation, and you get the new brightness. [1]
In the second example, you only add [1 - A] to the existing resistance R. Feed that into the current calculation, and you get the new brightness. [1] You can connect the unused End to the Middle like I did, or you can leave it unconnected. It works the same either way, until the pot gets old or gets dirt in it; then the difference is whether you want minimum brightness or nothing.
[1] This brightness is still in units of current, and the LED will produce so many lumens per amp at a fairly constant rate. However, your eyes don't see it that way. It's a bit like audio, where a physical multiplication results in a perceived addition.
All that being said, I would not recommend a pot for this. Yes, it would work...for a while...but as Scott mentioned, the pot itself will dissipate some power and get hot. You can use one anyway with an appropriately rated pot (since you have current already, Power = Current^2 * Resistance), or you can be more efficient with a PWM controller. The PWM version should probably be a separate question.