Should I write my code to be clear what I am doing and rely on the optimizer to clean up my code efficiency, or should I be obsessive about getting every last ounce of power out of my code?
And how much speed/size am I losing on by choosing one option over the other?
Both :-)
Seriously, premature optimization can be a problem - you might wind up spending hundreds of hours optimizing a routine that is only run once/week. Also, fully optimized code is often harder to read and debug.
Consider - you can spend a couple of days the processes 10,000 records per minute. It's easy to read, easy to maintain, easy to test.
Or, you can spend a week pushing that up to 50,000 records per minute. However, if a bug takes you another week to find and fix, you really haven't gained anything - all the time saved by 'fast code' is lot in 'extra debugging'.
Having said that (you can see that I'm heavily in favor of write clear code), you should still have some sense of what you're asking the compiler to do. Loops nested deeply will run slow, and extra operations inside the inner loop will be executed a lot more often than one might expect.
Write clear code until you run into a performance problem, then measure and test the code until you know exactly which bit you need to optimize.
Be sure to measure and test again after making your optimization changes so you know that they worked - they don't always.
There are 2 very different kinds of optimisations.
The first is micro-optimisations. These are things like (e.g.) changing x = (y * 4 + z) / 2 into x = y + y + z / 2, or x = y % 8 into x = y & 7. Compilers are very good at micro-optimisations, so don't bother.
The second is algorithmic optimisations. Things like replacing "array of structures" with "structure of arrays" to make code more suitable for SIMD, or using multiple threads instead of one to take advantage of multiple CPUs, or ensuring a list remains sorted while its being created and modified to avoid the need to sort it after, or using an array instead of a linked list to get rid of "unpredictable (for CPUs) pointer chasing". These are things that compilers struggle with. They're also things that can cause major quantities of effort if you attempt to retro-fit them into existing code; and it's far better to consider these things during the design phase before any of the code is implemented.
I think you have a wrong understanding about optimization, so here are some facts about it:
First of all, optimization is completely irrelevant for 99% of your code! That is the most important fact about optimization: The vast majority of any code is executed so seldomly, that there is virtually no payoff in optimizing it.
Thus, the most important skill you need to optimize your code, is to determine where to optimize. Most people use profilers for that.
However, the most important optimizations are usually on the algorithmic level. That is, if your basic code structure is bad, you can micro-optimize it all you want, it will never perform well, while an entirely crappy implementation using a more advantageous structure outperforms it anytime. And it is entirely up to you as a programmer, to find the better code structures, and to use them to your advantage. Your compiler can't do this, because it simply cannot see the big picture.
In my experience, the things that you need to look at on the algorithmic level are mainly:
Where do copies occur?
This was a common failure in C++ code until move construction came along. Still, it's better not to have to copy/move your data all the time. Especially, it needs to be avoided to incur quadratic copying costs, as you typically do when you concatenate strings in a loop.
Where do I use expensive operations like disk accesses/network communication?
You can perform a lot of computation while the system fetches a single block of data from a spinning disk. Thus, when you have operations with high costs, these are the ones that need to be optimized. Your compiler won't help you one bit with that. Other operations to look for: Locks, system calls, and memory allocations.
Can I represent my data in such a way that all the questions I have to ask about the data can be answered in a trivial, low-cost way?
This is the most tricky, effective, and fun thing to do. When you do it earnestly, you'll find that you can get the complexities of many problems down to a linear or even constant time complexity.
So, to answer your question more directly:
Should I write my code to be clear what I am doing and rely on the optimizer to clean up my code efficiency, or should I be obsessive about getting every last ounce of power out of my code?
The 99% of performance irrelevant code should definitely see no micro-optimization. It's entirely contraproductive.
And how much speed/size am I losing on by choosing one option over the other?
None in 99% of your code. It's the last percent that matters. My experience is, that micro-optimizations easily yield a speedup of 2x or more. But as I said, the power of micro-optimizations is limited: What is a 2x improvement when you can improve the complexity of your algorithm from O(N^2) to O(1)? (Still, there are cases where you should go for the O(N^2) algorithm because your problem sizes are too small!)
TL,DR: Don't even think about micro-optimization before you have done a thorough analysis!
Why is this important to you? Is anyone complaining that something is not fast enough? You have limited time to do things. Unless speed is really of concern, your priority should be correctness, and features.
But "should I rely on the optimizer to clean up my code efficiency" sounds very bad to me. An optimizer will do micro optimisations which saves you from the horrible task of doing micro optimisations yourself. It doesn't do magic. You should know what your tools and your libraries do, how efficient they are. Use the most efficient tool for the job, don't do unnecessary work. Most of the time this only requires knowledge and not any actual effort.
Your question gives us the choice between two unhealthy alternatives. The right thing is to learn your business, know what you are doing, and do things in a naturally efficient way. Most of the time that will be plenty efficient enough, and when it's not, you have a good base to start with, and time to improve it because you didn't waste it elsewhere. Writing rubbish code and relying on the optimizer to clean up is bad. Being "obsessive about getting out the last ounce of power" is equally bad.
Other answers are good. I can only add that
There are speedups that the compiler can do, so you should not bother with them, like reordering instructions or juggling registers to save memory fetches.
There are speedups that only you can do; the compiler cannot do them, like invisible I/O you never suspected, or too much memory management.
So what I do is two phases:
First, on a debug build, get the speedups that only I can do, like here. I do this on a debug build because it is essentially a debugging process, and it is very difficult to debug an optimized build.
Second, after having done all the speedups I can, turn on the compiler's optimizer, and let it do its magic.
This sounds pretty straightforward, but believe it or not, lots of people think you should only profile compiler-optimized programs. This is silly because it confuses your goals - finding speedups vs. measuring how fast it isn't.
