Whether you say that a signal is inverted, or phase shifted by 180°, depends on context.
In the context of frequency response, where we are interested in the output of a system in respsonse to an input sinusoid at any given frequency, the concept of inversion wouldn't be used to describe behaviour. Take this filter for example:
simulate this circuit – Schematic created using CircuitLab
If you were to input a sinusoidal signal of frequency 0.1Hz to this system, the output would be another sinusoid of almost exactly the same phase, but at 1kHz the output would appear "inverted" with respect to the original. We wouldn't say it's "inverted", though, since at frequencies in between, the phase relationship between an input sinusoid and the output sinusoid would fall somewhere between those extremes of 0° and 180°, with a somewhat smooth transition as frequency changes.
It's not clear whether that filter inverts on not, it depends on frequency. Furthermore, for signals that are not sinusoidal, like square or sawtooth waveforms, which consist of many sinusoids in superposition (harmonics), some of them will be phase shifted more than others, and the output may not resemble the input at all. From this perspective, "inversion" is not a meaningful description of behaviour of such a system, which could alter wave shape in much more varied ways than "flipping upside down".
In this context, the concept of phase is very apposite, but the concept of inversion is not.
But, there exist systems that only "flip upside down", such as the common-emitter transistor circuit you mentioned, or this one:
simulate this circuit
For these circuits, whatever frequency of sinusoid you apply at the input, the output will be another sinusoid exactly 180° phase shifted from the input. In other words, any sinusoid at any frequency (below the bandwidth of the op-amp) is "inverted".
Importantly, the output is a "flipped upside down" copy of any input waveform, be it sinusoidal, square, sawtooth, or even aperiodic. In that sense, this amplifier can truly be said to "invert", because it does not discriminate between frequencies or shapes. The term "inverting" would be a very appropriate description of its behaviour under all input conditions.
You would not be wrong to say it shifts phase by 180°, but that description wouldn't be telling the whole story.
One last example might help. Let's say you have a rectangular PWM waveform, and a copy of that waveform delayed by a half-cycle:
Those waveforms are clearly not inverted with respect to each other, and yet they could be described as being 180° out of phase with each other. The harmonics that comprise this waveform will all be phase-shifted by some amount other than 180°, though. Whatever circuit that took the orange wave, and produced the blue wave in response, cannot be called inverting, and yet it does produce a delay of 500μs, which some might call a phase shift of 180°. I personally never would. I prefer the term "500μs delay", since it avoids the implication that all harmonics are equally phase shifted.
Your common emitter transistor amplifier inverts, and produces 180° phase shift for all harmonics of any input waveform, so in this sense it is not misleading to use either description, though the term "inversion" implies both anyway.
I recommend you use the term "invert" strictly in the sense of "flipping a waveform upside-down", and the term "phase" strictly in the context of sinusoids. There are always exceptions, though, and as long as people understand what you mean, I suppose you can use them interchangeably in certain circumstances.
