a) How worried should I be about this in general, does this cause problems at higher frequencies (5 MHz for example)? What if this period is longer, say 20 or 50ns?
As always, it depends. Some things to watch out for:
- Thermal issues. You're drawing a burst of current during the transition, which can overheat components.
- Gross overload failures. Output impedance is too high for many logic circuits to really cause this sort of failure - but can be a problem in some cases.
- Power supply droops. Again, you're drawing a burst of current during the transition. If you don't have proper decoupling (not just 'how much' but also 'how much impedance' and 'how well-placed are said capacitors'), this can cause problems.
- Interference. Transient shorts can cause 'amusements' with RF noise, as well as general power supply noise being coupled into other components.
- Current draw. More of a problem with batteries, but still worth noting.
- Longer-term failures due to e.g. electromigration. More of an issue with e.g. digital IO of microcontrollers than it is simple logic chips, but can still be a concern.
Shorter bus contention is generally - not always - less of an issue than longer bus contention. Among other things, there is always a certain amount of inductance present, which helps limit max current for a bit.
In this case:
Absolute maximum output current is given as 35mA. I'd be concerned about this. Especially given this chip has a specified minimum input rise/fall rate (indicating that internal output shoot-through is likely a potential problem - which doesn't bode well for external shoot-through either) (annoyingly, this is specified as a maximum time/input voltage change.), and doesn't specify an output voltage-versus-current curve beyond 6mA.
Personally? I wouldn't use this design unless either:
- Failures aren't a big deal, including failure modes like "the chip shorts VCC to an input signal" and "the chip catches fire" (The latter is unlikely! But not impossible.) (What is the chip driving? Are there potential problems if the output signal ends up floating for an extended period at a forbidden logic level? Etc.)
- I added enough resistance to the output to limit current to below spec'd ratings even when shorted.
- I measured and made sure that the transient wasn't hitting above spec'd limits on the actual specific parts being used. (Fine for a one-off, not so fine if I'm making a hundred units.)
- I got something from the manufacturer saying it's fine. (It may be worth asking! Worse comes to worse they ignore you.)
- I was using a part that specified an output voltage-versus-current curve all the way to half VCC (and the resulting values were within spec).
- I adopted something like the below:
b) If this is an issue, how is it usually handled? Do I place buffer gates to try best match the propagation delays? Or should I place resistors at all outputs of every driver gate in my schematic?
If this is an issue, one approach may be a break-before-make design:

simulate this circuit – Schematic created using CircuitLab
(Adjust numbers as appropriate.)
(Given that your OEs are inverted, you probably want to swap this around to a make-before-break with an OR and NAND. Basic idea is to ensure that you never have both enabled at once.)
Be aware that both gates here should be schmitt-type. Be aware that this will result in the outputs potentially floating for a moment.
(Depending on what said signal is being driven with, you may be able to simply drive the two inputs from e.g. two IO pins instead. Do take care with this approach, as it now means something misdriving the OE signals can cause hardware failure.)