I'd give two answers : one general to the title, one with your specific case :
Answer to the title
By default, I'd say yes it is a bad practice.
However it doesn't mean that there is absolutely no single reason where a multi-class (so not a fully global) factory could be that bad.
Let's say that your factory isn't global but limited to everything that can move on the map. This factory is multi class but yet not exactly global.
You could for instance need to initialize all those classes with some common parameters and instead of bringing along all those parameters across all functions, having them all in your factory and pass them down to the instance of classes. It is basically what a DI container is doing.
The trap here is that there is already stuff doing this : DI container so why repeat yourself ? The basic usage of DI container is often to create "Singleton" Service class (Singlton between quote because you will instantiate only one thanks to the DI container but you're not using the Singleton pattern, making testing easier).
But DI container can also handle the creation of objects on demand, this might have a cost on performance if you need really a lof of instances. And this is where your factory might be usefull : create your factory from the DI container in order to have everything you need. Use your factory to create everything you need.
Answer to the content
I'am not sure that what you are asking is only a simple multi-class factory (I think your question is what we call an XY problem here). To me it seems like the goals of this class is to be used to initialize everything in your game. For instance let's suppose we're going for chess, you will need to initialise 8 pawns, 1 queen, 1 king, .... You could have your global factory and one other class performing the initialization through calling your factory.
But I do not think this would be the proper distribution of responsabilities. I think that one class in charge of fully initialize the game by instantiating everything needed would be more appropriate for chess.