All PCIe buses must have a "Root Complex" (RC) which is what configures and enumerates all devices. Whether that is a desktop PC, and FPGA, or some other SoC device is entirely up to you as long as it provides the required functionality.
You can get many SoC devices, including ARM SoC/FPGA hybrids which will allow you to provide RC functionality in a compact form. The thing to remember though is that once you have enumerated the bus - once the RC has detected all cards, assigned addresses, etc. - you still need to be able to interface with whatever device you are using. That typically means a computer running an OS that has the required drivers for the device.
Many of these SoC devices can run fully fledged Linux installations which would, assuming there are Linux drivers for your device, be able to host the PCIe bus and control all of the devices - interface with compute modules, GPUs, etc.
Of course you don't always need an OS running to use PCIe. PCIe is just a generic communication protocol. Once you have enumerated the bus with your RC, how you communicate between two devices is entirely up to you if you are designing the devices yourself. For example you could set up two or more FPGA devices on a backplane with one as the RC and others as endpoints. You can then program your FPGA however you like.
If configured with bus master capability during enumeration they can talk to other devices, transfer data, send interrupts, anything they like. But more importantly, they can do it all with complete autonomy - no need for a computer or operating system to instruct them. The only requirement is that there is some defined structure for how this is done - the other cards have to understand what is being said to them and know how to react. You can build systems of FPGAs talking to each other and transferring data using pure logic circuitry instead of an OS.
This is not just limited to FPGAs. You can include GPUs and other devices in your bus. However in order to do so, you need to know how to communicate low-level with the device. You need to know what registers do what, what sequences of commands to send, and so forth. For a device like an FPGA that you have programmed yourself, that is fine, but for things like compute modules and GPUs, this low level information is not readily available, and you wouldn't have the luxury of using the drivers that the device maker have written for mainstream OSes.
