1

I need to synchronize 4 multiple MIPI CSI Cameras. I'm using the following MCU https://www.nxp.com/products/processors-and-microcontrollers/arm-microcontrollers/i-mx-rt-crossover-mcus/i-mx-rt1050-crossover-mcu-with-arm-cortex-m7-core:i.MX-RT1050

I would like to know the proper way of connecting multiple CSI Cameras and synchronizing between them.

The idea also is to use an IMX8 processor as Main CPU and use the imxrt01050 as slaves that are connected to the cameras.

Which setup is possible with correct synchronization between all of them ? Kindly note I plan to use 1080p resolution at 30FPS.

John
  • 13
  • 4

1 Answers1

2

Seeing you need synchronization only on a frame basis, this should be rather straightforward: nothing in your microcontroller needed to initiate the camera giving images takes in the order of 1/60 s.

So, you'd just write firmware that takes some start signal (for example, simply a GPIO input going high) and tells the camera via I²C to start sending data. If you share the same trigger with all microcontrollers, this should be rather straightforward, and the timing insecurity, especially if the triggering happens in an interrupt, should be far below 1/FPS.

Regarding how to connect 8 of these cameras to one application processor: well, tough one, because this is a lot of data.

The 100 Mb/s ethernet integrated in your MCU is not fast enough for uncompressed full HD at 30 FPS (30 F/s * 1092*1080 px * 12 bit/px is about 283 Mb/s). So, that can't work.

The only other fast interface is the USB2 High-Speed interface, which, with 480 Mb/s is nominally fast enough for 12 bit per pixel, but you'll find that it's not trivial to write USB firmware fast enough. So, you'll need to write a firmware that deals with camera data, and makes sure it is DMA'ed into the USB peripheral buffer. Of course, you'll have to implement the USB stack. NXP does have an app note on how to implement a USB Webcam style device with your MCU, but it does 640x480 at 20 fps, not six times that resolution at 1.5 times that frame rate.

Then, you will have to aggregate all that data over USB at your central processor. I doubt you've made a good choice there – you will need, bandwidth-wise, a high-speed USB controller per camera, and your central processor doesn't have 8 of these.

So, all in all: I think both your MCU and your central controller are the wrong tools here.

Realistically, you'll want an FPGA on which you can instantiate 8 MIPI CSI interfaces, and then have one link to your central controller CPU; probably via multiple PCIe lanes. That'd solve the USB bottleneck, but it does not solve the fact that this is much data, and it's very questionable your cute little quad-a72 core processor (since your workload is fully identical 8 times, the littleBig little-cores cores don't help that much) will be able to deal with 8 camera streams. Don't forget that having one or two video processing chains does not make dealing with 8 video streams simple, by any means.

Marcus Müller
  • 88,280
  • 5
  • 131
  • 237
  • There are only 4 cameras not 8, maybe that assumption make things little easier ? – John Jul 11 '21 at 13:49
  • not really. The interfacing problem remains exactly the same. This is simply very much data. – Marcus Müller Jul 11 '21 at 13:50
  • Can you suggest an alternative CPU same from NXP to achieve that ? Thanks so much! a great analysis – John Jul 11 '21 at 13:51
  • an FPGA solution is out of the choice, as we don't have an FPGA guy or solution that does it – John Jul 11 '21 at 13:53
  • well, you don't have a choice, sorry. You simply can't do full HD at 30 fps and bit depths > 12 bit with a device that can't even use the full USB2.0 High speed bandwidth. You need a fast interface to your central CPU, no matter what you do. sometimes the architecture you've made in your head doesn't fit your system requirements - this is the case here. Have you even tested whether your stereoscopic application would run in real time on your i.MX8 when the data is already in RAM? if not, you should do that- – Marcus Müller Jul 11 '21 at 13:55
  • Can't I use the MIPI CSI interface of the main CPU by any means ? – John Jul 11 '21 at 13:56
  • the main CPU has 2 interfaces. you need 4. end of story, really. – Marcus Müller Jul 11 '21 at 13:57
  • Lowering the resolution would help :) ? – John Jul 11 '21 at 13:58
  • @andreahmed this is going nowhere. You haven't really even finished the specs of your system, you just thought "oh, these would be nice components": you're wasting time. Come up with definite specs first, then describe what every component of your system needs to fulfill to achieve these, **then** start looking for components. – Marcus Müller Jul 11 '21 at 14:03
  • You are right. But I'm thinking about connecting the 4 cameras directly to the Main CPU.. does ram requirements are enough ? – John Jul 11 '21 at 14:18
  • the main cpu has 5 I2C so is it possible ? – John Jul 11 '21 at 14:20
  • it has two CSI ports. CSI is not I2C.... I²C is practically never the problem, that is only used for (relatively low-speed) control. – Marcus Müller Jul 11 '21 at 16:00
  • Can you elaborate how this works ? https://www.e-consystems.com/three-synchronized-4k-cameras-for-nvidia-jetson-tx2.asp#key-features – John Jul 11 '21 at 16:02
  • using completely different hardware than you plan to. In fact, they claim they're using some ASIC for image processing, not some microcontroller. Again, we're wasting time here: you haven't speced out your system, yet already know what kind of hardware you plan to use. That's not how this works. Stop looking at hardware solutions. Write down clearly what you need to do. Then design a system architecture, write down the specs for the individual components in that, AND ONLY AFTER THAT'S DONE, start looking for components that do that. – Marcus Müller Jul 11 '21 at 16:03