Syncing two D455 Cameras, one as master and one as slave unsuccessful
Cameras: two D455s
SDK and Viewer 2.40.0
PC New with Intel 7 CPU
Been unable to get cameras to operate in sync master/slave mode. I have tried various settings without success.
Based on the two white papers regarding synchronization, pin 5 is connected to pin 5 and pin 9 connected to pin 9. Ref: External Synchronization of Intel® RealSense™ Depth cameras being one of the two white papers. Used the jumper wire and 9-pin connector referenced in the white papers for testing.
Currently have in Viewer, under Stereo Module, Controls, Inter Cam Sync Mode is set to 1 for the Master and 2 for the Slave camera. Also under control, for the master, selected Output Trigger Enabled. Frame rates and size matched for the two cameras.
Streaming was turned on for RGB, Stereo, and IMU for both cameras. One camera is connected to a Thunderbolt 4 port and the other camera connected to a USB 3.2 port to prevent USB SCP overflow.
Writing to a fast external SSD via an USB 3.2 port. Have been successful in writing streams from two cameras to the SSD when not synchronized.
In Viewer, there is a Sync button next to the Record button. What is it for? How is it "undimmed" for usage? How is it used?
What is the correct Viewer operation sequence to have master/slave synchronization?
As a documentation note, in the two white papers for synchronization, the photo showing two cameras and the pin 5 and pin 9 connections with pin 9 to the right. In the subsequent wiring diagram, the orientation is flipped with pin 9 now on the left. This may initially confuse new users when making the camera connections.
Hi Jimmek In the link below, Dorodnic the RealSense SDK Manager explains that the greyed-out button sat the top of the RealSense Viewer's options side-panel relates to software sync and not hardware sync. For hardware sync, it should be sufficient just to set the Inter Cam Sync Mode for each camera.
Setting the Master to '1' is correct. Setting the Slave to '2' will sync depth. To sync the color camera too, the Slave should be set to '3', which is Full Slave mode. This was introduced in the External Synchronization (Genlock) paper.
You may also find the recent reference about sync cable assembly useful.
Thank you Marty for the reply.
I reviewed the referenced material and have a few more questions.
After everything is setup using viewer for one master(1) and one slave(3) where a sync cable connects the two cameras, and with streaming (RGB,depth,motion) turned on but not recording for both cameras, how does one start recording to the two bag files as there are two record buttons, one for each camera? Is there a specific order - slave then master or just master? I ask this as I have not yet been successful at recording two bag files that are synchronized.
I also will be trying Inter Cam Sync Mode 4 GenLock. Again, how is recording two bag files initiated?
I am also looking into external HW sync. The GenLock paper you referenced used an Arduino Micro with a level-shifter to bring the sync signal down to the required 1.8V. I am new to Arduino. Almost all my work has been with National Instruments LabWindows/CVI and their DAQ cards. Do you have access to the wiring diagram used in the setup as described in https://dev.intelrealsense.com/docs/external-synchronization-of-intel-realsense-depth-cameras on page4 including the manufacturer and part number for the level-shifter? I already have gotten the Arduino Micro. Of coures, if you have access to the software source code, would you let me have a copy? I do see that the Arduino Community website does have a library routine for a pulse train. Eventually, we are hoping to use a variable trigger driven by distance measurement, but first crawl, walk, jog, then run as we gain experience.
With external HW sync, I imagine if both cameras are in GenLock 4, the Start/Stop sequence would be Record on for both cameras, then both are idle (not recording) until the triggers are detected. That is Cam1 Record On, Cam2 Record On, Start Pulsetrain, End Pulsetrain, Cam1 Off, and Cam2 Off would be the proper sequence.
The RealSense Viewer does not have a function for stitching together the output of individual cameras. RealSense team member ev-mp offers advice in the link below about where to start in regard to creating a solution to address this.
I researched your Arduino Micro question carefully but was unable to find information about using it with RealSense other than the mention in the white paper. My understanding though is that a very simple signal generator such as the Arduino Micro is sufficient with Genlock and that it is not nearly as difficult as the pre-genlock era of the original hardware sync paper when the signal had to be tuned extremely precisely.
I do not have a wiring diagram or know the particular voltage level shifter featured in the paper, though a RealSense user who used a level shifter to convert to 1.8v stated that they used one similar to the SparkFun 12009 component mentioned in the guide below.
Progress Update and Trouble-Shooting Questions.
For an external trigger for HW sync, an alternative to an Arduino Micro referenced in "External Synchronization of Intel® RealSense™ Depth cameras", there are inexpensive Pulse Train generators that do not require any software programming such as the DROK ZK-PP1K. Pinouts on the device and instruction sheet did not match but was easy to figure out. Also used SparkFun BOB-12009 for the level shifter. Fortunately had access to required power and the two reference voltages for the level-shifter where output pulse peak is 1.8V. For testing, used 1Hz HW Sync rate.
Tried a single camera setup. For the External HW Sync, Inter Cam Sync Mode set to 4 in the Viewer in order to capture a frame every trigger. Frame rate in Viewer set to more than twice the HW Sync trigger rate. In the above referenced paper, it states "When the Genlock mode is selected, the camera will wait indefinitely for a trigger". After setting frame rates and size and Inter Cam Sync Mode 4 and then streaming turned on, the RGB camera streams at the frame rate set in the Viewer and the Depth Camera Receives no frames. RGB camera does not "... wait indefinitely for a trigger". Are other Viewer settings not properly set? Suggestions for troubleshooting?
In the non-Genlock slave modes such as '2', the Slave camera should wait for a certain period on each frame and then do an unsyched independent capture if it does not detect the trigger pulse within that time period.
On the Genlock mode ('4'), if the camera is waiting indefinitely for the trigger then it would make sense that if capture couldn't proceed until the trigger was detected then there would be No Frames Received (due to no unsynched capture being allowed).
I tested this myself with a D455 as the Master (1) and a D435i as the Slave (4). Like your experience, on the Slave camera I received no depth frames but did receive RGB.
For the next test, I set both D455 and D435i to Slave (4) to replicate a situation where the trigger signal is being provided by an external signal generator. In this test, the Depth stream of both cameras had No Frames Received (because I have no signal generator equipment to test with and so there is no trigger pulse for the Slave cameras to receive).
Reading the genlock white paper, it does not state that RGB will be synced (and thus wait for a trigger) when Inter Cam Sync Mode is 4 or greater. Instead, it just states that RGB will be synced when Inter Cam sync Mode = '3' (Full Slave mode). Sync behaves differently at 4+, where it is in Genlock Slave mode.
SW SDK Win10 ver 126.96.36.199.2657
FW ver 5.12.10
Cameras two D455
Still having a problem having one camera as Master (1) and another as a Full Slave (3) using Viewer.
Cable connection pin 5 to 5 (signal) and 9 to 9 (ground).
Except for Sync Mode, configuration is the same for both cameras.
- start all streams
- turn record on first for slave and then master one or two seconds later
- collect data
- turn record off first for master and then slave
Result: On slave 219 depth and color frames. On master 86 depth and color frames.
It appears that the cameras are not running synchronized.
First question: Am I missing something in the configuration?
Second question. Is the procedure provided above in terms of starting and stopping correct?
In attempting to eliminate a possible hardware problem with the cable connection / master camera pulse generation, I used another computer with a National Instruments DAQ card and the NI Measurement & Automation Explorer to monitor the camera connection for a pulse signal (trigger) and found no signal. The test cable I used had the 9 pin connector for the camera and corresponding short cables connected to the data acquisition card. I did this test individually for each camera set as Master (1). I also did a continuity test on the test cable which is not easy to do given the very small female pin size on the connector. Continuity test passed.
Third question: What should I do next?
Thank you for your continuing assistance.
I will be able to look at your new questions in detail tomorrow. Thanks for your patience!
I see that I already provided you with a link to cable wiring information from Intel to check against your own wiring design.
Where can a developer for syncing two D455 cameras in 1) master/slave mode (fixed FPS), and 2) both cameras in Mode 4 where there is an external pules train / trigger (variable FPS) as input be found? Do you personally know of some good developers that can meet the above needs?
Hi Jimmek Whilst researching your latest questions, I recalled that color sync for D455 in the External Synchronization system is a forthcoming feature that is being worked on and not enabled yet.
In regard to your question about whether your start-stop procedure is correct: in the original hardware sync system that had only modes 0 to 2, the Slave camera (2) would listen on each frame for a pulse trigger signal, such as one generated by the Master camera (1). If the Slave camera did not detect a trigger pulse within a certain time then it would stop listening and trigger itself without synchronization.
If that listening mechanism still holds true for mode 3 (Full Slave) then I would expect it not to matter which camera was enabled first, as if a camera was set as a slave instead of '0' (no sync) then it should capture unsynced until a time that it detects a trigger pulse. I am not certain on this principle regarding Full Slave though.
I would also expect it not to matter which camera had its recording activated first (if you are talking about using the Record button in the Viewer for each camera in the options side-panel to record to bag file).
I am not familiar enough with the External Synchronization system to answer your pulse train query. The RealSense support team member @... on this forum may be able to advise you.
The discussion in the link below is also worth reading regarding genlocking color with mode 4+
Hi @... Thanks very much for repying. As far as I know, the term pulse train is only found in a reference in Section 2 of the External Synchronization paper. I would speculate that it refers to a sequence of trigger pulses, like a chain of train carriages. I will quote the section below for your convenience.
When streaming in Genlock mode, the camera can be configured during initialization to any of the normal input resolutions and frame rates. Specifically, the “native” frame rate of the camera can be set to 6 fps, 15fps, 30fps, 60fps, or 90fps. This native frame rate has an impact on two significant aspects:
First, it does not set the trigger rate, but it does set an upper limit on the frame rate that can be supported by an external trigger. When sending a trigger pulse train, the trigger signal cannot be repeated faster than HALF the native camera mode frame rate, as shown in the table below. In other words, once a trigger is received the camera will be blind to any other trigger signals for 2x the frame time of the “native mode” of the camera.
It may be a reference to a statement earlier in the discussion. "For an external trigger for HW sync, an alternative to an Arduino Micro referenced in "External Synchronization of Intel® RealSense™ Depth cameras", there are inexpensive Pulse Train generators that do not require any software programming such as the DROK ZK-PP1K".
Jimmek later asks "Where can a developer for syncing two D455 cameras in 1) master/slave mode (fixed FPS), and 2) both cameras in Mode 4 where there is an external pules train / trigger (variable FPS) as input be found? Do you personally know of some good developers that can meet the above needs?"
I believe that Jimmek is seeking advice about setting up sync between two D455 cameras in a master-slave setup that use Genlock mode 4 and utilize an external pulse train instead of the Arduino Micro + level shifter suggested in the External Synchronization paper. It may be best for Jimmek to confirm whether this is what they need help with. Thanks again @...
Hello MartG and Garcia,
1. Using the Intel® RealSenseTM Depth cameras D4xx in Multi-Camera Configurations
2. External Synchronization of Intel® RealSense™ Depth cameras
First Setup Test: Run two D455 cameras with one in Master (Inter Sync Mode 1) and the other in Full Slave (Inter Sync Mode 3). Cable connection as described in Ref 1 above, pin 5 to pin 5 (signal) and pin 9 to 9 (ground) as described in Section 1. The sync signal can is from the master camera for master/slave operation or external source when all cameras are in Inter Sync Mode 4+. Currently testing this Master/Slave configuration with a very short sync cable. No need for capacitors or resistors. Using Viewer to configure cameras. In this setup, the Master camera supplies a sync signal to the Slave camera. Ref 2, Section 2, Figure 1 shows what the sync signal generally looks like.
Result: Camera outputs are not synchronized. Slave camera starts recording before master camera Record is activated and continues collecting after Master camera Record Off
Diagnostic Approach and Result:
Signal monitoring setup consists of National Instrument data acquisition HW and their Measurement & Exploration Explorer software configured as an Oscilloscope to monitor any signal over the sync cable. Sync cable connected only to the camera with bare connections at the other end connected to the NI signal monitoring setup. When camera run in master mode, all streams on, and Record On, no signal is observed.
- Should a signal be observed as depicted in Ref 2, Figure 1 with a 1.8V peak when the sync cable is attached only to one camera?
- Is the sync cable required to be attached to a Master and at least one Slave camera before the sync signal is generated/observed?
Second Setup to be tested: Run both cameras in sync mode 4 (GENLOCK) using external 1.8V trigger source as described in Ref 2. Will provide results soon.
Connection Note: The white 9 pin connector is very difficult to insert in camera as the pin 9 edge of the connector and the edge of the camera slot on the pin 9 side interfere which each other. It was necessary to slightly sand down the right side of the pin 9 edge of the connector to clear the camera opening in order to properly align and insert connector into camera. This has occurred on all three cameras I have in the lab.
We have made some progress and have not heard back from you. We would very much appreciate your help in confirming the following regarding the D455 cameras. Note that we have confirmed that the camera in master mode does output a 1.8V trigger signal. We also have an external 1.8V trigger source.
1. This camera is a 400 series camera. Does it have all the 400-series Inter Cam Sync mode options listed in your documentation and the two papers on Multi-Camera/External configurations.
2. Does this camera model have the master(1)/slave(2) and master(1)/full slave(3) mode options? Please confirm that Intel has tested these cameras in master-slave mode.
3. Does this camera model have an updated documentation for wiring and functionality and software?
4. Regarding single or multi-camera controls via the USB, do you have sample codes and compilation instructions that we could start from? In looking at some of the sample programs, it would appear that a d455.hpp file may be needed. Does it exist anywhere? Is there documentation for all the routines in the realsense library?
5. We have not found any external expertise with this camera. No response from www.codementor.io/freelance-developers/realsense (AKA Arc). If you know anyone that could provide consulting support for the issues we have been discussing, please connect us with them.
6. Please confirm that Intel has tested these cameras in external sync mode(4) with an external 1.8-volt trigger signal.
Hi Jimmek As some of your questions relate to product testing by Intel and I do not have that information, i will refer the above questions to @... again to ensure that you receive correct information. Thanks very much for your patience.
1) Mode 1(Master) and Mode 2(Slave) are officially validated so they are officially supported by customer support team, but Mode 3 or 4+ are not officially validated yet as of today, however they are working from engineering point of view. For Depth cam, all D400 series support "Mode 1" and "Mode 2", but D455 requires the latest firmware (FW5.12.10 now is available).
2) Sorry, I am not the right person supporting Mode 3 or Mode 4+, but I know they are working on D455 as I just checked with FW188.8.131.52. The WP "figure 3" shows how the sync cable looks like. I am using exactly same one. I took 1x D435 as Master cam for trigger source (Default mode "Mode 0" is fine, just don't forget to check the box of "Output Trigger Enabled". Or you can just simply use "Mode 1"). I set the D435 at 6FPS. Yes, it does output 1.8Vp-p pulse to the sync cable. And then I took 1x D455 (set at 30FPS) as Full Slave cam: either Mode 3 or Mode 4 to Mode 8 .... functionality wise, they worked.
I am not sure about other questions, probably need help from other people.
Thanks and good luck
- The current set of documentation you have found is the full set available for all cameras.
- It seems you have already seen all of the available documentation and sample code for multi-camera controls. Have you seen the multicam example? There is no d455.hpp file. Here is the librealsense API documentation.
Here is a list of service vendors that may be able to help you: https://www.intelrealsense.com/development-service-vendors/. Note that your use case is not common so you may find it difficult to find experienced technical consultants that can help you develop your solution.
Intel Customer Support
Could you please let me know if you could able to get the synchronous frames with master -full slave configuration (or) with external sync signal connected to two cameras connected in Genlock mode?
I am facing the similar problem now
I am working on the same thing. Trying to put two D455 cameras into HW sync.
I am using two individual USB3.2 ports for communication to the camera, and followed the instruction in the white paper to build the sync cable and connected the cameras accordingly. Also I am running the application on a very capable laptop with ubuntu18.04 and the latest SDK (2.45.0) and firmware (184.108.40.206) for D455.
In for all the tests, the depth resolution is set to 848 x 480.
I have verified that both cameras can be put into "inter_cam_sync_mode = 1" via realsense-viewer, ROS wrapper, and python api. Once in this mode, the camera will output a pulse signal with a frequency equals to the frame rate (fps) in pin 5 ("signal pin").
However, doing so won't cause any noticeable difference in the terms of the frame rate in the slave camera (with "inter_cam_sync_mode" set to 2) if the fps setting in both cameras are already the same (the difference is very small though, usually within 1 fps). However, according to this video: https://www.youtube.com/watch?v=DSm7NYG5bZE both camera's fps can be locked to 30.01 fps.
If I set the master camera's depth fps to 60 and slave to 30 fps, HW sync will force the slave to be running at 60 fps (but the depth image is kind of broken). If I do it the other way around, nothing will happen (they will stay in their own fps setting).
So my questions are:
- Are these the expected behaviors? Meaning it is normal that HW sync can only regulate the firing of cameras within ~1fps? (Due to differences in lighting conditions maybe? If so, it really doesn't make a difference w/o HW sync since the fps tracking can already achieve that precision in the default mode)
- Also I noticed that when connecting only one camera, the fps of one of my D455 is more stable than the other. The unstable one jumps +/-2 fps while the stable stays at 29.99 fps when set to 30 fps. Should I file a return / exchange for that unstable camera?
- What is the purpose of the GPIO pin (pin 9)?
- Should I also connect the ground pin as suggested in https://vcl3d.github.io/VolumetricCapture/docs/hardware/rs2_hardware/ ?
- What is the latest development on inter_cam_sync_mode >2 ? Can I use them on D455?
Please sign in to leave a comment.