Unable to capture from IMX477

[acehoss]

Hello! I am running an Orin NX 16 in the ARK PAB Jetson Carrier. I built and flashed using the instructions from the main branch of this repo.

I followed your example and activated the "Camera ARK IMX477 Single" on header 2 and rebooted. After the reboot, the camera probes and there is a video node at /dev/video0.

jetson@jetson:~$ sudo dmesg | grep imx
[    9.060473] imx477 9-001a: tegracam sensor driver:imx477_v2.0.6
[    9.361615] tegra-camrtc-capture-vi tegra-capture-vi: subdev imx477 9-001a bound
jetson@jetson:~$ ls /dev/video*
/dev/video0

When I try to run a capture, I get an error:

jetson@jetson:~$ gst-launch-1.0 nvarguscamerasrc num-buffers=300 ! nvvidconv ! x264enc ! qtmux ! filesink location=test.mp4
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:792 No cameras available
Redistribute latency...
Redistribute latency...
Got EOS from element "pipeline0".
Execution ended after 0:00:00.018264483
Setting pipeline to NULL ...
Freeing pipeline ...

No messages are printed to dmesg. There is a little in the journal:

Nov 22 12:21:14 jetson nvargus-daemon[864]: === NVIDIA Libargus Camera Service (0.99.33)=== Listening for connections...=== gst-launch-1.0[3244]: Connection established (FFFFAC72B840)OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module0
Nov 22 12:21:14 jetson nvargus-daemon[864]: OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module1
Nov 22 12:21:14 jetson nvargus-daemon[864]: OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module2
Nov 22 12:21:14 jetson nvargus-daemon[864]: OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module3
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclHwGetModuleList: No module data found
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclHwGetModuleList: No module data found
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclHwGetModuleList: No module data found
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclHwGetModuleList: No module data found
Nov 22 12:21:14 jetson nvargus-daemon[864]: OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
Nov 22 12:21:14 jetson nvargus-daemon[864]: ---- imager: No override file found. ----
Nov 22 12:21:14 jetson nvargus-daemon[864]: (NvCamV4l2) Error ModuleNotPresent: V4L2Device not available (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function findDevice(), line 256)
Nov 22 12:21:14 jetson nvargus-daemon[864]: (NvCamV4l2) Error ModuleNotPresent:  (propagating from /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function initialize(), line 60)
Nov 22 12:21:14 jetson nvargus-daemon[864]: (NvOdmDevice) Error ModuleNotPresent:  (propagating from dvs/git/dirty/git-master_linux/camera-partner/imager/src/devices/V4L2SensorViCsi.cpp, function initialize(), line 111)
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclDriverInitializeData: Unable to initialize driver v4l2_sensor
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclInitializeDrivers: error: Failed to init camera sub module v4l2_sensor
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclStartPlatformDrivers: Failed to start module drivers
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclDriver_V4L2_Focuser_Stub_Close: Invalid NULL input pPclDriver
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclStateControllerOpen: Failed ImagerGUID 1. (error 0xA000E)
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclOpen: PCL Open Failed. Error: 0xf
Nov 22 12:21:14 jetson nvargus-daemon[864]: SCF: Error BadParameter: Sensor could not be opened. (in src/services/capture/CaptureServiceDeviceSensor.cpp, function getSourceFromGuid(), line 725)
Nov 22 12:21:14 jetson nvargus-daemon[864]: SCF: Error BadParameter:  (propagating from src/services/capture/CaptureService.cpp, function addSourceByGuid(), line 453)
Nov 22 12:21:14 jetson nvargus-daemon[864]: SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function addSourceByIndex(), line 347)
Nov 22 12:21:14 jetson nvargus-daemon[864]: SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function getSource(), line 519)
Nov 22 12:21:14 jetson nvargus-daemon[864]: ---- imager: No override file found. ----
Nov 22 12:21:14 jetson nvargus-daemon[864]: (NvCamV4l2) Error ModuleNotPresent: V4L2Device not available (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function findDevice(), line 256)
Nov 22 12:21:14 jetson nvargus-daemon[864]: (NvCamV4l2) Error ModuleNotPresent:  (propagating from /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function initialize(), line 60)
Nov 22 12:21:14 jetson nvargus-daemon[864]: (NvOdmDevice) Error ModuleNotPresent:  (propagating from dvs/git/dirty/git-master_linux/camera-partner/imager/src/devices/V4L2SensorViCsi.cpp, function initialize(), line 111)
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclDriverInitializeData: Unable to initialize driver v4l2_sensor
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclInitializeDrivers: error: Failed to init camera sub module v4l2_sensor
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclStartPlatformDrivers: Failed to start module drivers
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclDriver_V4L2_Focuser_Stub_Close: Invalid NULL input pPclDriver
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclStateControllerOpen: Failed ImagerGUID 4. (error 0xA000E)
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclOpen: PCL Open Failed. Error: 0xf
Nov 22 12:21:14 jetson nvargus-daemon[864]: SCF: Error BadParameter: Sensor could not be opened. (in src/services/capture/CaptureServiceDeviceSensor.cpp, function getSourceFromGuid(), line 725)
Nov 22 12:21:14 jetson nvargus-daemon[864]: SCF: Error BadParameter:  (propagating from src/services/capture/CaptureService.cpp, function addSourceByGuid(), line 453)
Nov 22 12:21:14 jetson nvargus-daemon[864]: SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function addSourceByIndex(), line 347)
Nov 22 12:21:14 jetson nvargus-daemon[864]: SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function getSource(), line 519)
Nov 22 12:21:14 jetson nvargus-daemon[864]: ---- imager: No override file found. ----
Nov 22 12:21:14 jetson nvargus-daemon[864]: (NvCamV4l2) Error ModuleNotPresent: V4L2Device not available (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function findDevice(), line 256)
Nov 22 12:21:14 jetson nvargus-daemon[864]: (NvCamV4l2) Error ModuleNotPresent:  (propagating from /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function initialize(), line 60)
Nov 22 12:21:14 jetson nvargus-daemon[864]: (NvOdmDevice) Error ModuleNotPresent:  (propagating from dvs/git/dirty/git-master_linux/camera-partner/imager/src/devices/V4L2SensorViCsi.cpp, function initialize(), line 111)
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclDriverInitializeData: Unable to initialize driver v4l2_sensor
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclInitializeDrivers: error: Failed to init camera sub module v4l2_sensor
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclStartPlatformDrivers: Failed to start module drivers
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclDriver_V4L2_Focuser_Stub_Close: Invalid NULL input pPclDriver
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclStateControllerOpen: Failed ImagerGUID 1. (error 0xA000E)
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclOpen: PCL Open Failed. Error: 0xf
Nov 22 12:21:14 jetson nvargus-daemon[864]: SCF: Error BadParameter: Sensor could not be opened. (in src/services/capture/CaptureServiceDeviceSensor.cpp, function getSourceFromGuid(), line 725)
Nov 22 12:21:14 jetson nvargus-daemon[864]: SCF: Error BadParameter:  (propagating from src/services/capture/CaptureService.cpp, function addSourceByGuid(), line 453)
Nov 22 12:21:14 jetson nvargus-daemon[864]: SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function addSourceByIndex(), line 347)
Nov 22 12:21:14 jetson nvargus-daemon[864]: SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function getSource(), line 519)
Nov 22 12:21:14 jetson nvargus-daemon[864]: ---- imager: No override file found. ----
Nov 22 12:21:14 jetson nvargus-daemon[864]: (NvCamV4l2) Error ModuleNotPresent: V4L2Device not available (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function findDevice(), line 256)
Nov 22 12:21:14 jetson nvargus-daemon[864]: (NvCamV4l2) Error ModuleNotPresent:  (propagating from /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function initialize(), line 60)
Nov 22 12:21:14 jetson nvargus-daemon[864]: (NvOdmDevice) Error ModuleNotPresent:  (propagating from dvs/git/dirty/git-master_linux/camera-partner/imager/src/devices/V4L2SensorViCsi.cpp, function initialize(), line 111)
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclDriverInitializeData: Unable to initialize driver v4l2_sensor
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclInitializeDrivers: error: Failed to init camera sub module v4l2_sensor
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclStartPlatformDrivers: Failed to start module drivers
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclDriver_V4L2_Focuser_Stub_Close: Invalid NULL input pPclDriver
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclStateControllerOpen: Failed ImagerGUID 2. (error 0xA000E)
Nov 22 12:21:14 jetson nvargus-daemon[864]: NvPclOpen: PCL Open Failed. Error: 0xf
Nov 22 12:21:14 jetson nvargus-daemon[864]: SCF: Error BadParameter: Sensor could not be opened. (in src/services/capture/CaptureServiceDeviceSensor.cpp, function getSourceFromGuid(), line 725)
Nov 22 12:21:14 jetson nvargus-daemon[864]: SCF: Error BadParameter:  (propagating from src/services/capture/CaptureService.cpp, function addSourceByGuid(), line 453)
Nov 22 12:21:14 jetson nvargus-daemon[864]: SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function addSourceByIndex(), line 347)
Nov 22 12:21:14 jetson nvargus-daemon[864]: SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function getSource(), line 519)

I also tried using nvv4l2camerasrc and that acted differently but still didn't work. The pipeline would start but not actually start rolling. After a couple Ctrl-Cs gstreamer terminated and the test.mp4 file was present but 0 bytes.

jetson@jetson:~$ gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! nvvidconv ! x264enc ! qtmux ! filesink location=test.mp4
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:46.977499191
Setting pipeline to NULL ...
^C
jetson@jetson:~$ ls -l test.mp4
-rw-rw-r-- 1 jetson jetson 0 Nov 22 12:34 test.mp4

Without the camera plugged in, /dev/video0 doesn't appear, so I know it is being detected. What else can I try? I have tried two different cameras and two different cables.

Note: I also tried the IMX219 Quad config with 4 of the regular RPi cameras and this does work for me:

jetson@jetson:~$ sudo dmesg | grep imx
[sudo] password for jetson: 
[    7.695229] imx219 9-0010: tegracam sensor driver:imx219_v2.0.6
[    7.711833] tegra-camrtc-capture-vi tegra-capture-vi: subdev imx219 9-0010 bound
[    7.739500] imx219 10-0010: tegracam sensor driver:imx219_v2.0.6
[    7.755820] tegra-camrtc-capture-vi tegra-capture-vi: subdev imx219 10-0010 bound
[    7.790975] imx219 11-0010: tegracam sensor driver:imx219_v2.0.6
[    7.807965] tegra-camrtc-capture-vi tegra-capture-vi: subdev imx219 11-0010 bound
[    7.835703] imx219 12-0010: tegracam sensor driver:imx219_v2.0.6
[    7.851983] tegra-camrtc-capture-vi tegra-capture-vi: subdev imx219 12-0010 bound
jetson@jetson:~$ ls /dev/video*
/dev/video0  /dev/video1  /dev/video2  /dev/video3
jetson@jetson:~$ gst-launch-1.0 nvarguscamerasrc ! nvvidconv ! x264enc ! qtmux ! filesink location=test.mp4
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3280 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3280 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 2 
   Output Stream W = 1920 H = 1080 
   seconds to Run    = 0 
   Frame Rate = 29.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
Redistribute latency...
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:07.572249504
Setting pipeline to NULL ...
GST_ARGUS: Cleaning up
CONSUMER: Done Success
GST_ARGUS: Done Success
Freeing pipeline ...

Thanks!

[acehoss]

Some additional info: V4L2 seems to be able to stream from the camera to some extent:

jetson@jetson:/proc/device-tree$ v4l2-ctl --device=/dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Type: Video Capture

	[0]: 'RG10' (10-bit Bayer RGRG/GBGB)
		Size: Discrete 3840x2160
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1920x1080
			Interval: Discrete 0.017s (60.000 fps)

jetson@jetson:/proc/device-tree$ v4l2-ctl --device=/dev/video0 --stream-mmap --stream-count=100
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
<<<<<<<<

However it doesn't seem to actually be able to stream video:

jetson@jetson:~$ gst-launch-1.0 v4l2src device=/dev/video0 ! xvimagesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:00.000299144
Setting pipeline to NULL ...
Freeing pipeline ...

[acehoss]

I got it working. Evidently there was a bug in r36.3 that causes issues with the mapping between sensor position and sensor id. There are a couple potentially pertinent issues fixed in the 36.4 release notes:

4385287 When only one IMX219 is connected to Jetson Orin NX 16GB/8GB, and a dual IMX219
overlay is applied, the preview and capture with argus_samples and argus_camera fails.
To work around this issue, connect the dual IMX219 cameras when applying dual overlay
support.

4327644 When an IMX219 or an IMX477 camera is connected to Jetson Orin NX and Jetson Orin
Nano devices through the CAM0 interface, to correctly complete a camera recording, users
need to explicitly specify sensor-id=1 instead of the default sensor-id=0 in the
gstreamer nvarguscamera element.

Due to these bugs, the camera needs to be connected to the second camera port (J27), but will be accessed through sensor id 0.

Below is thetegra234-p3767-camera-p3768-ark-imx477-single.dts that I used to get this working. I haven't taken the time to strip this down to the essential elements; it's the end result of a day of hacking at just getting it working.

// SPDX-License-Identifier: GPL-2.0-only
// SPDX-FileCopyrightText: Copyright (c) 2023-2024, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

/dts-v1/;
/plugin/;

#define CAM0_PWDN	TEGRA234_MAIN_GPIO(H, 6)
#define CAM1_PWDN	TEGRA234_MAIN_GPIO(AC, 0)
#define CAM2_PWDN	TEGRA234_MAIN_GPIO(Y, 4)
#define CAM3_PWDN	TEGRA234_AON_GPIO(CC, 3)
#define CAMERA_I2C_MUX_BUS(x) (0x1E + x)

#include <dt-bindings/tegra234-p3767-0000-common.h>

/ {
	overlay-name = "Camera ARK IMX477 Single";
	jetson-header-name = "Jetson 24pin CSI Connector";
	compatible = JETSON_COMPATIBLE_P3768;

	fragment@0 {
		target-path = "/";
		__overlay__ {
			tegra-capture-vi  {
				num-channels = <1>;
				ports {
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						reg = <0>;
						ark_imx477_vi_in0: endpoint {
							port-index = <1>;
							bus-width = <2>;
							remote-endpoint = <&ark_imx477_csi_out0>;
						};
					};
				};
			};
			tegra-camera-platform {
				compatible = "nvidia, tegra-camera-platform";
				/**
				* Physical settings to calculate max ISO BW
				*
				* num_csi_lanes = <>;
				* Total number of CSI lanes when all cameras are active
				*
				* max_lane_speed = <>;
				* Max lane speed in Kbit/s
				*
				* min_bits_per_pixel = <>;
				* Min bits per pixel
				*
				* vi_peak_byte_per_pixel = <>;
				* Max byte per pixel for the VI ISO case
				*
				* vi_bw_margin_pct = <>;
				* Vi bandwidth margin in percentage
				*
				* max_pixel_rate = <>;
				* Max pixel rate in Kpixel/s for the ISP ISO case
				*
				* isp_peak_byte_per_pixel = <>;
				* Max byte per pixel for the ISP ISO case
				*
				* isp_bw_margin_pct = <>;
				* Isp bandwidth margin in percentage
				*/
				num_csi_lanes = <2>;
				max_lane_speed = <1500000>;
				min_bits_per_pixel = <10>;
				vi_peak_byte_per_pixel = <2>;
				vi_bw_margin_pct = <25>;
				max_pixel_rate = <600000>;
				isp_peak_byte_per_pixel = <5>;
				isp_bw_margin_pct = <25>;
				/**
				 * The general guideline for naming badge_info contains 3 parts, and is as follows,
				 * The first part is the camera_board_id for the module; if the module is in a FFD
				 * platform, then use the platform name for this part.
				 * The second part contains the position of the module, ex. "rear" or "front".
				 * The third part contains the last 6 characters of a part number which is found
				 * in the module's specsheet from the vendor.
				 */
				modules {
					module0 {

					};
					module1 {
						badge = "jakku_bottomright_RBPCV3";
						position = "bottomright";
						orientation = "1";
						drivernode0 {
							pcl_id = "v4l2_sensor";
							sysfs-device-tree = "/sys/firmware/devicetree/base/bus@0/i2c@3180000/tca9546@70/i2c@1/ark_imx477_b@1a";

						};
						// drivernode1 {
						// 		pcl_id = "v4l2_lens";
						// 		sysfs-device-tree = "/sys/firmware/devicetree/base/bus@0/lens_imx219@RBPCV2/";
						// };
					};
					module2 {

					};
					module3 {

					};
				};
			};
			bus@0 {
				host1x@13e00000 {
					nvcsi@15a00000 {
						num-channels = <1>;
						#address-cells = <1>;
						#size-cells = <0>;
						channel@0 {
							reg = <0>;
							ports {
								#address-cells = <1>;
								#size-cells = <0>;
								port@0 {
									reg = <0>;
									ark_imx477_csi_in0: endpoint@0 {
										port-index = <1>;
										bus-width = <2>;
										remote-endpoint = <&ark_imx477_out0>;
									};
								};
								port@1 {
									reg = <1>;
									ark_imx477_csi_out0: endpoint@1 {
										remote-endpoint = <&ark_imx477_vi_in0>;
									};
								};
							};
						};
					};
				};
				// lens_imx219@RBPCV2 {
				// 		min_focus_distance = "0.0";
				// 		hyper_focal = "0.0";
				// 		focal_length = "3.04";
				// 		f_number = "2.0";
				// 	aperture = "0.0";
				// };
				i2c@3180000 {
					tca9546@70 {
						status = "okay";
						compatible = "nxp,pca9546";
						#address-cells = <1>;
						#size-cells = <0>;
						reg = <0x70>;
						i2c-parent = <&cam_i2c>;
						skip_mux_detect = "yes";
						force_bus_start = <CAMERA_I2C_MUX_BUS(1)>;
						i2c@0 {
							reg = <0>;
							#address-cells = <1>;
							#size-cells = <0>;
							i2c-mux,deselect-on-exit;
							status = "okay";

							ark_imx219_a@10 {
								status = "disabled";
							};
						};
						i2c@1 {
							reg = <1>;
							#address-cells = <1>;
							#size-cells = <0>;
							i2c-mux,deselect-on-exit;
							status = "okay";

							ark_imx219_b@10 {
								status = "disabled";
							};

							ark_imx477_b@1a {
								status = "okay";
								reset-gpios = <&gpio CAM1_PWDN GPIO_ACTIVE_HIGH>;
								compatible = "ridgerun,imx477";
								/* I2C device address */
								reg = <0x1a>;
								/* V4L2 device node location */
								devnode = "video0";
								/* Physical dimensions of sensor */
								physical_w = "3.680";
								physical_h = "2.760";
								sensor_model = "imx477";
								use_sensor_mode_id = "true";
								/**
								* ==== Modes ====
								* A modeX node is required to support v4l2 driver
								* implementation with NVIDIA camera software stack
								*
								* == Signal properties ==
								*
								* phy_mode = "";
								* PHY mode used by the MIPI lanes for this device
								*
								* tegra_sinterface = "";
								* CSI Serial interface connected to tegra
								* Incase of virtual HW devices, use virtual
								* For SW emulated devices, use host
								*
								* pix_clk_hz = "";
								* Sensor pixel clock used for calculations like exposure and framerate
								*
								* readout_orientation = "0";
								* Based on camera module orientation.
								* Only change readout_orientation if you specifically
								* Program a different readout order for this mode
								*
								* lane_polarity
								* Based on the camera connector pin.
								* CSIx_D0 | CSIx_D1 | CSI(X+1)_D0 | CSI(X+1)CSIx_D1
								*    LSB  |   BIT1  |     BIT2    |      MSB
								* if there is a polarity swap on any lane, the bit corrsponding
								* to the lane should be set
								* e.g. polarity swap on CSIx_D0 only -> lane_polarity = "1"; 0001
								* e.g. polarity swap on CSIx_D1 and CSI(X+1)_D0 -> lane_polarity = "6"; 0110
								*
								* == Image format Properties ==
								*
								* active_w = "";
								* Pixel active region width
								*
								* active_h = "";
								* Pixel active region height
								*
								* pixel_t = "";
								* The sensor readout pixel pattern
								*
								* line_length = "";
								* Pixel line length (width) for sensor mode.
								*
								* == Source Control Settings ==
								*
								* Gain factor used to convert fixed point integer to float
								* Gain range [min_gain/gain_factor, max_gain/gain_factor]
								* Gain step [step_gain/gain_factor is the smallest step that can be configured]
								* Default gain [Default gain to be initialized for the control.
								*     use min_gain_val as default for optimal results]
								* Framerate factor used to convert fixed point integer to float
								* Framerate range [min_framerate/framerate_factor, max_framerate/framerate_factor]
								* Framerate step [step_framerate/framerate_factor is the smallest step that can be configured]
								* Default Framerate [Default framerate to be initialized for the control.
								*     use max_framerate to get required performance]
								* Exposure factor used to convert fixed point integer to float
								* For convenience use 1 sec = 1000000us as conversion factor
								* Exposure range [min_exp_time/exposure_factor, max_exp_time/exposure_factor]
								* Exposure step [step_exp_time/exposure_factor is the smallest step that can be configured]
								* Default Exposure Time [Default exposure to be initialized for the control.
								*     Set default exposure based on the default_framerate for optimal exposure settings]
								*
								* gain_factor = ""; (integer factor used for floating to fixed point conversion)
								* min_gain_val = ""; (ceil to integer)
								* max_gain_val = ""; (ceil to integer)
								* step_gain_val = ""; (ceil to integer)
								* default_gain = ""; (ceil to integer)
								* Gain limits for mode
								*
								* exposure_factor = ""; (integer factor used for floating to fixed point conversion)
								* min_exp_time = ""; (ceil to integer)
								* max_exp_time = ""; (ceil to integer)
								* step_exp_time = ""; (ceil to integer)
								* default_exp_time = ""; (ceil to integer)
								* Exposure Time limits for mode (sec)
								*
								* framerate_factor = ""; (integer factor used for floating to fixed point conversion)
								* min_framerate = ""; (ceil to integer)
								* max_framerate = ""; (ceil to integer)
								* step_framerate = ""; (ceil to integer)
								* default_framerate = ""; (ceil to integer)
								* Framerate limits for mode (fps)
								*
								* embedded_metadata_height = "";
								* Sensor embedded metadata height in units of rows.
								* If sensor does not support embedded metadata value should be 0.
								*/
								mode0 { /* IMX477_MODE_3840x2160 */
									mclk_khz = "24000";
									num_lanes = "2";
									tegra_sinterface = "serial_b";
									phy_mode = "DPHY";
									discontinuous_clk = "no";
									dpcm_enable = "false";
									cil_settletime = "0";
									lane_polarity = "6";
									active_w = "3840";
									active_h = "2160";
									mode_type = "bayer";
									pixel_phase = "rggb";
									csi_pixel_bit_depth = "10";
									readout_orientation = "90";
									line_length = "11200";
									inherent_gain = "1";
									mclk_multiplier = "80";
									pix_clk_hz = "300000000";
									gain_factor = "16";
									framerate_factor = "1000000";
									exposure_factor = "1000000";
									min_gain_val = "16"; /* 1.00x */
									max_gain_val = "356"; /* 22x */
									step_gain_val = "1";
									default_gain = "16"; /* 1.00x */
									min_hdr_ratio = "1";
									max_hdr_ratio = "1";
									min_framerate = "2000000"; /* 2.0 fps */
									max_framerate = "30000000"; /* 30.0 fps */
									step_framerate = "1";
									default_framerate = "30000000"; /* 30.0 fps */
									min_exp_time = "13"; /* us */
									max_exp_time = "683709"; /* us */
									step_exp_time = "1";
									default_exp_time = "2495"; /* us */
									embedded_metadata_height = "2";
								};
								mode1 { /* IMX477_MODE_1920X1080 */
									mclk_khz = "24000";
									num_lanes = "2";
									tegra_sinterface = "serial_b";
									phy_mode = "DPHY";
									discontinuous_clk = "no";
									dpcm_enable = "false";
									cil_settletime = "0";
									lane_polarity = "6";
									active_w = "1920";
									active_h = "1080";
									mode_type = "bayer";
									pixel_phase = "rggb";
									csi_pixel_bit_depth = "10";
									readout_orientation = "90";
									line_length = "7000";
									inherent_gain = "1";
									mclk_multiplier = "80";
									pix_clk_hz = "300000000";
									gain_factor = "16";
									framerate_factor = "1000000";
									exposure_factor = "1000000";
									min_gain_val = "16"; /* 1.00x */
									max_gain_val = "356"; /* 22x */
									step_gain_val = "1";
									default_gain = "16"; /* 1.00x */
									min_hdr_ratio = "1";
									max_hdr_ratio = "1";
									min_framerate = "2000000"; /* 2.0 fps */
									max_framerate = "60000000"; /* 60.0 fps */
									step_framerate = "1";
									default_framerate = "60000000"; /* 60.0 fps */
									min_exp_time = "13"; /* us */
									max_exp_time = "683709"; /* us */
									step_exp_time = "1";
									default_exp_time = "2495"; /* us */
									embedded_metadata_height = "2";
								};
								ports {
									#address-cells = <1>;
									#size-cells = <0>;
									port@0 {
										reg = <0>;
										ark_imx477_out0: endpoint {
											status = "okay";
											port-index = <1>;
											bus-width = <2>;
											remote-endpoint = <&ark_imx477_csi_in0>;
										};
									};
								};
							};
						};
						i2c@2 {
							reg = <0>;
							#address-cells = <1>;
							#size-cells = <0>;
							i2c-mux,deselect-on-exit;
							status = "okay";

							ark_imx219_c@10 {
								status = "disabled";
							};
						};
						i2c@3 {
							reg = <0>;
							#address-cells = <1>;
							#size-cells = <0>;
							i2c-mux,deselect-on-exit;
							status = "okay";
							ark_imx219_d@10 {
								status = "disabled";
							};
						};
					};
				};

				// gpio@2200000 {
				// 	camera-control-output-high {
				// 	gpios = <CAM1_PWDN 1>;
				// 	label = "cam1-pwdn";
				// 	};
				// };
			};
		};
	};
};

[dakejahl]

Thanks for all the information! If you want to open a PR I can take a look and we can get this into our next kernel build.

[dakejahl]

@acehoss I'm looking into this now. So with your changes above, you are able to stream the camera on CSI1 but using sensor-id=0? Sounds like for some reason CSI0 does not work at all?

[acehoss]

I didn't quite get to the bottom of it. From the information I've seen, it's a bug in LibArgus that mishandles the mapping between position in the device tree and sensor-id. It may work in CSI0 by just changing the position or using sensor-id 1. Unfortunately once I got it working I didn't have enough time to test different combinations to see what works and what doesn't.

[dakejahl]

I figured this out today and got it fixed, will be releasing a new kernel for Jetpack 6.1 (r36.4). I will also backport this fix and create a new tag for 36.3 and ensure the README instructions are updated so that users can easily regenerate and add the corrected dtbo

The fix is in this commit
ARK-Electronics/ark_jetson_orin_nano_nx_device_tree@be36ab1

I think LibArgus requires the postion/orientation fields to correspond exactly with the module indices
https://docs.nvidia.com/jetson/archives/r36.4/DeveloperGuide/SD/CameraDevelopment/SensorSoftwareDriverProgramming.html#camera-modules-and-the-device-tree

[acehoss]

Thanks @dakejahl. When I was working with this before the holiday break, the behavior seemed more complex than that, and corresponded pretty well to a swap of the first two sensor id to position labels compared to the documentation. The documentation says front is sensor-id=0 and rear is sensor-id=1, but we were seeing those ids swapped.

Anyway, we were racing towards a demo which did work out. Given you've put some time into this and feel settled on it, I think we close it and if down the road I find there's still a problem I'll reopen it. Thanks again.

[dakejahl]

@acehoss I just tested the fix on 36.3 and it doesn't work, so you're right there is something else causing problems in 36.3.

Since it's now working in 36.4 I would suggest anyone looking to solve this problem update to 36.4. I should have it tested and merged this week #17