AU/Vista HDMI input through mipi port #17
Comments
|
This is already theoretically possible. Runcam makes "unlicensed" cameras using mipi. While I agree that this would be very cool, it is outside the scope of margerine (IMO). Cheers, I too look forward to the day when we can broadcast HDMI over DJI. This would especially be nice for Simulators. PS: I'm In no way saying that we couldn't use custom software to inject display data over USB into an air unit for broadcast, but your question was about MIPI - which I agree is the preferred method. |
I looked a bit more and it seems that the pinout has already been discovered.. Not sure but you may need a 4 channel HDMI to MIPI board (most seem to be only 2 channels). If so, they ARE available. Seems the C780B board has 4 channels. GL |
|
I think the hardest part now will be sourcing the socket for those dji mipi connectors |
|
I agree
…On Wed, May 4, 2022, 8:42 AM Duckodope ***@***.***> wrote:
I think the hardest part now will be sourcing the socket for those dji
mipi connectors
—
Reply to this email directly, view it on GitHub
<#17 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AGFWWLZAVJM6UETOLQU7OT3VIJWCHANCNFSM5UF47RJQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
It should be possible if you manage to source the connectors (I know someone who tried and wait and waited and never got them) and match the MIPI stream config exactly as the RTOS on the Vista/AU is expecting a very specifically configured video stream. I don't know the details but they can probably be reverse engineered from the RTOS binaries/logs. |
Perhaps just use an existing cable? Solder it to the hdmi adapter |
|
That'll surely work for prototypes, but I think he meant for production.
Although I think the $10 the cable cost is going to be trivial for most
people who want this feature.
…On Sat, May 7, 2022, 12:45 AM a1axx ***@***.***> wrote:
I think the hardest part now will be sourcing the socket for those dji
mipi connectors
Perhaps just use an existing cable?
—
Reply to this email directly, view it on GitHub
<#17 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AGFWWL6NWOVBJ7C4OWKTH3DVIXYODANCNFSM5UF47RJQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
In my opinion someone would need a socket soldeted to a custom hdmi to mipi converter. I dont think cutting an existing cable and then splicing and soldering the tiny coax cabels to a pcb is a viable or even possible option. Does anyone know what this socket is called? Let alone even knowing the manufacturer would be a huge step forward. I'm sadly not able to figure out what kind of chips it would need to convert the hdmi signal quickly enough into mipi. And how the signal even has to look in the first place. If anyone needs help in CAD design for a project i would be happy to help in exchange for help with this problem. |
|
My understanding is that MIPI is a standard. Meaning that supplying the
data on the correct pins should do it..? Not 100%. I fully disagree about
the connection tho.. one can easily cut one end off of a DJI coaxial cable
and solder the wires to the end of the ribbon cable that comes with these
HDMI > MIPI Boards. Not the most elegant solution but would be fine for
testing.
…On Sat, May 7, 2022, 7:32 AM Duckodope ***@***.***> wrote:
I think the hardest part now will be sourcing the socket for those dji
mipi connectors
Perhaps just use an existing cable? Solder it to the hdmi adapter
In my opinion someone would need a socket soldeted to a custom hdmi to
mipi converter. I dont think cutting an existing cable and then splicing
and soldering the tiny coax cabels to a pcb is a viable or even possible
option. Does anyone know what this socket is called? Let alone even knowing
the manufacturer would be a huge step forward. I'm sadly not able to figure
out what kind of chips it would need to convert the hdmi signal quickly
enough into mipi. And how the signal even has to look in the first place.
If anyone needs help in CAD design for a project i would be happy to help
in exchange for help with this problem.
:D
—
Reply to this email directly, view it on GitHub
<#17 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AGFWWL4WAT744TP54ANT2SDVIZIDBANCNFSM5UF47RJQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
Perhaps the easiest solution would be to pipe the display data into a
rooted Air Unit over USB C and make the air unit transmit. I'm not sure if
the Air Unit's type C is 3.0 or not.. would matter for bandwidth.. I'm 99%
sure this is the easier solution and doesn't require modifications to
hardware. Also I would bet this would be the lower latency solution.
…On Sat, May 7, 2022, 7:37 AM Ryan ***@***.***> wrote:
My understanding is that MIPI is a standard. Meaning that supplying the
data on the correct pins should do it..? Not 100%. I fully disagree about
the connection tho.. one can easily cut one end off of a DJI coaxial cable
and solder the wires to the end of the ribbon cable that comes with these
HDMI > MIPI Boards. Not the most elegant solution but would be fine for
testing.
On Sat, May 7, 2022, 7:32 AM Duckodope ***@***.***> wrote:
> I think the hardest part now will be sourcing the socket for those dji
> mipi connectors
>
> Perhaps just use an existing cable? Solder it to the hdmi adapter
>
> In my opinion someone would need a socket soldeted to a custom hdmi to
> mipi converter. I dont think cutting an existing cable and then splicing
> and soldering the tiny coax cabels to a pcb is a viable or even possible
> option. Does anyone know what this socket is called? Let alone even knowing
> the manufacturer would be a huge step forward. I'm sadly not able to figure
> out what kind of chips it would need to convert the hdmi signal quickly
> enough into mipi. And how the signal even has to look in the first place.
>
> If anyone needs help in CAD design for a project i would be happy to help
> in exchange for help with this problem.
> :D
>
> —
> Reply to this email directly, view it on GitHub
> <#17 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AGFWWL4WAT744TP54ANT2SDVIZIDBANCNFSM5UF47RJQ>
> .
> You are receiving this because you commented.Message ID:
> ***@***.***>
>
|
|
Is this method you are talking about already working? Do you think you could use a hdmi to usbc converter to make this possible? |
|
I am unaware of any current projects. Just one of the many, many things
that are possible with root access.
I would say (without looking) that a type C video adapter would not work.
Certain thing would be required which I doubt they put into the air unit.
The type C would have to support "HDMI Alt Mode" and I'm pretty sure that
requires a GPU. I highly doubt this is supported by the hardware. Also it
would still require supporting software. This doesn't mean that it's not
possible, just can't use a Type C HDMI adapter.
We could test if the USB Type C is 3.0 or greater by testing transfer rate
from a fast micro SD to PC using Mass Storage Mode on PC since USB 2.0 only
supports 480MBPS while USB3.0 Supports 5GBPS and 3.1 supports 10GBPS. This
won't tell us if it is 3.0 or 3.1 but will tell us if it is at least 3.0.
(unlikely to find a microSD faster than 5GBPS)
…On Sat, May 7, 2022, 7:48 AM Duckodope ***@***.***> wrote:
Is this method you are talking about already working? Do you think you
could use a hdmi to usbc converter to make this possible?
—
Reply to this email directly, view it on GitHub
<#17 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AGFWWL3B33FQWKKBCN6JOSLVIZKAXANCNFSM5UF47RJQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
Is there any news about this topic? I am looking for a HDMI Input feature on the air-unit side for a while now. The "splitting cable method" was also my last option. https://www.digikey.de/short/vffprdv7 |
|
No news that I'm aware of, don't think anyone is actively working on it. A couple of notes:
Both approaches are very hard at best. The MIPI one stands a better chance in my mind for sure, as it doesn't require a ton of very hard custom software. You'd still need to figure out the exact video stream format by reverse engineering the RTOS (or getting Caddx/Runcam to spill the beans) and finding a converter that is able to output that exact format. I would say the more realistic option is to look at proffesional solutions that can do this, such as the DJI Transmission. |
|
I started down this path as well a little while back :) I quickly came to the conclusion that messing with the hot metal box transmitter thingie is a non-starter as it requires hacking deep into the guts of the thing. I settled on a man-in-the-middle on the MIPI stream itself as that would allow a more modular and clean solution. The connector is this one. I confirmed this by building a small board here. The micro-coax cables fit properly. The pinout referred to above is almost correct: its missing the I2C as well as clock signals that are mis-labelled as ground. I looked at the I2C traffic to the camera briefly and its interesting: after initial camera configuration, the metal box sends what is probably auto exposure/gain to the camera using the 1MHz I2C protocol. Simply adding a MIPI source is unlikely to keep the metal box happy: a spoof of the camera is probably required at a minimum so it gets the right responses back from the camera: not too tough to do but work required here. With my little adapter, I should be able to capture the I2C data quite easily for analysis. Havent put in all this data into the repo yet. The above work is open source. Happy to collaborate if someone is interested. On a commercial front, I'm currently building a Lattice FPGA (Crosslink NX 33U) based 1"x1" module (to be released in 3Q'23) that will handle MIPI <-> USB3 conversion. Should be able to hijack the MIPI stream and sniff/inject MIPI. |
|
Which camera were you using?
…On Thu, Jul 6, 2023 at 2:21 PM vr2045 ***@***.***> wrote:
I started down this path as well a little while back :) I quickly came to
the conclusion that messing with the hot metal box transmitter thingie is a
non-starter as it requires hacking deep into the guts of the thing. I
settled on a man-in-the-middle on the MIPI stream itself as that would
allow a more modular and clean solution.
The connector is this one
<https://www.lcsc.com/product-detail/_HRS-Hirose-_C5338332.html>. I
confirmed this by building a small board here
<https://github.com/tinyvision-ai-inc/fpv_hacking/blob/main/PCB/CADDX_passthrough/Schematic/CADDX.pdf>.
The micro-coax cables fit properly.
The pinout referred to above is almost correct: its missing the I2C as
well as clock signals that are mis-labelled as ground.
Unfortunately, I am off by one on the power on my board so I had to hack
my hack to get the passthrough to work but I seem to have killed my camera
in the process :(
Am planning to build another board that will have this issue fixed and
allow me to probe the MIPI signals to figure out the data rate and other
settings of the MIPI stream.
I looked at the I2C traffic to the camera briefly and its interesting:
after initial camera configuration, the metal box sends what is probably
auto exposure/gain to the camera using the 1MHz I2C protocol. Simply adding
a MIPI source is unlikely to keep the metal box happy: a spoof of the
camera is probably required at a minimum so it gets the right responses
back from the camera: not too tough to do but work required here. With my
little adapter, I should be able to capture the I2C data quite easily for
analysis.
Havent put in all this data into the repo yet.
The above work is open source. Happy to collaborate if someone is
interested.
On a commercial front, I'm currently building a Lattice FPGA (Crosslink NX
33U) based 1"x1" module (to be released in 3Q'23) that will handle MIPI <->
USB3 conversion. Should be able to hijack the MIPI stream and sniff/inject
MIPI.
—
Reply to this email directly, view it on GitHub
<#17 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABTO2IABWLOJHRMBOFAKS3XO4M4RANCNFSM5UF47RJQ>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
|
Nice - if you use a 60fps camera you'll find it's quite a bit simpler as they don't rely on I2C in the same way. The 120fps cameras (AP1xxx ISP) rely on parameterization and firmware from the host side over I2C, the other cameras won't be using I2C at all as far as I can tell as they rely on some fixed parameterization installed on the camera and aren't configurable. |
|
The I2C comms are relatively trivial to sniff and reverse engineer from a device / the firmware btw, but yeah if you go for 60fps then I2C shouldn't be required at all. I think AVB should be handled internally in the AP1302 firmware rather than by the host ARM SoC, but I might be mistaken. |
Wait so does anyone know what the true pinout for the connector is? I'm working on a similar design to adapt digital FPV to the raspberry pi CM4 and I can't find pinout information on any of these things that doesn't contradict itself. |
|
Finally a group of like minded individuals. I’m working on getting hdmi input to my goggles 2. Either via the vista or air unit via mipi adaptor (anyone know the estimated lag?) or having the input linked directly to the screens on the goggles 2. I’d just power the screens via a switch and have hdmi in from there? Anyone know what model screens are in the goggles 2? talking about getting hdmi input to the vista or air unit, imagine the range capabilities of having say your vista unit a few yards away and running say hd zero? im really interested in getting this going as long as the lag isn’t unplayable on a sim for example |
I've been messing around with a Runcam Wasp digital FPV camera (it's 120fps, so I'm not sure how relevant this is to 60fps cameras) I have on hand. It has an FH8323 video processing chip on board and my guess is that this is used to touch up the image from the sensor before it goes through the MIPI interface back to the transmitter. There's not a lot of info on the chip besides a short product brief in Chinese and the datasheet isn't readily available after a quick google search. I decided i'd email Fulhan and see if I could get the datasheet on it because it might contain info on the video format over MIPI. It's completely in Chinese, but if anyone wants it here it is. https://drive.google.com/drive/folders/1E9Nua7Ku6e4FVbhYvsGz-NqtvuZIeDL7 I'm guessing/hoping that the MIPI format should be relatively the same for DJI fpv cameras, so maybe this would let us drive arbitrary video into the air unit or get video out of the cameras. (I.e if you could send video in the same format this chip does, the air unit wouldn't be able to tell the difference) UPDATE: Managed to run the PDF through google translate. I can't be sure that anything in there is actually correct but i've added the translated datasheet to the drive I posted earlier) I will put any other information I find on this chip in the drive as well. |
|
Anyone have any updates on this? Ive been looking at it too but seem to be getting bogged down in translation and slow shipping from aliexpress on parts |
I'll be honest I'm not really sure it's possible anymore. I've basically given up on the project. If DJI was a little less proprietary about what protocols they use then it would probably be more feasible. Lack of docs and support + the need for custom hardware AND firmware to encode into and out of the various formats just drove this into the ground. In the meantime though, I'll +1 this too. Hopefully someone smarter and more capable than I has some better insights to figure it out. Would love to be able to get video off of one of these cameras and into a Pi or something. |
|
i was hoping to be able inject analog signal into an old vista unit (via the mipi connector and remove the camera). Use it as a VRX to get analog into goggles 2 |
Yep, I think that's the main problem most people are trying to fix. I'm more invested in the other half for getting video off the cameras. Two halves of the same problem I guess. We figure out the protocol off of a camera we can emulate it and get arbitrary video into an AU (and vice versa). |
|
A user called “Auster” in the Discord has been somewhat successful with
video input:
https://discord.com/channels/849931046883819550/961604870937387018/1192238350988484709
The main issue is that there are basically two modes for video input, and
they are not commonly supported by HDMI to CSI bridges.
What we would like to find is an experienced FPGA developer with knowledge
of CSI. Making a working HDMI or Analog to CSI bridge should be a trivial
task for such a person since Auster managed to define the necessary CSI
parameters.
…On Wed, Jan 24, 2024 at 10:22 AM AlureonTheVirus ***@***.***> wrote:
i was hoping to be able inject analog signal into an old vista unit (via
the mipi connector and remove the camera). Use it as a VRX to get analog
into goggles 2
Yep, I think that's the main problem most people are trying to fix. I'm
more invested in the other half for getting video off the cameras. Two
halves of the same problem I guess. We figure out the protocol off of a
camera we can emulate it and get arbitrary video into an AU (and vice
versa).
—
Reply to this email directly, view it on GitHub
<#17 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABTO2PES4PDGUJU5E63ILTYQE7LTAVCNFSM5UF47RJ2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJQHA2TQOJXGM3Q>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
Very interesting and very promising. I wish I had knowledge of FPGA but unfortunately I don't. In saying that I am obviously willing to help anyway way I can. All such a shame though that DJI does not allow HDMI input over the USB-C port and obviously the reach of the market for goggles 2 don't warrant enough people trying to root them to allow software changes for this to happen. |
|
For some reason, the discord link you posted doesnt lead me anywhere.
I am experienced with Lattice FPGA's and CSI and can look into
the feasibility, heres the project I'm working on that could be used for
this purpose as its light weight: https://tinyclunx33.tinyvision.ai/
…On Wed, Jan 24, 2024 at 9:55 AM TelefraggerRick ***@***.***> wrote:
Very interesting and very promising. I wish I had knowledge of FPGA but
unfortunately I don't. In saying that I am obviously willing to help anyway
way I can.
All such a shame though that DJI does not allow HDMI input over the USB-C
port and obviously the reach of the market for goggles 2 don't warrant
enough people trying to root them to allow software changes for this to
happen.
—
Reply to this email directly, view it on GitHub
<#17 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AHDV6C3KC4VE6LPI4YZNJV3YQFDJRAVCNFSM5UF47RJ2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJQHA3DINJVGQZQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
I will repost it here for you. You need to be part of the wtf.os discord to see it. Auster In order to unlock compatibility with normal cameras the last problem must be solved. It is 1080p 24-60fps HDMI input. During my research only two MIPI CSI modes were found:
With the first approach, video must be down-scaled to 720p before HDMI-CSI bridge. Also, at least 50 fps must be provided which might be a problem too. In the "AP1302 mode", the challenge is a conversion from
PLEASE, if you have any ideas let me know. I'm scared to go into the FPGA approach. |
|
Try joining the Discord first: https://discord.gg/RsWEagrxyH ? (sorry, Discord is a pain sometimes). Your project looks almost ideal, basically what we are trying to do is tinyCamera in reverse. It also just looks cool in general and I would be interested in some for other uses as well :) |
|
Auster has also posted this video of it working, though through RCA-HDMI so latency is quite high |
|
I joined the Discord. Will try to participate in the discussion there.
The tinyCamera is called that but nothing prevents us from using the FPGA
to receive or drive CSI. We can do a man-in-the-middle approach and suck
MIPI in one side and spit out MIPI the other side and do whatever format
conversions need to be done along the way in the FPGA.
I've not played with HDMI though. Not clear to me what users want or why
HDMI is needed in the first place.
…On Wed, Jan 24, 2024 at 10:51 AM Brian Ledbetter ***@***.***> wrote:
Try joining the Discord first: https://discord.gg/RsWEagrxyH ? (sorry,
Discord is a pain sometimes).
Your project looks almost ideal, basically what we are trying to do is
tinyCamera in reverse. It also just looks cool in general and I would be
interested in some for other uses as well :)
—
Reply to this email directly, view it on GitHub
<#17 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AHDV6CY4SSE4ZMAN6QEZJOLYQFJZ3AVCNFSM5UF47RJ2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJQHA3TEOJQGQYA>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
where to buy? :) |
|
I have red a few month some news on MadsManTec Blog about this adapter (currently sold out): Can't wait to test one, what do you think - is it worth to try`? |
|
At first glance, it looks promising. The picture from the review appears to
be the setup we would use.
…On Fri, Sep 27, 2024, 04:22 Lorenz ***@***.***> wrote:
I have red a few month some news on MadsManTec Blog about this adapter
(currently sold out):
https://store.cinetronix.co.uk/products/video-input-dev-board-hdmi-compatible
this one might be kind of the same??:
https://de.aliexpress.com/item/1005007329590614.html
Can't wait to test one, what do you think - is it worth to try`?
—
Reply to this email directly, view it on GitHub
<#17 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABO65BDG26MBPF4GUU6IFITZYUIT3AVCNFSM6AAAAABO6UYMZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNZYG4ZDIOBRGQ>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
Hi there, i wonder if it will be possible in the future to add a attach a female hdmi port to the vistas mipi input to transmit whatever signal you like. That would be pristine for gimbal operating to get framing perfect. Yes latency would go up but thats not a big problem if it stays below 200ms or smth like that.
Greets
The text was updated successfully, but these errors were encountered: