NDI - Video over IP feature

[ayanami0]

I saw this last night, and thought it would be interesting for Mister to be able to output NDI signal to NDI compatible devices (including OBS) through the network. And even dump digital raw video/audio data into HDD. But I am not sure if this is anything the community would be interested, or if this official implementation is sutable for a opensource project like MiSTer.

https://www.ndi.tv/sdk/

＂The NDI Embedded SDK is designed for NDI integration into embedded devices. The ARM-based SDK, provides stream level access to NDI, FPGA implementation and full source code to permit an off the shelf FPGA dev-kit to be used as a starting point for NDI encoding projects.＂

[aberu]

Given that the MiSTer FPGA project's HDMI output is a result of direct connection between the FPGA side of things and the HDMI connector as close as possible, wouldn't having the NDI happening on the ARM side of things require new HDMI logic to be added which routes HDMI to the ARM? Or am I missing something?

[wolrah]

Given that the MiSTer FPGA project's HDMI output is a result of direct connection between the FPGA side of things and the HDMI connector as close as possible, wouldn't having the NDI happening on the ARM side of things require new HDMI logic to be added which routes HDMI to the ARM? Or am I missing something?

As I understand it only the analog video output is directly linked to the core, for the HDMI output the core writes a native resolution image to a framebuffer in the second half of the DDR3 that's not used by Linux, and the MiSTer scaler component then picks that up and does what it does. That part of memory is still accessible to Linux though, as demonstrated by https://github.com/alanswx/Screenshot_MiSTer.

All it'd take to stream video would be doing that X times a second and passing the data off to the network stack, so the basic concept of streaming video of some variety is certainly plausible.

I see two major potential limiting factors that would affect how useful this feature could be:

1. Does having the Linux side constantly scraping the scaler memory have an effect on the scaler's ability to access the same data or whatever else a given core might use the DDR3 for?

2. How much of the limited ARM CPU power does the video codec require? NDI uses their own proprietary codec from what I can find, apparently loosely related to MPEG2 in some ways. It's probably not too complicated to encode but it's not going to be free either.

If the first of those is a problem I can't really see any solution with current hardware, but as for the second because we have gigabit networking there's always the brute force option. Any common resolutions up to SVGA (800x600) could be transmitted as raw uncompressed 24 bit color 60 FPS over a gigabit network with plenty of bandwidth to spare. A few of the computer platforms support higher resolutions that couldn't be used this way but I'd imagine the interest in getting real-time video over the network from a MiSTer is primarily from video game streamers who would mostly be using even those computer platforms at lower resolutions.

[aberu]

Ah nice! Then yeah that would be awesome!

[ayanami0]

I have no idea what the dev kit does, to be honest, but from the description the devkit is for FPGA kits, not an SOC so I was hoping the SDK is for hleping super responsive video Encoding and transmission protocol using FPGA. Also I have no idea how much resource the SDK will occupy the FPGA chip to make it worth while..