Overlays for live streaming

[connyg]

First of all: excellent project!!

My wife does cooking class live streams every week and I am usually creating overlays showing the ingredients with a dtp tool and export that as png 4K to add to the streaming software. The software is Ecamm Live.
That's a cumbersome process if you have to create these overlays repetitively every week.

As I am working on creating a recipe DB & recipe card rendering tool creating them in HTML and rendering them to PDF I will have all the data and images at hand shortly.
From that I could render all these overlays realtime, just from the data.

I think that this is something that fits nicely with the react part of this framework, to render the stuff with react components.
What I would need now is:

• a cockpit with buttons that will play the different animations / overlays
• the click of a button would start the video/animation in a predefined windows / screen, that empty/transparent by default
• a method for the streaming software to pick it up. The software can receive video streams (via a NDI app/screen capture for example https://ndi.video/type/ndi-tools/ ), it can accept html directly (not sure if that would include react components)

Any proposal how that is easy to set up, ideally all starting with one command?

Any plans to elaborate on this use case more in the future?

[connyg]

Thinking about this some more.
I am working with Puppeteer for the above mentioned PDF rendering of HTML.
Puppeteer opens a hidden Chrome for Testing browser window to do the rendering.
Would that be a solution, too?

I could then make the overlay app a rest server on the backend side, that accepts commands for playing the content.
That overlay app would launch a chrome browser for testing that does the rendering. And NDI screen capture would capture the contents of that screen as virtual camera.
Another web app is displayed on a tablet with buttons to launch the overlays. The buttons would do rest calls to that "overlay backend".

Is that overdone or the right way?

[connyg]

I could use 2nd virtual screen created by
https://github.com/Brezel31/BetterDummy
and stream that with NDI scan converter as a virtual camera.

[regodo]

is there a linux wersion for this?

[connyg]

That is Mac software. I would try Google.

[connyg]

Got it to work without the studio. ✔
Found that the streaming software can do screen capture from apps / the browser directly. ✔
And BetterDummy works fine, I get an additional virtual screen where I can define the screen resolution as I want (same as video resolution) - so I can make the browser window to be the same than the screen resolution. ✔

Left is now to remote control the remotion player from a tablet.

[JonnyBurger]

Hey @connyg!

Cool story and experiments that you are running!
I personally think Remotion is not so suited for broadcasting. It is slower than other frameworks to render to the DOM and requires a fixed duration to be specified.

However for pre-rendering transparent overlays it is not a bad idea and I have for example created an OBS stinger before. https://github.com/JonnyBurger/ledevevent-stinger

Still very cool that you got it working nonetheless!

[connyg]

Yes, it's not a perfect for for my use case it seems. Exactly what you mentioned - fixed duration - was a point I was thinking about. Would need three animations, one to fly something in, one keep it for a manually determined time and one to fly out. So I would need to script that sequence, too.
Any recommendation on an alternative framework for streaming / live rendering?

It's not slow for me, but it's running on nice Apple M1 hardware :-)