Live video effects in fishjam

Hey, I am curious about how possible it is to implement live effects into fishjam webrtc streams from the server using a custom endpoint with something like video-compositor in real time. How possible is this with membrane currently?

Excuse the video but here is an example of an effect I am talking about https://www.youtube.com/shorts/4YaE5u1mjlc

YouTubegiliy 18

Pouring a glass of milk with the whirlpool effect tik tok

Wojciech_Barczynski•7/4/24, 9:35 PM

Hi,
with Membrane itself, it should be fairly simple to implement such an effect using membrane_live_compositor_plugin.

You must construct a pipeline with LiveCompositor Plugin, register your shader with that effect, and use it on the input stream (input pad).

You can find a similar pipeline example here. We have a very similar shader in the LiveCompositor repo.

For info on shaders themselves, refer to this intro.

I'm not sure about integration with FishJam. @Jellyfish/WebRTC you help with that?

Wojciech_Barczynski•7/5/24, 10:01 AM

cc @mickel8 @kamilstasiak

Michał Śledź•7/5/24, 10:23 AM

You would have to create a new endpoint that will use membrane_live_compositor_plugin under the hood. This plugin. AFAIK, will send data to the Live Compositor, which will create an animation and send the video back to the fishjam endpoint. Then you just publish this returned video as a new track and it will be consumed by the other endpoints

Michał Śledź•7/5/24, 10:23 AM

More or less

JdynOP•7/5/24, 11:58 PM

Great I appreciate the links and examples. I think I have a decent idea of what to do after both of your guys help. Cheers

Live video effects in fishjam

Similar Threads

Similar Threads

Similar Threads