SMM
Software Mansion / MembraneAske

H264.FFmpeg.Decoded frames to MP4.Muxer

I'm attempting to open a local mp4, demux it, and write it back to an mp4, just to get started. I want to do stuff with overlay images and add sound clips once this basic thing is working. This is my spec: 
    structure = [
      child(:video_source, %Membrane.File.Source{
        location: "example_data/example.mp4"
      })
      # Video Parser
      |> child(:demuxer, Membrane.MP4.Demuxer.ISOM)
      |> via_out(Pad.ref(:output, 1))
      |> child(:video_parser, %Membrane.H264.Parser{
        # Required for the FFmpeg.Decoder/Encoder
        output_stream_structure: :annexb
      })
      |> child(:video_decoder, Membrane.H264.FFmpeg.Decoder),
      # Audio Parser
      get_child(:demuxer)
      |> via_out(Pad.ref(:output, 2))
      |> child(:audio_parser, %Membrane.AAC.Parser{
        # Required for MP4.Muxer
        output_config: :esds
      }),
      # TODO: Overlays + audio clips
      # Mux to back to MP4
      child(:muxer, %Membrane.MP4.Muxer.ISOM{})
      |> child(:sink, %Membrane.File.Sink{location: "example_data/processed.mp4"}),
      get_child(:audio_parser) |> get_child(:muxer),
      get_child(:video_decoder)
      |> child(%Membrane.H264.FFmpeg.Encoder{
        preset: :ultrafast
      })
      |> get_child(:muxer)
    ]
    structure = [
      child(:video_source, %Membrane.File.Source{
        location: "example_data/example.mp4"
      })
      # Video Parser
      |> child(:demuxer, Membrane.MP4.Demuxer.ISOM)
      |> via_out(Pad.ref(:output, 1))
      |> child(:video_parser, %Membrane.H264.Parser{
        # Required for the FFmpeg.Decoder/Encoder
        output_stream_structure: :annexb
      })
      |> child(:video_decoder, Membrane.H264.FFmpeg.Decoder),
      # Audio Parser
      get_child(:demuxer)
      |> via_out(Pad.ref(:output, 2))
      |> child(:audio_parser, %Membrane.AAC.Parser{
        # Required for MP4.Muxer
        output_config: :esds
      }),
      # TODO: Overlays + audio clips
      # Mux to back to MP4
      child(:muxer, %Membrane.MP4.Muxer.ISOM{})
      |> child(:sink, %Membrane.File.Sink{location: "example_data/processed.mp4"}),
      get_child(:audio_parser) |> get_child(:muxer),
      get_child(:video_decoder)
      |> child(%Membrane.H264.FFmpeg.Encoder{
        preset: :ultrafast
      })
      |> get_child(:muxer)
    ]
I get this error: 
[error] GenServer #PID<0.1357.0> terminating
** (Membrane.StreamFormatError) Stream format: %Membrane.H264{width: 640, height: 360, profile: nil, alignment: :au, nalu_in_metadata?: false, framerate: nil, stream_structure: :annexb} is not matching accepted format pattern "%Membrane.AAC{config: {:esds, _esds}}%Membrane.H264{stream_structure: {:avc1, _dcr}, alignment: :au}%Membrane.H265{stream_structure: {_hevc, _dcr}, alignment: :au}%Membrane.Opus{self_delimiting?: false}" in def_input_pad
for pad :input in Membrane.MP4.Muxer.ISOM
[error] GenServer #PID<0.1357.0> terminating
** (Membrane.StreamFormatError) Stream format: %Membrane.H264{width: 640, height: 360, profile: nil, alignment: :au, nalu_in_metadata?: false, framerate: nil, stream_structure: :annexb} is not matching accepted format pattern "%Membrane.AAC{config: {:esds, _esds}}%Membrane.H264{stream_structure: {:avc1, _dcr}, alignment: :au}%Membrane.H265{stream_structure: {_hevc, _dcr}, alignment: :au}%Membrane.Opus{self_delimiting?: false}" in def_input_pad
for pad :input in Membrane.MP4.Muxer.ISOM
The muxer expects :avc1 while the FFmpeg.Encoder outputs annexb. Either I'm missing a node, or I've misunderstood something somewhere. Any tips?
A
Aske•35d ago
Ah... Adding an H264.Parser after the Encoder did the trick.. Was confused by the name.
Want results from more Discord servers?
Add your server
More Posts
Wiring up Javascript FE Using membrane-webrtc-jsSorry if this obvious. I'm looking through the example in the membrane_rtc_engine (link below). It'sFilter with `push` flow_controlHello, I have a filter that transcribes audio as it receives it by sending it to a transcription serLL-HLS broadcastingHello everyone! I am trying to make LL-HLS broadcasting work. I used the demo from webrtc_to_hls aPipeline children started twiceHello, I'm seeing children in a Membrane pipeline get started twice: I think this might be an issu