4 replies

Recommended way to time-sync WAV playback (DMX lightning)?

Hello !

I am reading a WAV/MP3 track with a Membrane pipeline, with good success so far.

I am starting to annotate the WAV so that I know when the main beats occur ; I now want to generate DMX (lightning) events out of that. Sync latency is essential here.

My first stab is using a Membrane filter that does not modify the buffer, but counts the played samples:

# very WIP
defmodule Ticker do
  use Membrane.Filter

  alias Membrane.RawAudio

  def_input_pad :input, accepted_format: RawAudio, flow_control: :auto
  def_output_pad :output, accepted_format: RawAudio, flow_control: :auto

  def handle_init(ctx, options) do
    {[], %{sample_count: 0}}
  end
  
  def handle_buffer(:input, buffer, ctx, state) do
    stream_format = ctx.pads.input.stream_format
    sample_rate = stream_format.sample_rate
    sample_size = RawAudio.sample_size(stream_format)
    state = Map.put(state, :sample_count, state.sample_count + Membrane.Payload.size(buffer.payload)) 
    elapsed_time = state.sample_count / (sample_rate * sample_size * 2)
    IO.inspect(elapsed_time)
    {[buffer: {:output, buffer}], state}
  end
end

# very WIP
defmodule Ticker do
  use Membrane.Filter

  alias Membrane.RawAudio

  def_input_pad :input, accepted_format: RawAudio, flow_control: :auto
  def_output_pad :output, accepted_format: RawAudio, flow_control: :auto

  def handle_init(ctx, options) do
    {[], %{sample_count: 0}}
  end
  
  def handle_buffer(:input, buffer, ctx, state) do
    stream_format = ctx.pads.input.stream_format
    sample_rate = stream_format.sample_rate
    sample_size = RawAudio.sample_size(stream_format)
    state = Map.put(state, :sample_count, state.sample_count + Membrane.Payload.size(buffer.payload)) 
    elapsed_time = state.sample_count / (sample_rate * sample_size * 2)
    IO.inspect(elapsed_time)
    {[buffer: {:output, buffer}], state}
  end
end

(this is very WIP)

Is it the best approach though?

I have no idea if the moment the method is called matches more or less the playback time etc (it could be massively off gradually depending on how buffering works).

I have seen the notion of Timers etc, is it a better way to do that?

Thank you!

-- Thibaut

Software Mansion•10mo ago•

4 replies

thbar

Recommended way to time-sync WAV playback (DMX lightning)?

# very WIP
defmodule Ticker do
  use Membrane.Filter

  alias Membrane.RawAudio

  def_input_pad :input, accepted_format: RawAudio, flow_control: :auto
  def_output_pad :output, accepted_format: RawAudio, flow_control: :auto

  def handle_init(ctx, options) do
    {[], %{sample_count: 0}}
  end
  
  def handle_buffer(:input, buffer, ctx, state) do
    stream_format = ctx.pads.input.stream_format
    sample_rate = stream_format.sample_rate
    sample_size = RawAudio.sample_size(stream_format)
    state = Map.put(state, :sample_count, state.sample_count + Membrane.Payload.size(buffer.payload)) 
    elapsed_time = state.sample_count / (sample_rate * sample_size * 2)
    IO.inspect(elapsed_time)
    {[buffer: {:output, buffer}], state}
  end
end

# very WIP
defmodule Ticker do
  use Membrane.Filter

  alias Membrane.RawAudio

  def_input_pad :input, accepted_format: RawAudio, flow_control: :auto
  def_output_pad :output, accepted_format: RawAudio, flow_control: :auto

  def handle_init(ctx, options) do
    {[], %{sample_count: 0}}
  end
  
  def handle_buffer(:input, buffer, ctx, state) do
    stream_format = ctx.pads.input.stream_format
    sample_rate = stream_format.sample_rate
    sample_size = RawAudio.sample_size(stream_format)
    state = Map.put(state, :sample_count, state.sample_count + Membrane.Payload.size(buffer.payload)) 
    elapsed_time = state.sample_count / (sample_rate * sample_size * 2)
    IO.inspect(elapsed_time)
    {[buffer: {:output, buffer}], state}
  end
end

Recommended way to time-sync WAV playback (DMX lightning)?

Similar Threads

Recommended way to time-sync WAV playback (DMX lightning)?

Similar Threads

Similar Threads

Similar Threads