MP3 output is audible, but test not pass

Hi everyone. I made small changes in membrane_mp3_lame_plugin to support other input config (the original repo only support 44100/32/1). (patch branch: https://github.com/yujonglee/membrane_mp3_lame_plugin/commits/patch/) After the change, I run the test, but test does not pass. But when I play the generated output file, it is audible and feels same as ref.mp3. (To get generated audio, I commented out https://github.com/yujonglee/membrane_mp3_lame_plugin/blob/patch/test/integration/integration_test.exs#L27 and run test) I wrote a README on how I create new test audio file: https://github.com/yujonglee/membrane_mp3_lame_plugin/blob/patch/README.md Can anyone share some insight on this?
M
mat_hekβ€’72d ago
πŸ‘‹ that’s because this plugin uses snapshot testing. We check if the mp3 is exactly the same as the fixture that we confirmed to be audible and playable by popular players. That prevents introducing unintentional changes to the output mp3. However, if you change the encoder parameters, it’s necessary to manually verify that the output is correct and update the fixture πŸ˜‰
Y
yujongleeβ€’71d ago
@mat_hek Thank you! Actually, I regenerated the fixture with ffmpeg command, and ran the tests. https://github.com/yujonglee/membrane_mp3_lame_plugin/blob/patch/README.md#test-file-generation So output of my ffmpeg command is now new ref.mp3 . Both <generated_from_pipeline>.mp3 and ref.mp3 is audible, but they don't match.
GitHub
membrane_mp3_lame_plugin/README.md at patch Β· yujonglee/membrane_mp...
Membrane MP3 encoder based on Lame. Contribute to yujonglee/membrane_mp3_lame_plugin development by creating an account on GitHub.
M
mat_hekβ€’71d ago
It's unlikely that the mp3 generated with FFmpeg will be exactly the same as generated by Membrane. You should generate the output with Membrane, check if it's correct and then use it as the reference πŸ˜‰
Y
yujongleeβ€’70d ago
Didn't know that they are not same πŸ™‚ Thank you