Kevin Powell - Community•3mo ago

How to generate first frame of video as image?

Hello. I'm working on social network app with Next.js and MongoDB. Users should be able to post images and videos as content (btw all posts in DB have content field which is array of links to that content, which is either image or video). I'm facing an issue when trying to display posts to the user. I have a <PostCard /> component which renders first element of content as "thumbnail" and problem occurs when first element is video, which obviously can't be rendered as image. So I was thinking, if first element is video, it should somehow take first frame of that video and use it as thumbnail. But it would probably be bad to render it every time user visits the page, so I was thinking of generating it once when user creates a post, and save it as something like poster as separate field in DB. But I have no idea how to do that lol. Any suggestions/advices/opinions?

31 Replies

ἔρως•3mo ago

you can use the fluent-ffmpeg package apparently, all you need to do is this:

ffmpeg('/path/to/file.avi')
  .output('screenshot.png')
  .run();

ffmpeg('/path/to/file.avi')
  .output('screenshot.png')
  .run();

Tenkes•3mo ago

Doesn't seem to work for me. Here's what I got:

|   main.js
|   package-lock.json
|   package.json
|
+---assets
|   +---images
|   \---video
|           city.avi
|           city.mp4
|
\---node_modules

|   main.js
|   package-lock.json
|   package.json
|
+---assets
|   +---images
|   \---video
|           city.avi
|           city.mp4
|
\---node_modules

main.js:

const ffmpeg = require('fluent-ffmpeg')

ffmpeg('/assets/video/city.mp4').output('/assets/images/output.png').run()

const ffmpeg = require('fluent-ffmpeg')

ffmpeg('/assets/video/city.mp4').output('/assets/images/output.png').run()

And I am getting error in console

node:events:492
      throw er; // Unhandled 'error' event
      ^

Error: Cannot find ffmpeg
    at D:\Boris\vs-code\node\testing\node_modules\fluent-ffmpeg\lib\processor.js:136:22
    at D:\Boris\vs-code\node\testing\node_modules\fluent-ffmpeg\lib\capabilities.js:123:9 
    ...

node:events:492
      throw er; // Unhandled 'error' event
      ^

Error: Cannot find ffmpeg
    at D:\Boris\vs-code\node\testing\node_modules\fluent-ffmpeg\lib\processor.js:136:22
    at D:\Boris\vs-code\node\testing\node_modules\fluent-ffmpeg\lib\capabilities.js:123:9 
    ...

ἔρως•3mo ago

try the screenshots LOL installl ffmpeg

Tenkes•3mo ago

I did? 😂

    "dependencies": {
        "fluent-ffmpeg": "^2.1.2",
        "nodemon": "^3.1.0"
    }

    "dependencies": {
        "fluent-ffmpeg": "^2.1.2",
        "nodemon": "^3.1.0"
    }

ἔρως•3mo ago

that's not ffmpeg https://ffmpeg.org/download.html you need that i recommend you to use this on linux instead

Tenkes•3mo ago

And how would that work on my app? Would that work when I deploy it?

ἔρως•3mo ago

you can set the ffmpeg path with this well, generate it before you deploy this is important because deployment runners don't have much/any storage, usually

Tenkes•3mo ago

That's not what I want. I need to generate first frame as image from video that user uploads. So I have input[type=file] where user uploads video he wants to publish on my app. Then I store that video on uploadthing (I get URL to video as response), then I take that stored video, take first frame and store it to uploadthing again. So in my DB I'd have something like

{
  ... 
  content: ['https://uplaodthing.com/example1.mp4', https://uplaodthing.com/example2.png ...], 
  poster: 'https://uploadthing.com/poster-of-examples.png' 
  ...
}

{
  ... 
  content: ['https://uplaodthing.com/example1.mp4', https://uplaodthing.com/example2.png ...], 
  poster: 'https://uploadthing.com/poster-of-examples.png' 
  ...
}

ἔρως•3mo ago

i would expect uploadthing to do that automatically

Tenkes•3mo ago

That's how I planned it out, maybe there's a better way

ἔρως•3mo ago

if it doesn't, that ... eh

Tenkes•3mo ago

Well it's just for storing files lol

ἔρως•3mo ago

i know, but it could generate previews too but i get why it doesnt do that you can generate the frame in js

Jochem•3mo ago

no idea if they're functional, and I've never used them, but there are ports of ffmpeg to js / wasm

ἔρως•3mo ago

canvas can read a video i though you wanted to generate it all and then deploy what you want is kinda easy in the browser window https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API/Manipulating_video_using_canvas <-- this is an example of it but, to handle more codecs, a wasm ffmpeg is a good idea

Tenkes•3mo ago

I found few example of that. But most of them are done differently so I got confused lol. I'll have to look more into that. Just to make sure... It can work with File from JS right?

ἔρως•3mo ago

basically, the canvas 2d context reads the contents of the video it should

ἔρως•3mo ago

https://stackoverflow.com/questions/63888726/how-to-play-video-from-input-field-in-html

Stack Overflow

How to play video from input field in html

I want create a page that allows a user to choose a video file from their local machine, and then have that video play on html. I have each separate portion figured out, but how do I get them to in...

ἔρως•3mo ago

if the video plays, then it should be usable for the canvas

Tenkes•3mo ago

Sweet, thanks for help

ἔρως•3mo ago

you're welcome

Tenkes•3mo ago

I seem to love to overcomplicate things 😂 Here I am saying of generating image from video as user uploads it, saving it as separate field in DB etc when I can just check if content URL ends with video or image type and based on that render image or video lol.

{imageTypes.some(type => post.content[0].endsWith(type)) ? (
    <Image
        src={post.content[0]}
        alt={post.altText}
        width={330}
        height={315}
        className='w-full h-full object-cover'
    />
    ) : (
    <video
        src={post.content[0]}
        width={330}
        height={315}
        className='w-full h-full object-cover'
    />
)}

{imageTypes.some(type => post.content[0].endsWith(type)) ? (
    <Image
        src={post.content[0]}
        alt={post.altText}
        width={330}
        height={315}
        className='w-full h-full object-cover'
    />
    ) : (
    <video
        src={post.content[0]}
        width={330}
        height={315}
        className='w-full h-full object-cover'
    />
)}

I forgot video will just act like image if controls, autoplay etc isn't provided to it lol

ἔρως•3mo ago

that won't work videos usually have just a black backdrop before they load you can set it to autoload the metadata, which usually renders the first frame too

Tenkes•3mo ago

It works fine though Might be because I'm using Next.js, and it's code is rendered on server and HTML code is sent to client

ἔρως•3mo ago

is the video playing automatically?

Tenkes•3mo ago

it's not playing at all acts like regular image

ἔρως•3mo ago

that's good i guess it is loading just the metadata

Tenkes•3mo ago

it works, it's all that matters lol imma enjoy success while I can, there might be problems with this approach later

ἔρως•3mo ago

bandwidth is one that comes to mind images can be lazy loaded that video will load imediately

Tenkes•3mo ago

true I'll find better solution later.

ἔρως•3mo ago

the simplest one is to do what i've shown you reading the video into a canvas is super easy

Gaming

Programming

How to generate first frame of video as image?