Open Brush, Open Blocks, Icosa Gallery•5y ago

Not sure why Quest can't do mp4 video but I'm tempted to allow it to capture frame sequences as an o

Not sure why Quest can't do mp4 video but I'm tempted to allow it to capture frame sequences as an option and leave it up to the user to compile them into video, gif or whatever. The current gif mode should just be a quick convenience mode

andybakOP•11/7/21, 1:19 PM

Aandybak Not sure why Quest can't do mp4 video but I'm tempted to allow it to capture fra...

mikeage•11/7/21, 1:19 PM

I don't recall if we have

ffmpeg

ffmpeg

available on the Quest, but I think output a series of frames plus a bit of documentation listing common options like

ffmpeg -i ./%04d.png output.mp4

ffmpeg -i ./%04d.png output.mp4

should be quite useful; it offers a lot more flexibility than writing compressed video, and can fit into any workflow. If we have specific requirements for certain formats, then we can look at offering quick presets too

mikeage•11/7/21, 1:20 PM

I guess we don't have ffmpeg:

... new CopyRequest(FfmpegPipe.kFfmpegDir) { omitForAndroid = true } ...

... new CopyRequest(FfmpegPipe.kFfmpegDir) { omitForAndroid = true } ...

andybakOP•11/7/21, 1:21 PM

I don't think android allows multiple executables in an apk? or at least they are clamping down on that kind of thing

andybakOP•11/7/21, 1:21 PM

i seem to remember that was the issue with LPGL code - you can to create a single signed binary

andybakOP•11/7/21, 1:22 PM

Although we could release a "companion app" that made it all user friendly - but something like that probably already exists

mikeage•11/7/21, 1:22 PM

I don't know. We can distribute other files, including executables, using an

.obb

.obb

file (or possibly an

.aap

.aap

, I think it's called), but I'm not an expert on exactly how that packaging works

andybakOP•11/7/21, 1:22 PM

got to be aware there's people who have a quest but don't own a PC or Mac or anything separate.

mikeage•11/7/21, 1:24 PM

Fair, but I feel like once we have an on screen keyboard and can name our sketches (and thus do things like foo.tilt -> foo_output/1/xxxx.png), it'll be much easier to output just images and have people put them together. It won't be sufficient for those with no computer... but I think for an MVP, it's definitely a higher priority than one more canned output format that may or may not align with a specific requirement

mikeage•11/7/21, 1:24 PM

Or maybe we can do images -> video as a hosted icosa service?

andybakOP•11/7/21, 1:25 PM

there must be a dll version of ffmpeg that we can call directly from C#? I wonder how ffmpeg is being called presently.

mikeage•11/7/21, 1:26 PM

Assets/Scripts/Rendering/FfmpegPipe.cs

andybakOP•11/7/21, 1:27 PM

hmmmm. ffmpeg must have a C API?

andybakOP•11/7/21, 1:27 PM

ah - we might hit GPL issues

mikeage•11/7/21, 1:28 PM

libavcodec is LGPL IIRC

andybakOP•11/7/21, 1:28 PM

and we're back to exactly what does "linking" mean.

mikeage•11/7/21, 1:28 PM

Or maybe dual licensed GPL/LGPL. BRB, googling

mikeage•11/7/21, 1:29 PM

Yep, dual licensed.

mikeage•11/7/21, 1:30 PM

This looks like it might have some of the answers we need, but I'm about to head out, and don't have time to read everything. Leaving it here for later. https://stackoverflow.com/questions/63946068/how-do-i-embed-a-native-code-library-in-an-android-app-sdk-29

Stack Overflow

How do I embed a native code library in an Android app (SDK 29+)?

I have an app that uses a native code library -- a custom build of ffmpeg as it happens, though that isn't strictly relevant.
Up until now I have added the executable to my res/raw directory and th...

andybakOP•11/7/21, 1:30 PM

this is all a bit hypothetical. it's already passed my threshold of "too much effort"

andybakOP•11/7/21, 1:30 PM

considering there's a workaround

andybakOP•11/7/21, 1:31 PM

even if people don't have access to a PC - there's web based mp4/gif generators

andybakOP•11/7/21, 1:32 PM

making it all seamless is a nice to have that can wait until we have a team of developer elves to call upon

andybakOP•11/7/21, 1:32 PM

elves are hard to come by now days. they made a fortune selling pants and no longer want to program

andybakOP•11/7/21, 1:33 PM

andybakOP•11/7/21, 1:34 PM

back to the gif thing - @AncientWorlds - take a look at this object:

andybakOP•11/7/21, 1:35 PM

andybakOP•11/7/21, 1:35 PM

try adding new presets with different settings and see how you get on.

andybakOP•11/7/21, 1:36 PM

@Zandy - can't remember if you're running from inside Unity. If so you could try this too

andybakOP•11/7/21, 1:37 PM

If it works and you can come up with some useful new presets we can add them to a future release

andybakOP•11/7/21, 1:38 PM

In terms of gif "quality" - do you mean resolution or something else? Disabling the post processing was mentioned before. That appears to have been accidentally reenabled and would hit gif quality quite badly in some cases

Zandy•11/7/21, 1:44 PM

Will definitely give it a try @andybak !

andybakOP•11/7/21, 1:48 PM

@AncientWorlds I wish we could pin down exactly when gif quality degraded. Was it bad in the first Open Brush release? Is it bad in Multibrush? I can't see what would have changed

andybakOP•11/7/21, 1:49 PM

You said it was fine when you compared with the last Tilt Brush release. I wonder if it was already broken in the initial open source version

mikesky•11/7/21, 2:08 PM

The gif stuff wasn’t in the initial open sourcing, we had a PR put in for it using another open source library

mikesky•11/7/21, 2:08 PM

So it may not be the original library used in tilt brush

Mmikesky The gif stuff wasn’t in the initial open sourcing, we had a PR put in for it usi...

andybakOP•11/7/21, 2:16 PM

ah! that's helpful

andybakOP•11/7/21, 2:22 PM

So - I want to know if it is related to post processing or not. @AncientWorlds - are you pretty certain that Tilt Brush gifs had no bloom etc?

AAncientWorlds There's a wobble mode and a 5 second gif capture. The 5 second gif capture mak...

andybakOP•11/7/21, 2:28 PM

Can you try one other thing for me? In Scripts/Gif Creation/GifEncodeTask.cs line 166 - change:

ge.useGlobalColorTable = true;

ge.useGlobalColorTable = true;

ge.useGlobalColorTable = false;

ge.useGlobalColorTable = false;

and see if it makes a difference to the gif quality?

andybakOP•11/7/21, 2:29 PM

Seems like every frame is being forced to use the color palette of the first frame and in some cases this might be unsuitable. Might not be the biggest factor though. Depends how typical the first frame is

andybakOP•11/7/21, 2:35 PM

There's also a quality setting that ranges between 1 and 30. It defaults to 10. If you add the line:

ge.quality = 30;

it should change that. I'm not entirely sure if lower=better but try 1 and 30 and it should be fairly obvious

Aandybak In terms of gif "quality" - do you mean resolution or something else? Disabling ...

NLR•11/7/21, 6:35 PM

Gifs are pretty laggy, the motion is not smooth (especially for the 5 seconds gif).

punktvr•11/7/21, 6:46 PM

Hello wonderful Icosa people. I remember seeing talks about taking out the lens blur from video recordings a long time ago- could someone direct me on where i can find more info about that?

Aandybak Sorry - I don't have my headset with me at the moment and I want to check before...

Karmen Olmo•11/7/21, 9:22 PM

First thanks for your support and sorry i see it late. Yes i want to maje a movie navigating the piece. To place some music over. Like you said fly over the work. On mac only gif and picture. Nothing else. Like when use Oculus alone.

Aandybak Can you try one other thing for me? In Scripts/Gif Creation/GifEncodeTask.cs lin...

AncientWorlds•11/8/21, 8:31 AM

Really good info, will take a look at the tools and changes.
How should I have post processing set in the openbrush.cfg file, for this purpose?
As to when it began, probably when the gif file change was made initially.
The newer version does seem to introduce artifacts fairly often, and I'd definitely prefer overall quality improvements.
I considered resizing the video recorder tool window (the square frame is easier to navigate for some shots), but that isn't really the desired outcome.
Anyhow, appreciate the info.
Curious what the other two modes do

NNLR Gifs are pretty laggy, the motion is not smooth (especially for the 5 seconds gi...

andybakOP•11/8/21, 9:49 AM

Trying to understand what "laggy" means. Low frame rate?

andybakOP•11/8/21, 9:52 AM

The "frameDelay" setting for each preset in my earlier screengrabs seems to control frame rate. It's set to 0.05 which 20fps. I do wonder though if gifs are really the right format if you want higher quality. Why not video? Gifs are essentially just a weird historical relic and should probably have been put out to pasture 20 years ago

AncientWorlds•11/8/21, 9:54 AM

two reasons gifs are nice: Easy to post to social media. Square camera format is easy to walthrough navigate scenes with

andybakOP•11/8/21, 9:55 AM

You can have square video too.

andybakOP•11/8/21, 9:55 AM

What social media accepts gif but not video? (Twitter's horrible video compression aside)