Open Brush, Open Blocks, Icosa Gallery•4y ago

For context: currently in med school. I'm working with the anatomy dept to create interactive VR lea

For context: currently in med school. I'm working with the anatomy dept to create interactive VR learning modules. Currently, they use this thing, but it has a lot of limitations: https://www.medicalholodeck.com/en/

Medical Virtual Reality for Surgery Planning and Education

DICOM Viewer, human anatomy lab and 3D anatomy models for teamwork in virtual reality (VR) .

Oomn0mn0m For context: currently in med school. I'm working with the anatomy dept to creat...

andybak•8/21/22, 8:43 AM

Ah yes. I remember now.

andybak•8/21/22, 8:43 AM

Did you do some stuff with displaying volumetric textures as well?

Oomn0mn0m For context: currently in med school. I'm working with the anatomy dept to creat...

omn0mn0mOP•8/21/22, 8:43 AM

The goal is to use Open Brush's more robust drawing tools to be able to highlight structures (arteries, nerves, surgical landmarks) and then compare what the student draws with an answer key drawing

andybak•8/21/22, 8:45 AM

We've added point cloud import recently. Adding DICOM support would be a nice addition too.

andybak•8/21/22, 8:45 AM

Import ALL the things!

andybak•8/21/22, 8:45 AM

(had the urge to add MagicaVoxel support the other day too...)

Aandybak Did you do some stuff with displaying volumetric textures as well?

omn0mn0mOP•8/21/22, 8:46 AM

I did not, but it looks like gltfast has an extension that adds support for it

Oomn0mn0m I did not, but it looks like gltfast has an extension that adds support for it

andybak•8/21/22, 8:46 AM

Oh nice. I was just looking at https://github.com/mlavik1/UnityVolumeRendering

GitHub

GitHub - mlavik1/UnityVolumeRendering: Volume rendering, implemente...

Volume rendering, implemented in Unity3D. Want to support my project? Donate some money to Red Cross and send me a screenshot/message/issue, and I'll be greatly motivated! :) Any amount is ...

Aandybak We've added point cloud import recently. Adding DICOM support would be a nice ad...

omn0mn0mOP•8/21/22, 8:46 AM

Lmao the dept chair has already suggested I do this next

Aandybak Oh nice. I was just looking at https://github.com/mlavik1/UnityVolumeRendering

omn0mn0mOP•8/21/22, 8:47 AM

Wait nvm, it's just a planned feature https://github.com/atteneder/glTFast/issues/209

andybak•8/21/22, 8:49 AM

For any importer that doesn't insist on async it's pretty trivial to add support. I think point cloud import took me half a day.

andybak•8/21/22, 8:49 AM

I want to replace the obj importer too so it works on the Quest.

andybak•8/21/22, 8:50 AM

(Parsing obj files is pretty damn simple. I got a basic one done from scratch for my generative models branch)

andybak•8/21/22, 8:51 AM

Of course - re-exporting is tricker if the result isn't a normal mesh

omn0mn0mOP•8/21/22, 8:52 AM

It's funny how Unity is supposed to be this amazing cross-platform engine, but then there's so many things that don't work with VR cuz of the quest being Android

omn0mn0mOP•8/21/22, 8:52 AM

Still trying to find a (FOSS) webview solution :)

Oomn0mn0m Still trying to find a (FOSS) webview solution :)

mikesky•8/21/22, 9:01 AM

Let me know when you do

omn0mn0mOP•8/21/22, 9:06 AM

So sad that they're all like $150

Oomn0mn0m So sad that they're all like $150

andybak•8/21/22, 9:11 AM

A webview is such a huge blob of code, it would be better if each platform exposed it as part of their API

andybak•8/21/22, 9:13 AM

but then there would always be a few platforms that didn't so we could never rely on it 100%. Which means I guess we'd probably go back to wanting an embedded solution

andybak•8/21/22, 9:15 AM

https://github.com/Voltstro-Studios/UnityWebBrowser

GitHub

GitHub - Voltstro-Studios/UnityWebBrowser: Functional web browser f...

Functional web browser for Unity without any issues or hassle. - GitHub - Voltstro-Studios/UnityWebBrowser: Functional web browser for Unity without any issues or hassle.

andybak•8/21/22, 9:15 AM

no android. damn

andybak•8/21/22, 9:15 AM

lol. been here before! https://github.com/Voltstro-Studios/UnityWebBrowser/issues/75

GitHub

[Issue]: Readme "Operating System Support" doesn't mention Android ...

What platform are you experiencing this issue on? Other What architecture is your platform? Other What version of UWB are you using? Git Repo What Unity version are you running? n/a Describe what t...

andybak•8/21/22, 10:07 AM

re: Layers. I decided to go ahead and implement "Objects remember what layer they came from and return there when deselected"

andybak•8/21/22, 10:07 AM

but because I'm lazy and it was tricky - I actually implemented "A selection remembers the layer that the last object added to the selection came from"

andybak•8/21/22, 10:08 AM

Which is interesting because it means that in simple cases things work as expected.

andybak•8/21/22, 10:08 AM

But if the user does this:

Select some stuff from Layer 1
Switches to Layer 2 and selects some more stuff
Deselects everything

andybak•8/21/22, 10:09 AM

then the stuff from Layer 1 moves to Layer 2. Hey presto - a way to move stuff to a different layer without needing a specific button.

andybak•8/21/22, 10:09 AM

It's a bit of an "easter egg" feature but I wonder if it's a rare enough use case and rare enough to do my accident that we can leave it in?

andybak•8/21/22, 10:10 AM

@bootleg @ancientworlds waddya think?

Bootleg•8/21/22, 10:11 AM

to me personally it sounds like it can easily go wrong for specific types of artists

Bootleg•8/21/22, 10:12 AM

I often struggle to keep up with layers in traditional drawing applications and finding out later that layer magic messed up my work is immensely annoying

Bootleg•8/21/22, 10:12 AM

considering that layer UI is even less visible in Open Brush, I'm concerned that this might end up messy

andybak•8/21/22, 10:12 AM

Yeah. I've also realised that my brain is too small to figure out how this interacts with undo.

andybak•8/21/22, 10:13 AM

Back to the drawing board! Looks like we need to track original layer for every item in a selection. Or - prevent selections from spanning layers entirely. Which might be intuitive.

Bootleg•8/21/22, 10:13 AM

Or - prevent selections from spanning layers entirely. Which might be intuitive.

I think this is for the best

BBootleg > Or - prevent selections from spanning layers entirely. Which might be intuitiv...

andybak•8/21/22, 10:14 AM

I actually meant to write "unintuitive"!

Bootleg•8/21/22, 10:14 AM

it's intuitive to me

andybak•8/21/22, 10:14 AM

i guess we need to make it clear to the user why they can't add to a selection after switching layers

Bootleg•8/21/22, 10:14 AM

I guess so

Aandybak @_bootleg_ @ancientworlds waddya think?

AncientWorlds•8/21/22, 11:04 AM

I think I'd have to try it to actually know for sure. Personally I'd prefer a clearly visible option so I know whether I'm spanning layers or working within a scope such as a layer or group. Probably more work. I think Bootleg is correct about visibility and confusion with layers, something I talked about in my long comment.
Probably sufficient to limit to layers, at least in the initial form, since I can always put an object on its own layer (I think). This would give control over things like brushes and color changes, would rather have that than an automatic global effect for things like brush/color changes.
I expect to be doing a lot of rebrushing when all these tools get synchronized together

AncientWorlds•8/21/22, 11:10 AM

AAncientWorlds I think I'd have to try it to actually know for sure. Personally I'd prefer a c...

andybak•8/21/22, 11:11 AM

when all these tools get synchronized together

Everything is in the beta. It's "testing how the tools work together" that we hope people are already doing!

andybak•8/21/22, 11:11 AM

Rebrushing has been released. And layers is beta

AncientWorlds•8/21/22, 11:11 AM

With matching view windows. Still a lot to figure out here, making progress.
Appears that actually placing additional cameras is quite a bit trickier.
Aside from the code aspects, the selector appears to use a single icon to choose the image.
Might be a good place for a config file entry that changes the relevant values directly

Aandybak Rebrushing has been released. And layers is beta

AncientWorlds•8/21/22, 11:12 AM

Ya, is definitely about testing the tools working together.

andybak•8/22/22, 10:24 AM

hey @omn0mn0m i can't figure out how to get the API "model.import" to work with GLTFast. Let me know if you get a chance to take a look.

andybak•8/22/22, 10:24 AM

the code was always hacky. i barely understood why it worked before

andybak•8/22/22, 10:25 AM

        [ApiEndpoint("import.model", "Imports a model from your Open Brush\\Media Library\\Models folder")]
        public static void ImportModel(string path)
        {
            path = Path.Combine(App.MediaLibraryPath(), "Models", path);
            var model = new Model(Model.Location.File(path));
            model.LoadModel();

            var tr = new TrTransform();
            tr.translation = ApiManager.Instance.BrushPosition;
            tr.rotation = ApiManager.Instance.BrushRotation;
            CreateWidgetCommand createCommand = new CreateWidgetCommand(
                WidgetManager.m_Instance.ModelWidgetPrefab, tr);
            SketchMemoryScript.m_Instance.PerformAndRecordCommand(createCommand);
            ModelWidget modelWidget = createCommand.Widget as ModelWidget;
            modelWidget.Model = model;
            modelWidget.Show(true);
            createCommand.SetWidgetCost(modelWidget.GetTiltMeterCost());

            WidgetManager.m_Instance.WidgetsDormant = false;
            SketchControlsScript.m_Instance.EatGazeObjectInput();
            SelectionManager.m_Instance.RemoveFromSelection(false);

        }

        [ApiEndpoint("import.model", "Imports a model from your Open Brush\\Media Library\\Models folder")]
        public static void ImportModel(string path)
        {
            path = Path.Combine(App.MediaLibraryPath(), "Models", path);
            var model = new Model(Model.Location.File(path));
            model.LoadModel();

            var tr = new TrTransform();
            tr.translation = ApiManager.Instance.BrushPosition;
            tr.rotation = ApiManager.Instance.BrushRotation;
            CreateWidgetCommand createCommand = new CreateWidgetCommand(
                WidgetManager.m_Instance.ModelWidgetPrefab, tr);
            SketchMemoryScript.m_Instance.PerformAndRecordCommand(createCommand);
            ModelWidget modelWidget = createCommand.Widget as ModelWidget;
            modelWidget.Model = model;
            modelWidget.Show(true);
            createCommand.SetWidgetCost(modelWidget.GetTiltMeterCost());

            WidgetManager.m_Instance.WidgetsDormant = false;
            SketchControlsScript.m_Instance.EatGazeObjectInput();
            SelectionManager.m_Instance.RemoveFromSelection(false);

        }

andybak•8/22/22, 10:26 AM

The good news is that merging your GLTFast stuff with the 2021.3 branch went pretty smoothly

[ApiEndpoint("import.model", "Imports a model from your Open Brush\\Media Library\\Models folder")] public static void ImportModel(string path) { path = Path.Combine(App.MediaLibraryPath(), "Models", path); var model = new Model(Model.Location.File(path)); model.LoadModel(); var tr = new TrTransform(); tr.translation = ApiManager.Instance.BrushPosition; tr.rotation = ApiManager.Instance.BrushRotation; CreateWidgetCommand createCommand = new CreateWidgetCommand( WidgetManager.m_Instance.ModelWidgetPrefab, tr); SketchMemoryScript.m_Instance.PerformAndRecordCommand(createCommand); ModelWidget modelWidget = createCommand.Widget as ModelWidget; modelWidget.Model = model; modelWidget.Show(true); createCommand.SetWidgetCost(modelWidget.GetTiltMeterCost()); WidgetManager.m_Instance.WidgetsDormant = false; SketchControlsScript.m_Instance.EatGazeObjectInput(); SelectionManager.m_Instance.RemoveFromSelection(false); }