Fixing flake build on NixOS (hopefully)
build is failing because a custom build script for
stereokit-rust tries to run a non-nixos patched skshaderc which unsurprisingly fails.
it seems like there is a patch intended to fix exactly this on line 55 of /nix/stardust-xr-server.nix which i think references /nix/sk_gpu.nix and attempts to patch its fetched sk_gpu release.176 Replies
sounds right
i think i'm finally getting a bit closer to actually understanding how this flake works, but i'm still extremely confused by a few things.
i'm fairly sure most of my confusion is down to the fact that i don't have very much actual experience with nix.
for the time being i'm going to work on porting all of the rust environments of my personal projects from
shell.nix to flakes, then try to learn how to patch and package things for nix in general, rather than continuing to try to parse files i don't really understand.
doing all of that might take me a few days, but after that i'll come back and give this problem another shot
so it turns out porting my stuff was significantly easier than expected, because it was all pure rustyour stuff?
my personal projects i mean
ah cool
:3
that i was using nix-shell for originally
:anim_YIPPEE:
i hoped that by porting them i would get a better understanding of what to do with the stardust flake
but the problem i keep banging my head against is that stereokit uses cmake
yes that's what I went through too
hmm, maybe if one of them has time...
i think the skshaderc patch is working, but i can't even get to that point because cmake tries to pull from github, which is impossible during the build phase on nix
@Pandapip1 @matthewcroughan are the main 2 nix people in the server
hopefully they're available to help because nix is not my forte
fair, i'm quite bad at nix myself currently lol
CPM local packages not working huh? mood tbh
wait... i might be being an idiot, let me try a few things
Hehe, I just went on a VR journey actually, so some things are fresh in my mind
btw matthew we are switching off stereokit asap so you'll have pure crate dependencies hopefully soon
😄
should avoid half the BS you had to pull to get it working
but that's not ready yet
Yeah, and then ideally stardust doesn't have a flake and it's just in nixpkgs
oh wait flakes are bad now?
no, having any form of packaging in your repo might be bad though
instead you should just allow ecosystems to package your software, and resolve issues with that
instead of trying to package anything first-party
ah fair
well... given most people are too busy to do that i gotta do something myself
:p
but afaik nixpkgs already has stardust
I think of two approaches, either go 100% all-in on one way, or go 0% in
both are good
0% in means you don't maintain any packaging code in the repo, which is fantastic. Packaging outside only. The downside is testing suffers.
hey look at that
https://search.nixos.org/packages?channel=24.11&from=0&size=50&sort=relevance&type=packages&query=stardust
looks like the stuff is in there
Downside of that is that it's not up to date
and with the way VR develops, old code is awful
to be fair, we haven't had a release since then
ah, that's okay then
so can I run sway with my glasses now?
yes, i even added cursor support finally :p
and, like, actually use the keyboard?
yes, non-spatial-input works great
and maybe even spawn programs outside of sway, just in space
in addition to sway
pipe eclipse or manifold into simular and you put keyboard/mouse into any panel you look at
granted, that doesn't 100% work with the 3D apps since it's not azimuth
oh haha, so input is based on gaze
only simular
azimuth isn't gaze-based but i haven't updated it yet
3d apps?
been running around all over the place trying to stabilize things
what is a 3d app?
oh you meant like 2D things
like a terminal (kitty)
ok but moving them will be a challenge given i haven''t made sphereland yet
what is an example of a 3d app though?
hexagon launcher, comet
so should i still try to fix the flake?
the reason i wanted to fix the flake in the first place is for my own development stuff, and figured i should try to fix the existing flake and contribute it back
though idk how you could do comet without 6dof
I was hoping movement could be facilitated using fingers/pinching
yes for sure
I still think having a flake that is maintained will always beat something in nixpkgs
and we have that nice VM test I made to test stardust that would be a shame to stop looking at, and migrating as the setup changes
do your glasses even have cameras?
sadly not, it's just 3dof
well then there's only so much I can do :p
I just want to have multiple programs in space, I don't need 6dof I think
At least I want to try
and I know the xreals I have suck
alright, since you're here can i ask what exactly
$cargoDepsCopy is?
i literally can't find any documentation for it, i think i know roughly what it maps to, but knowing for sure would be extremely helpfulI want to upgrade to the next good glasses, do you know what they are?
ok then you really want sphereland and azimuth but i haven't had the time to make them yet because i've been so busy running around
unfortunately no, but let's move this convo elsewhere
I'm assuming this is the cargo dir
so nervyalloy and you can talk nix specifics
like. if you forget nix and everything and ran
cargo build, it would produce a directory where the sources, etc were fetched into
I think that's what cargoDepsCopy isyeah that's what i was assuming too, the
Copy bit was just throwing me a bit of doubt
wasn't sure if it was some special nix thing i haven't learned about yetI think it just means that it's a "copy of it"
yeah, that makes sense
i think i'm getting somewhat close to understanding everything in the flake
(with the exception of
gnome-graphical-test.nix which i don't even want to think about right now)well this is a vm test that spins up nixos/gnome/stardust/clients in a VM
then it tests that they work properly by executing them, and interacting with them
then it posts a screenshot of all of that having happened into #robot-palace
and that's just a nix build away
yeah, i understand what it does, i'm just really hoping i don't need to touch it
Well nobody pinged me lol
i forgor
forgor 😦
i got so used to it not updating that i forgot it existed because am so busy
all good
@matthewcroughan one more question if you don't mind, would the
postPatch in rustPlatform.buildRustPackage apply to running a normal cargo build inside a nix develop environment?
my assumption is no, but again i can't find much in the way of documentationonly if you run that phase
nix develop just gives you the buildInputs
it gives you all the tools you need to manually build a project, on your path
it does not execute any steps, or perform any actionsokay, thanks for the clarification
i got it to build!
not entirely certain it built correctly, but it does open a window
yippeeee
@Nova does the dev branch of flatland work currently?
just updated the flake, it compiles, but it fails the
resize_handles testuh shoot I don't think the tests work automated
ignore the tests if you can
alright, i'll see if i can figure out how to do that lol
btw the flatland flake was only broken because
crane.lib.${system} was deprecated and then removed in favor of crane.mkLib so i imagine that's also the case for some of the other flakesoh huh
feel free to PR everything
will do, planning to fix all of the flakes that i can
i just, uh, need to learn how to use github
my only experience with git is with repositories on my serverâ„¢ through ssh lol
do you know of anything else that has broken tests at the moment?
i found the variable i have to set to ignore them, but i'd rather not sit through two builds for each crate if i don't have to
uhhhh
decided that i'm just going to fix the easy ones for tonight, then tomorrow i'll learn github and make some PRs before trying to fix the more stubborn ones
yippeee
weirdly enough, i've found fixing these problems quite enjoyable
feels similar to the enjoyment i got from fixing my arch install whenever i managed to break it
and it certainly helps that once i'm done fixing these i get to mess around with stardust again
alright, haven't properly tested any of them yet, but i have
server atmosphere flatland gravity kiara magnetar poltergeist protostar and solar-sailer compiling again
and now, time to sleep:0
@Nova something is still very much wrong with most of those, i can't seem to get anything but atmosphere to create anything on the server, and even that looks a bit different than i remember.
judging by the server output, i think the issue is that the server (or client?) can't find any resources.
where does the server/client expect the resources to be?
more specifically, is it determined by the
STARDUST_RES_PREFIXES env variable?
if so, does the server's variable need to have the contents of each client's variable appended to it?
to clarify, whenever a client that includes a mkDerivation that creates a STARDUST_RES_PREFIXES env variable is built, it creates a unique path in the nix store.
for example, after checking env of the derivation created by the protostar and kiara flakes, here are their respective paths:
protostar: "STARDUST_RES_PREFIXES": "/nix/store/lh2qp403j65bd4xpjfjvmpl5i3amsn6m-resources",
kiara: "STARDUST_RES_PREFIXES": "/nix/store/9fp71cf4544am724mxky2s1gnc5gmaws-resources",
tl;dr, does the server need to know where the client resources are before the clients are started, or is there a protocol for the clients to send the server the relevant data at runtime (which i need to fix for nixOS)
(i hope all of that makes sense, i know i'm extremely bad at explaining things)yes
no
protocol
alright, thanks
What are you setting that env var to? It needs to point at the actual dir that holds the files, for most clients that will be
res, the default way for clients to register stuff probably wont work on nix since its relative to the cargo toml file, and i think that path differs for the server and the cargo build invocation?no I made it so on nix it copies the resources to the nix store by their hash
so you can use that for stardust res prefixes
we so need something better than the current resource system
i'm starting to think this might be more of a problem with my system configuration, rather than the flake
nix should eliminate that system config issue entirely
Not necessarily, only if the stardust flake provides a module that the user imports
oh
which it may not, I haven't looked at it in ages
okay, so it's not a system config problem. (i think i only assumed that because i'm so used to arch)
i'm hitting this problem again https://discord.com/channels/647222689777582080/659858947896770590/1085310466206732311
shoot uh
can you look to see what path it's giving the thing
everything works fine if i do
nix develop then cargo runalso make sure everything points to dev
like
everything has to depend on dev branches of everything else
also @matthewcroughan you wanted a monorepo.... this should hopefully be close enough
https://github.com/cyberneticmelon/StardustXR_Hangar#
it's a WIP
well, i'm only testing with the dev branch of server, and the dev branch of protostar (the single binary specifically, because it didn't spam the logs as much)
oh the single binary is wildly out of date
use hexagon launcher
sorry, been running all over the place to fix stuff so it's not all updated
not a problem, i'm perfectly happy that you're still working on this awesome project at all
huh, hexagon launcher doesn't work through the built binary or nix develop
what fails?
here's the server output from running
stardust-xr-server -f then cargo run --bin hexagon_launcher
oh my god that's a pretty fundamental failure
uhhhh
xkbcommon_rs::context: 100: NoId: failed to add default include path /usr/share/X11/xkbah shoot that'd do it so, need to figure out how to get that back or such
ah, right, i never actually fixed that...
this is why i need spatial reminders lol
:p
so as it turns out, i don't even have XKeyboardConfig on my system at all currently
:blobcatgoogly:
should prob add that as a dep for stardust
i kinda just assumed it was already installed lol
very weird that nothing else on my system has needed it though
i really need to find out if there's a way to get
nix build to not re-compile all of the rust dependencies for the server every time it run itthere's some caching mechanism but i forgor what it is
fairly sure every other repo is using crane for caching, but i wouldn't know how to incorporate that into the server flake
which is a problem because this compile has been running for like 20 minutes lol
okay, i still haven't made a proper fix for the xkbcommon path (because i don't really want to re-compile the server again right now), but starting the server with
XKB_CONFIG_ROOT set to the correct nix store path fixes it for now
with that workaround, i'm back to the old problem
hexagon launcher works perfectly with cargo run --bin hexagon_launcher inside of a nix develop shell, but the compiled binary still complains that it can't find resourcesshare the whole log where it complains
will do, going to put it in a text file instead of coping from my terminal this time though so 1s
:blobcatnod:
oh, uh... nope
that's much worse
gimme the whole log
just trust me
ok uh i am gonna need you to do something 10x worse
start it with
RUST_LOG=trace and gimme that file/build/9jw6dr13a9j5rxpx4b3b8g7jhcynjlj4-source/hexagon_launcher/../res
that is the path that it is using
so, you need to fix that
show me the flakefor protostar?
yea
didn't really change much, i only edited the crane bit
edited it how?
i changed
to
and removed
inputs.nixpkgs.follows = "nixpkgs"; from the crane input because nix stated the crane input was overriding a non-existent nixpkgshuh weird
so, you need the
STARDUST_RES_PREFIXES to be an env var at build time
maybe crane changed how that works?but then why would the path it's trying to use be mostly correct?
that's not correct at all
if
/build/ were changed to /nix/store/ it would be validthis path should be
/nix/store/gljhadsbngjlkasdhgnljhkadsbnfgjlhksad-res/
it makes a new derivation for the resources then bakes it into the thing/nix/store/9jw6dr13a9j5rxpx4b3b8g7jhcynjlj4-source/hexagon_launcher/../res does exist on my system with the correct contents thoughright, but it's not reliable enough because it doesn't always happen that way when building it
does it need to be the same between builds?
since the built binary has the nix store identifier (albeit with the wrong prefix), and that location exists on the filesystem, i'd imagine that identifier was the correct one at build time
it should be, yes
yeah so
nix derivation show consistently reports STARDUST_RES_PREFIXES as /nix/store/lh2qp403j65bd4xpjfjvmpl5i3amsn6m-resources, and the derivation is creating that path with correct contents. so i guess it must be that crane changed how env stuff works like you said
just weird that it's getting anything at all
i'll try to clean things and read up on the documentation for everything
now that you've given me an idea of how everything is supposed to work, i should hopefully be able to actually work out a fix from herei might just be missing something here, but is this line still correct? (in main.rs of hexagon_launcher)
should it not be
.setup_resources instead?hmmm
uhh might be using an old version
my protostar is up to date with the dev branch
and here's the link to
core/fusion/src/client.rs as referenced by protostar's cargo.lock
https://github.com/StardustXR/core/blob/d2964d8db079afaadb7faa4987e34814e62d6279/fusion/src/client.rs
which is after STARDUST_RES_PREFIXES was placed in setup_resources
if that isn't the issue i officially have no clue how to get the protostar flake to work
either way, i think it's about time that i stopped bashing my head against the brick wall that is nix documentation for the day
no need to rush and check that though, i'm going to sleepi think i did that, iirc i didn't find
setup_resources so i just used the direct one, i thought they were the samethey should be, yeah
no the one on the root just sends the paths to the server, it doesn't run the custom logic of setup_resources
oh shoot i refactored it, right!!!
so, i'm not crazy?
only as crazy as me
you know, i really should have checked the code before i re-wrote the flake twice
haha yeah don't assume I write the best code
but it gave me more experience with nix, so i can't complain
nah, i can't blame anyone for that bug
looks like it just got missed after a refactor
i might try to fix it myself soonâ„¢, but i'd like to get some of the other flakes working first
my friends are bugging me to play video games again, so i guess i'm putting this on hold for the day
fair
have fun haha
lol
alright, my requisite gaming hours have been satisfied for some unknowable amount of time, back to fixing flakes
i uh, think i made that PR correctly?
yes
i'm just testing to see if the AI is worth keeping or not
fair
there's not enough people on the project to code review rn so, i'll take the help i can get :p
understandable, just wanted to make sure i didn't goof something lol
does it run for you?
yes
what did the
xkeyboard_config dep fix again?i've been using the built binary from it for testing the other flakes
this ^
oh wait, no
it's still broken
my bad
need to actually fix the path for nix
whooops :p
i thought i might be forgetting something
uh, do i need to delete that PR and make a new one? (after i actually fix the thing)
or can i somehow add a commit
you can add a new commit to the branch and the PR auto-updates
because the PR is from the head of a branch to the head of another
makes sense, thanks
time to actually fix the thing i've forgotten twice now
alright, i actually fixed the thing this time
not certain that the method i used is the most correct nix way to do things, but it does now work properly
@matthewcroughan wanna double check or should i merge now?
I mean, if it passes ci then use it
let me know if anything needs changes, i'll be back in a bit after i make some food
is this normal rendering for atmosphere?
i seem to remember it looking a bit different, but i could easily be wrong
yea that's normal
alright, atmosphere flake is done then
flatland also seems to work correctly, apart from everything freezing after trying to spawn firefox
ah shoot that bug
funny thing is, i wasn't even trying to spawn firefox
i actually wanted to spawn feishin (my music player) but couldn't tell which was which on hexagon launcher through my vive resolution lol
ohno :p
(to be clear, i still haven't fixed that hexagon launcher bug, i'm just using a dev shell as a workaround for now)
ugh, i have to take another break from testing stuff
the rubber light blocker thing on my vive just fell apart and now i have to wait for the adhesive to cure
i should probably buy a new HMD at some point
can you use flatscreen mode? -f on the server
i could, but the piece doesn't want to stick in the right place, so i have to keep repositioning it every few minutes until it firms a bit more
and i know if i start doing something i'll forget to check it, and then it'll end up in just the right place to poke me in the eye or something
and after it's stuck back on i need to do another step to smooth out the cracks in the rubber
fun times
made PRs for both atmosphere and flatland, i have a few more flakes that have the same fix applied but i'd ideally like to test them a bit before creating PRs for them (even though i'm almost certain they're fine)
just wanted to say that i'm still planning on fixing all of the other flakes, i'm just dealing with life stuff currently that's kinda using up all of my proper brain function for now :(
no problem! we're in no hurry :)