Unexpected crash - Rust crashes after building with libgssapi-krb5 reference

Hey, first time posting an error - my Railway deployment is crashing with the following error:

error while loading shared libraries: libgssapi_krb5.so.2: cannot open shared object file: No such file or directory

The deployment for this code was working fine until I added the diesel dependency in my Cargo.toml. I first needed to add libpq-dev as an apt dependency (expected this one) to fix build errors, and then I got it to build and deploy, and ran into the crash error mentioned earlier.

Here is my Cargo.toml for the project:

[dependencies]
actix-web = "4"
actix-cors = "0.6.4"
actix = "0.13.0"
actix-web-actors = "4.2.0"
reqwest = { version = "0.11.14", features = ["json"] }
dotenv = "0.15.0"
serde_json = "1.0.94"
serde = { version = "1.0.155", features = ["derive"] }
diesel = { version = "2.0.4", features = [
"postgres",
"chrono",
"uuid",
"r2d2",
"serde_json",
] }
env_logger = "0.10.0"
rand = "0.8.4"

Here is my nixpacks.toml (need libpq-dev for Diesel's postgres feature):

[phases.setup]
aptPkgs = ["...", "libpq-dev"]

Thank you, any help would be appreciated!

ed084bf3-fd73-4163-a606-254021daaf02

try adding libgssapi-krb5-2 to your aptPkgs array

oh thank you for the suggestion @Brody I will try that
ok heres what I have now
[phases.setup]
aptPkgs = ["...", "libpq-dev", "libgssapi-krb5-2"]
just pushed!
have you ever seen this error before? any idea where it comes from?
curious what I've done right on my local windows machine to make this work well
lol
also i just got the project redeployed and im getting the same error
i wonder if there is potentially a cache I need to clear or if the dependency is not the root of the problem?
i just removed the deployment and I am trying again

i have not seen it, error comes from a package not finding libgssapi_krb5.so.2
the issue is the final deployment is missing libgssapi_krb5.so.2

right which should come if I have that dependency

yeah keyword, should

hmmm
i was looking for a way to verify it is running the apt-get commands for the packages
it looks like in build logs it is saying layer already exists

your nixpacks.toml file is correct

ok
yeah the delete + redeploy gives the same problem

[phases.setup]
aptPkgs = ["...", "libpq-dev"]
nixPkgs = ["...", "libkrb5"]

try that

yessir

of course it would

🤞
im super impressed with railway btw
you guys are amazing

i dont work for railway, but i still appreciate that

damn same problem
im tempted to stop using diesel and just write sql queries
ive had so many problems with this ORM

ive seen so many problems with rust just wanting random libs installed

it seems that way sometimes yeah
i did have to completely remove diesel trying to build an android app with tauri
rust runs into some problems, but with diesel its a whole other level

the only other solution i can think of would be move to a dockerfile, but im not confident in my ability to write a dockerfile for rust

i have one that was written by a tauri dev
i can probably modify it
would be good to get to know docker better anyways

i could maybe help? ive never done rust before

yeah lemme see what I have
i can paste longer files in here right?

you can send the file as a file?

like text
oh sure
still looking I think I already worked on removing android portions of the file
cus the dockerfile was originally set up to cross compile to android
ill just share the repo i found it on cus im not able find my modified file i think i figured i didnt need docker
https://github.com/simonhyll/devcontainer/tree/v2
im going to go through and delete all the android related stuff

yeah that does a whole lot of things you dont need to do

right right
dont need pnpm
dont need java/tauri/mold
do you know what the devcontainer.json file does?
doesnt look like i will need anything there

i do not, sorry

i trimmed it down a bit

its installing node?

ooh
right
this is just my server
LOL
thank you

it also is manually downloading rust, just use a rust image

ooh
would i include a link to some public image?

im sure rust has published an image on the docker hub

you are indeed correct

what version of rust do you have installed locally?

rustup 1.26.0 (5af9b9484 2023-04-05)
info: This is the version for the rustup toolchain manager, not the rustc compiler.
info: The currently active rustc version is rustc 1.71.0 (8ede3aae2 2023-07-12)
https://hub.docker.com/_/rust

then use this image rust:1.71.0-slim

so id say like FROM rust:1.71.0-slim

yep!

sorry is that it?
ive never added my own new like to a dockerfile 😄
just delete and change things in a line lol

well youd need to add the run the commands to build your rust app and then run it

ohh
so with docker
i go away from nixpacks

correct

and i wont get to use the command in the railway setup
i see thank you!

think of a dockerfile as a bunch of commands that you would run to download the dependencies, build the app, and run the app from scratch
of course you dont need to download rust because thats what that rust image has already done for you

yeah that makes sense
im not sure where my working directory would be
should i just run the container and get a feel for the environment?

nah WORKDIR /app

oh i see
app would be the name of my folder?

the workdir has nothing to do with your project folder, its just simply where files will be worked on and copied to when in the docker image
keep it /app

oh alright
theyre handling some of the for me

whos "theyre"

docker i guess
if its a monorepo and my server is in a folder called server relative to parent directory
i should do WORKDIR /app/server and then run the commands?

workdir has nothing to do with your project folders or structures, its just the folder withing the docker image that you will be working on files
you would have set your root directory as /server in railway already

oh wow

and this dockerfile would be in your /server folder in the repo

ok that is great
ok im just going to see what railways got
WORKDIR /app

RUN cargo run
should be good right? 🤣

well what does cargo run do

it runs my app
if youre in the server folder

then no that wont work
you need to build the app into a binary file

ok sorry it it builds and then runs it

still wont work
^
without running it
we do not want to run it during build

ok i havent built a binary before i think its pretty easy though

youd know the command to do that better than me
i know how to write dockerfiles, but i cant if i dont know the commands to build a rust bin

i have a .exe from cargo build
bin...
hmm

yeah thats fine, in the image youd get a bin since its linux

https://doc.rust-lang.org/cargo/commands/cargo-build.html
oh great

make sure *.exe is in your gitignore file

yeah
it creates a target folder with a ton of stuff in it
including the .exe, which is gitignored

does cargo build also install the crates?

yes

keep that target folder in your .gitignore too
that target folder is platform dependent, and we dont put platform dependent stuff in the repo

ok thats all set

send dockerfile

sure

you still got a crap ton of stuff fron node and tauri?

oh you mean the apt stuff?
i am paranoid i dont have enough dependencies LOL
i know it doesnt work like that

also, please dont be pushing every little change you make on the dockerfile to railway, ill let you know when i think the dockerfile has a chance of running

i think you need a few of those for rust
ok i have been keeping it local for now

give me the path to the exe when you run cargo build --release

one sec i gotta run it locally with release flag
nearly there

damn rust takes that long to build??

it does lol

yikes

especially release builds

bruh go takes 15 seconds tops.. uncached

when its cached and there are only small changes it is quick
but i havent run release build
LOL
i was considering learning go, but i just like the orange highlighted type annotations in my IDE
you know how it is
target/release/battle-plains-server.exe

type annotations?

i could have assumed it would be that

lol

what ide?

vscode
it only happens when i use rust

is that an extension?
ive edited rust with vscode and it didnt do that lol

i think its the rust-analyzer extension haha

i had that installed at the time

oh interesting

getting off track though
use this

FROM rust:1.71.0-slim

WORKDIR /app

COPY . ./

RUN cargo build --release

ENTRYPOINT ["/apptarget/release/battle-plains-server"]

should I still keep any of the other stuff or is that literally all I need?

your dockerfile had too much stuff, if we need apt packages, we can add that back later
all you need for now

wow

you can now push that to railway
yeah you can see what i meant by "ton of crap" lol

COPY . ./
does that get the code from github or something?

yeah how else are you gonna get your code into the image

ah yes it wants apt packages

well railway will pull your repo into a temp folder, so COPY just copys the code from the same directory as the dockerfile
do tell
screenshot

libssl-dev, pkg-config
screenshot better

pictures are worth 1000 words

itd be a few screenshots
would you want text?

one sec
https://bookmarklet.up.railway.app/log-downloader/
use that

holy shit
what

what?

thats so cool

i made that 🙂

damn dude
they need to hire you what are they doing

railway wouldn't do it, so i had to
im not qualified

🤣
maybe soon

its beautiful
ryuko_bot_discord?
or the exe haha

would be even nicer looking if we used an alpine image
linux bins dont have an extension lol

alpine?

dont worry about alpine
out of the scope lol

i am worried about alpine
but later
I changed it to ENTRYPOINT ["/app/target/release/battle-plains-server"]
i assumed ryuko_bot_discord was something else

fine, the image you are currently using is based on debian, the rust image with alpine in the name is based on alpine, just a different linux distro

oh i see
you are too helpful

omfg

i have two dockerfiles open and i sent you the wrong one

hilarious the only thing i thought was messed up was the entrypoint

thats the only thing that was different lol

and the copy line
COPY . .

true

it was COPY . ./
but maybe thats ok

same thing, its just semantics

im going to laugh very hard if we end up having the same list of apt dependencies
as the original lol

its possibly

well its also good to go incrementally to be lean

getting fancy!

alpine

it seems to be running much longer than the other tries

awesome

the rust is building

pog

how long have you been writing software?

writing software is a stretch
I just play around

haha
you are getting deep into rusts scary side

heres that msg + another look at a similar error msg

ive seen that before

use the bookmarklet please

oh ok

looks like min we need libpq-dev

yeah but that's an apt package
apt isn't applicable with alpine

ohh

alpine uses apk (not to be confused with android .apk files)

i found this https://pkgs.alpinelinux.org/package/edge/main/x86/libpq-dev

cool
though we will probably need that, the error we see isn't caused by not having that

yeah curious what you think about that
ive seen the linking with cc before

I'm in the process of thinking, but what have you done in the past to fix it

it mentions -lpq on the second note (after linking with 'cc' failed)
so i looked that up and thats where people mention postgres
i think thats where i added libpq-dev

FROM rust:1.71.0-alpine3.18

RUN apk add libpq-dev openssl-dev musl-dev

WORKDIR /app

COPY . ./

RUN cargo build --release

ENTRYPOINT ["/apptarget/release/battle-plains-server"]

let's try it, I have a backup plan if that doesn't change anything

legend
i wonder if we can look at what nixpacks is doing for rust

something cringe probably

cringepacks

real

not a fan of nix
dudeee the image is publishing
poggers

build worked!

yesss
i think its gonna work
i think it was a nixpacks problem

it's fine until you ask it to install an apt package that would normally fix the build in a dockerfile
there's 100% a way to get it working with nixpacks, I just don't know that way

yeah i want to share a stackoverflow post i wonder if it highlights the problem somehow

a stackoverflow post for what?

regarding the error i had on nixpacks build

that's what your here for right now, stackoverflow can't help lol

omg it deployed
now for your satisfaction this is what you got online

now does it work though, my money's on you forgot some environment variable

haha awesome
even got a custom domain
but yuck www

ohhh
yeah i havent done the domain right

what's your provider

google domains
🙂
lol

oof

well i think they got bought

Google domains doesn't support root level cnames
or whoever bought them out doesn't either

you can just use cloudflare's nameservers, no need to do a domain transfer

im down to check that out for sure
oh wait shit it crashed

it's simple, run through the setup process in cloudflare and they will transfer everything over

odd it doesnt say much in the deploy logs

sadge

why does it say compiling?

i wonder if my cargo run in the railway setup
is conflicting

yes it is
remove please

son of a
so no start command

correct

everything else should be good
and now i can just hit redeploy

for you, the ENTRYPOINT is like a start command

yeah i saw the compiling
and was like o shit
well
you did
good catch lol

also, dont log your database connection string
youll want to go and regen that

yeah i might want to remove that

free database

free database anyone?
huahuaha

in the plugins settings

ok im good
on the db
im guessing that was it on the deployment as well
it was running

so has it crashed?

well
sometimes you forget to hit the save button
then you gotta run the rust build again
ill know in another couple minutes
lol

how many crates are you using

[dependencies]
actix-web = "4"
actix-cors = "0.6.4"
actix = "0.13.0"
actix-web-actors = "4.2.0"
reqwest = { version = "0.11.14", features = ["json"] }
dotenv = "0.15.0"
serde_json = "1.0.94"
serde = { version = "1.0.155", features = ["derive"] }
diesel = { version = "2.1.0", features = [
"postgres",
"chrono",
"uuid",
"r2d2",
"serde_json",
] }
env_logger = "0.10.0"
rand = "0.8.4"
isnt that wild

hey thats not even a lot
https://github.com/railwayapp/cli/blob/master/Cargo.toml

ah got another failed one
i didnt push code just updated the db string and removed cargo run

updated the db string?

well in my env
after i had to change it

oh no, you arent using variable references like you should be, are you?

LOL
they even say that....
yeah im not

they do
you should be

well thats very convenient isnt it
interesting the build fails 'failed to create task for container'

it sure it
"/apptarget/release/battle-plains-server"
that doesnt look right

oh snap
should be target

send the dockerfile back please

FROM rust:1.71.0-alpine3.18

RUN apk add libpq-dev openssl-dev musl-dev

WORKDIR /app

COPY . ./

RUN cargo build --release

ENTRYPOINT ["/apptarget/release/battle-plains-server"]

ok were deployed again!
it seems to be spamming like its running the app over and over again
like running my init logic
every 5 seconds or so

railway will restart if your app exits
or crashes

rust are you crashing quietly?
lol
dude you are a champion
im going to go to bed, this is a great point to be at

sounds good

thank you so much again
❤️ ❤️ peace bro
btw it is crashing on my db connection for sure im not sure why but i have a place to debug which is almost def on my side

let me know how it goes when you pick this back up

Hey @Brody I ended up working more on it and got stuck, seems like the deployment is having trouble connecting to the DB
I tried it local with the same connection string and it worked

makes sense
go into the service settings and disable private networking

I think it's disabled already
Says 'enable private networking' in the UI

okay then let's see a screenshot of your service variables

here you go kind sir

okay, let's see the error message

It doesn't have an error msg per se
It crashes after it fails connecting

ya gotta log your errors
can't do much debugging without error logs

I am actually not sure how to log this error, I have tried and ended up settling with a msg after that says 'no error yet...' lol
I can spend some more time trying to get an error

yeah we need an error message to proceed
and an actual error message, not just "there was an error"

Yeah

let me know when you have an error message

idk if this helps but im using a connection pool to try and share the db connection around my app
let pool = Pool::builder()
.max_lifetime(Some(Duration::new(1800, 0)))
.connection_timeout(Duration::new(30, 0))
.max_size(15)
.build(manager)
.map_err(|err| {
println!("Error: {}", err);
Box::new(err) as Box<dyn Error>
});

println!("No error yet...");
im at work rn will have to wait til after to get the error msg worked on
and the map_err portion seems to do nothing on error lol

looks fine, the pool builder just creates the pool and doesnt connect to the database, so its understandable that its not causing the crash
sounds good, ill be around

hey @Brody for whenever you are online next I found this https://github.com/diesel-rs/diesel/discussions/2947
explains that the code exits without any message or log
then suggests a fix installing some dll

thats a solution for windows

yeah
i cant get it to spit any error msg

that's very likely your issue, you just need to find a solution for linux

and then its all about adding this process to the dockerfile somehow?

perslaps

perslaps ma naps

well cross that bridge when we get to it

diesel makes me sadge and cringe

same, but for rust
every rust help thread is a missing lib
if rust is so good why does it rely on so many c libs?

lol you do speak the truth
but!
i managed to get it to work I decided to try the non-alpine dockerfile
i added a couple more deps that it was asking for
heres the final file

FROM rust:1.71.0-slim

RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
openssl \
libpq-dev \
libssl-dev \
pkg-config

WORKDIR /app

COPY . ./

RUN cargo build

ENTRYPOINT ["/app/target/debug/battle-plains-server"]

no --release?

WELL
oops caps
well its slower i think
i will probably update to --release after
i think --release just does some optimizations for runtime but its the same program basically

something like that yeah

now my game is multiplayer 🙂
so tldr
worth trying go?
hahaha

YES!

ily brody
thanks so much
i would tip u if i could
absolute chad

haha thanks!