R
Railway•14mo ago
waltcow

railway internal network is unreadable

No description
Solution:
try setting the ENABLE_ALPINE_PRIVATE_NETWORKING=true envvar per the docs on all the alpine containers?
Jump to solution
26 Replies
Percy
Percy•14mo ago
Project ID: 1f736e16-6743-4b65-ac36-e7b8aca242da
waltcow
waltcow•14mo ago
projectID : 1f736e16-6743-4b65-ac36-e7b8aca242da
Brody
Brody•14mo ago
is that the correct domain name for the service?
waltcow
waltcow•14mo ago
No description
Brody
Brody•14mo ago
at what point in your apps lifecycle do you make a call this this internal service
waltcow
waltcow•14mo ago
exactly the internal domain i use frontend web framework nextjs, and proxy api/* the backend internal sever call 'dify-flask'
waltcow
waltcow•14mo ago
No description
Brody
Brody•14mo ago
I'm sorry but that doesn't quite answer the question
waltcow
waltcow•14mo ago
it use nodejs proxy to backend api server log show dns lookup fail
Brody
Brody•14mo ago
at what point in your apps lifecycle do you make a call this this internal service
waltcow
waltcow•14mo ago
Brody
Brody•14mo ago
okay thank you has this exact code worked in the past?
waltcow
waltcow•14mo ago
yes
Brody
Brody•14mo ago
have you tried a redoployment?
waltcow
waltcow•14mo ago
redoployment twice . both frontend and backend
Brody
Brody•14mo ago
@fp could there be a problem with private network resolution?
Faraz
Faraz•14mo ago
Interesting. Let me raise internally and find out.
char8
char8•14mo ago
Is the container using alpine by chance? Or musl libc? The DNS resolution for that is slightly different and we have a note in the docs about it: https://docs.railway.app/reference/private-networking#workaround-for-alpine-based-images (We're working on a fix for that particular problem)
waltcow
waltcow•14mo ago
FROM node:18.17.0-alpine
Solution
char8
char8•14mo ago
try setting the ENABLE_ALPINE_PRIVATE_NETWORKING=true envvar per the docs on all the alpine containers?
waltcow
waltcow•14mo ago
OK, i try it now
char8
char8•14mo ago
to give some background, we put both our internal DNS server and a public DNS resolver in resolv.conf . The dns lookup functions in glibc will treat the second entry as a fallback (desired behaviour since we want public DNS to work even if our internal resolver is unavailable), MUSL libc (used by alpine) is different: it sends out parallel requests to all resolvers in resolv.conf and errors on the first failure. Because public DNS doesn't have the railway.internal zone, privnet lookups can fail on Alpine.
waltcow
waltcow•14mo ago
it works now
char8
char8•14mo ago
what the magic envvar above does is it removes the fallback resolv.conf entry, the only caveat with this is that it means public DNS resolutions are only available with privnet initialization (which is usually ~ 100ms after container startup). Again this sucks, so we're actively playing with a fix for it
waltcow
waltcow•14mo ago
network issue occur again today
at connResetException (node:internal/errors:720:14)

at Socket.socketCloseListener (node:_http_client:474:25)

at Socket.emit (node:events:526:35)

at TCP.<anonymous> (node:net:323:12)

at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {

code: 'ECONNRESET'

}

Error: socket hang up

at connResetException (node:internal/errors:720:14)

at Socket.socketCloseListener (node:_http_client:474:25)

at Socket.emit (node:events:526:35)

at TCP.<anonymous> (node:net:323:12)

at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {

code: 'ECONNRESET'

}
at connResetException (node:internal/errors:720:14)

at Socket.socketCloseListener (node:_http_client:474:25)

at Socket.emit (node:events:526:35)

at TCP.<anonymous> (node:net:323:12)

at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {

code: 'ECONNRESET'

}

Error: socket hang up

at connResetException (node:internal/errors:720:14)

at Socket.socketCloseListener (node:_http_client:474:25)

at Socket.emit (node:events:526:35)

at TCP.<anonymous> (node:net:323:12)

at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {

code: 'ECONNRESET'

}
@char8
Brody
Brody•14mo ago
do not ping the team #🛂|readme #5
Want results from more Discord servers?
Add your server