Wildcard domain stuck issuing TLS
I have an app that uses a wildcard domain (multi-tenant), but I've been unable to get things working in Railway. Root domain works great, but even with multiple retries (removing the wildcard domain and DNS records, re-adding, etc.) things seem stuck in the provisioning TLS certificate stage.
Any help is greatly appreciated - we are trying to move our app off of Vercel for many reasons, and this is the final stage to build the proof of concept and make a decision.
Thanks!
79 Replies
Project ID:
77a7cdf4-e635-4e13-aba1-aaca47f04c1477a7cdf4-e635-4e13-aba1-aaca47f04c14
who's your DNS provider?
Cloudflare! i have turned off proxy for both cname records
app. is working perfectly. Seems to be an issue with wildcard. I've tried removing it and re-adding it - is there a way to manually provision things?
show me a screenshot of the railway domains please
what happens if you remove the wildcard domain from railway, and then add it back? (without touching dns in cloudflare)
^ So it's seeing the correct values in cloudflare
it technically shouldn't matter, those are random values and only serve as a means to resolve the correct ip
Got it. Yeah it seems this part is fine, but it gets firmly stuck on TLS issuing
Shall I update these values on Cloudflare and see what happens?
you may have hit the cert issuing limit
What's that? And is there a way around it? Or a way to resolve or debug?
i would have to flag the team
Ok! If you're able to help with this it would be hugely appreciated. We very ready to move our whole team over, but need to validate that this works as expected before we make the jump.
And it's a bit urgent for us because of performance issues on Vercel 😭
Thank you so much for your help!
Thread has been flagged to Railway team by @Brody.
may i ask if you are pro?
I am not yet, but as soon as we have this resolved we will migrate there
If moving to pro helps unblock, happy to do ti now
haha no thats not why i was asking, im not gonna ask you to upgrade while you are having issues.
just wanted to make sure you had the right discord badges is all 🙂
❤️ hahaha fair that would be a solid sales+support move 😂
But yes, we will be doing this shortly and moving the whole team over. Plz free us from Vercel!
Step change in performance already in our main app moving to Railway
glad to hear it, we welcome you to railway, and the community!
did you try end up trying this?
Yep! That's what lead to the above screenshot (I haven't changed anything since then)
okay thanks for confirming
np!
Hey!
https://help.railway.app/questions/wildcard-domain-stuck-issuing-tls-93093f7e was this you?
It's been flagged to our infra eng - appears to be an issue on our end. It's the start of the day in North America so it'll get looked at pretty soon, apologies for the delay
Railway Help Station
Wildcard domain stuck issuing TLS
I have an app that uses a wildcard domain (multi-tenant), but I've been unable to get things working in Railway. Root domain works great, but even with multiple retries (removing the wildcard domain and DNS records, re-adding, etc.) things seem stuck in the provisioning TLS certificate stage.
Any help is greatly appreciated - we are trying to ...
That's me too! yes!
Thanks so much @Ray ! And all good. Things have been so awesome otherwise.
And good morning 🙂
Hi!
You around I'd love to help you fix this
I'm not able to pull dns records for it
I can dig it and get the correct results back
What's correct result in this case?
these
Both these look incorrect...
I'm getting Required value:
g7t2czuh.authorize.railwaydns.net
(And, BTW the job to obtain it expired)
For some reason it only goes for 5 minutes
@SMT If you can retry issuing it super quickly that would be greati think they r&r'd the domain in railway and didnt update the dns in cloudflare, but updating the cname in cloudflare shouldn’t be absolutely necessary since they are just random cnames, right?
that would explain all the "stuck issuing tls" help threads ive always seen
Yea
Hi!
I'm back again
What did you want me to check?
Current DNS:
Based on the latest values given in Railway, but it's still stuck.
Let me know if there's anything I can try
Can you delete the wildcard for me?
And retry
From where? In Cloudflare?
Both
Yep. Doing it now
Basically retry issuing it from scratch as if you'd never done it
So delete it in Cloudflare and Railway
Then, let DNS purge
Then, hit create in Railway
I'll walk alongside you and figure out wtf is wrong
Ok - back to this:
Ran a dig on both and the records are gone
Slick!
Okay, remove it on Railway
(Please)
Yep, done. Then re-added it in Railway, and just added records to cloudflare
dig now shows both entriesAnd we're back to this:
Okay! Lemme validate this...
Trying to pull the info sec
I see two txt records
solving challenge: *.onorder.xyz: [*.onorder.xyz] authorization failed: HTTP 403 urn:ietf:params:acme:error:unauthorized - Incorrect TXT record \"7-glVtTwZT9qY9qVnOolZMaO8V9iv0pqqEmrnoFBeho\" (and 1 more) found at _acme-challenge.onorder.xyz
Can you delete the TXT records?
Oh, cause you have an app.onorder.xyz + a *.onorder.xyz
@SMT Sorry for tag it's time sensitive cause the workflow will retry
Can you tell me why your'e trying to do both here?
I wanted to get
app. working since that's where we host our main app
and wildcard wasn't working
Should I remove that?I think so ye
I think it's messing with the DNS but not 100% certain
Ok, done!
I think the issue existed before that, but no harm in trying!
Dunno where these guys are coming from
That's super weird
Yeah. Reading that thread to see if we can do anything
Ohhhh cloudflare has universal SSL
verified by TXT
Ah
Yea you'll have to turn that off
Try again?
This would be a great one to add to docs - I didn't turn this on, maybe new cloudflare default
If that's the issue defs
I'll try and dig the record
(The thing retries on a set schedule. Next one is in 8 minutes)
Looks better!
Ha
Beat me to it
!remind me to check back in 8 minutes
Got it, I will remind you to
check back at Tue, 27 Feb 2024 20:24:34 GMThaha
awesome. Will check back shortly. Thanks
Hey @JustJake, remember to check back - https://discord.com/channels/713503345364697088/1212038626372751450/1212131181437648897
Marked as successful on our end!
Can you give it a poke and check on yours?
Oh damn!!!
it worked!
Thanks!
Great one to add to any docs and notes about cloudflare 🙂
But all makes sense!
Thank you so much!
super helpfull for me too, now i know where to look when a similar issue with wildcards happen, thanks cooper!
wow he's good https://github.com/railwayapp/docs/pull/449#issuecomment-1967540615
Just a real 👏 support experience all around
GIVE THESE PEOPLE A RAISE!
Hahah thanks so much. Team is taking a final look and we'll migrate everything over. Everything is working great.
Solution