mastra cloud works until it doesn't
Hello, I can deploy to my mastra cloud, and it will work for a while. Then eventually stop working until I re-deploy with this error in the logs:
1:28:41 pm remote: Invalid username or token. Password authentication is not supported for Git operations.
fatal: Authentication failed for 'https://github.com/flippyhead/copa.fyi.git/'
error
1:28:41 pm Process failed: ExecaError: Command failed with exit code 128: git fetch origin main
remote: Invalid username or token. Password authentication is not supported for Git operations.
fatal: Authentication failed for 'https://github.com/flippyhead/copa.fyi.git/'
error
1:28:41 pm Build pipeline failed: {"message":"Command failed with exit code 128: git fetch origin main\n\nremote: Invalid username or token. Password authentication is not supported for Git operations.\nfatal: Authentication failed for 'https://github.com/flippyhead/copa.fyi.git/'","details":{"message":"Command failed with exit code 128: git fetch origin main\n\nremote: Invalid username or token. Password authentication is not supported for Git operations.\nfatal: Authentication failed for 'https://github.com/flippyhead/copa.fyi.git/'","domain":"BUILDER","category":"USER","details":{}},"code":"GIT_FAIL_FETCH_BRANCH"}
error
1:28:41 pm Command failed with exit code 128: git fetch origin main
remote: Invalid username or token. Password authentication is not supported for Git operations.
fatal: Authentication failed for 'https://github.com/flippyhead/copa.fyi.git/'
10 Replies
This is my deployment ID: 235919b4-2d93-4d9c-9799-cbdd09cb57f7
We have to just keep hitting deploy in order to keep this production site up and running.
I'm not sure what we might be doing incorrectly...
And now of course we're getting 521 errors
and now 524 errors
It says it deployed correctly. It occasionally works. Then just stops.
Here's the error body:
đź“ť Created GitHub issue: https://github.com/mastra-ai/mastra/issues/9603
GitHub
[DISCORD:1433916422743920743] mastra cloud works until it doesn't ...
This issue was created from Discord post: https://discord.com/channels/1309558646228779139/1433916422743920743 Hello, I can deploy to my mastra cloud, and it will work for a while. Then eventually ...
@roamin I hate to bother you but for production issues, not sure what else I can do
Well, we gave up. Deleted our project and re-created it, now it seems to work.
and now it's not working again with 527 errors
If anyone else hits this. The likely culprit is an expiring credential (fine‑grained PAT with a 7‑day limit, a revoked GitHub App token, or a repo permission change).
We ended up refreshing our git connection to mastra and it seems to be working now.
I never had to do this with any other providers so who knows..
And it's not working again in the same way, but without the git errors
Restarting studio (whatever that is) does NOT fix the problem. ONLY deploying does.
đź“ť Created GitHub issue: https://github.com/mastra-ai/mastra/issues/9696
GitHub
mastra cloud works until it doesn't · Issue #9696 · mastra-ai/mastra
This issue was created from Discord post 1433916422743920743: Hello, I can deploy to my mastra cloud, and it will work for a while. Then eventually stop working until I re-deploy with this error in...
Hey @flippyhead ! Sorry it's taking some time to look into your issue, we have additional questions.
Did this just suddenly happen, was it working before?
Could you share your project's slug? (can't find a project called
screeching-sparse-greece)
Are you guys using cloudflare? If yes, could you share your logs?I had to delete and re-create my project to get rid of those separare errors around git auth. The current one is numerous-hundreds-machine
We do not use cloudflar we connect directly to our mastra instance as described in your docs
We have ALWAYS had a problem where, after a while, mastra cloud stops working
Before, it would eventually start working again, and I assumed it was due to servers spinning up after lack of use
The in the last week or so, they stopped spinning back up at all and we get those 52X errors
Right now it seems to be working without failing for around 48 hours now, which is totally new.
But that could be that we have a service that just pings an endpoint every 5s and, lacking that, maybe it would continue to fail after a while
Pods do get shutdown after 1hr of inactivity, but they should restart when you start query them.
When you say it stops working after a while, do you mean it doesn't restart, are you getting any errors you could share?
I'll pass the rest of the info you provided to the team, thanks!
If I wasn't clear:
It shuts down after a while (I guess an hour) then NEVER restarts and has NO errors.
Before it would EVENTUALLY restart, but that often took on the order of minutes, causing most people to assume it was broken.
In the last ... 2 weeks it has started NEVER restarting WITHOUT a deployment
I beleive we have a work around for this by just pinging the server every few seconds.
And we are permanently down again