Performance breaks by unknown reasons
Hello, our production self-hosted zitadel is doing misterious things in the kubernetes. Out of nowhere he start big read/write operations and by that it stops working for about 5minutes till the process is done and then it functions normally again. is it doing some kind of maintance in the background or why is it doing that?
8 Replies
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
Thanks, is it possible to get an answer today? our production is having some outages because of this :). thanks
@boblack_zocker apologies for what you are facing we are still looking into this , we will get back to you.
Unknown User•3mo ago
Message Not Public
Sign In & Join Server To View
@adlerhurst what do you mean?
i have some errorlogs but not more
its running in its own pod in kubernetes
it started with this logs:
2025-02-24 11:10:56.494 time="2025-02-24T10:10:56Z" level=info msg="process events failed" caller="/home/runner/work/zitadel/zitadel/internal/eventstore/handler/v2/handler.go:413" error="context deadline exceeded" projection=projections.instance_domains2025-02-24 11:10:56.310
time="2025-02-24T10:10:56Z" level=info msg="process events failed" caller="/home/runner/work/zitadel/zitadel/internal/eventstore/handler/v2/handler.go:413" error="context deadline exceeded" projection=projections.device_auth_requests2 2025-02-24 11:10:56.096
time="2025-02-24T10:10:55Z" level=info msg="query failed" caller="/home/runner/work/zitadel/zitadel/internal/eventstore/repository/sql/query.go:129" error="timeout: context already done: context deadline exceeded" 2025-02-24 11:10:56.074
time="2025-02-24T10:10:55Z" level=info msg="query failed" caller="/home/runner/work/zitadel/zitadel/internal/eventstore/repository/sql/query.go:129" error="context deadline exceeded" is this a problem with zitadel or the database of zitadel?
time="2025-02-24T10:10:56Z" level=info msg="process events failed" caller="/home/runner/work/zitadel/zitadel/internal/eventstore/handler/v2/handler.go:413" error="context deadline exceeded" projection=projections.device_auth_requests2 2025-02-24 11:10:56.096
time="2025-02-24T10:10:55Z" level=info msg="query failed" caller="/home/runner/work/zitadel/zitadel/internal/eventstore/repository/sql/query.go:129" error="timeout: context already done: context deadline exceeded" 2025-02-24 11:10:56.074
time="2025-02-24T10:10:55Z" level=info msg="query failed" caller="/home/runner/work/zitadel/zitadel/internal/eventstore/repository/sql/query.go:129" error="context deadline exceeded" is this a problem with zitadel or the database of zitadel?
Unknown User•3mo ago
Message Not Public
Sign In & Join Server To View
we got the default.yaml zitadel config
what exacly do you need to see on the query insights?
Unknown User•3mo ago
Message Not Public
Sign In & Join Server To View