Strange timeouts on Vercel

I'm connecting to Neon from Vercel Serverless Functions with @neondatabase/serverless and drizzle-orm/neon-http I get strange timeouts:

error saving to db when updating conversation NeonDbError: Error connecting to database: fetch failed
    at execute (/var/task/apps/web/.next/server/chunks/375.js:53:7255)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async lI (/var/task/apps/web/.next/server/app/haku/[id]/page.js:56:2168) {
  severity: undefined,
  code: undefined,
  detail: undefined,
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: undefined,
  line: undefined,
  routine: undefined,
  sourceError: TypeError: fetch failed
      at node:internal/deps/undici/undici:12502:13
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async execute (/var/task/apps/web/.next/server/chunks/375.js:53:7166)
      at async lI (/var/task/apps/web/.next/server/app/haku/[id]/page.js:56:2168) {
    [cause]: ConnectTimeoutError: Connect Timeout Error (attempted addresses: 3.126.212.11:443, 3.124.121.135:443)
        at onConnectTimeout (node:internal/deps/undici/undici:6635:28)
        at node:internal/deps/undici/undici:6587:50
        at Immediate._onImmediate (node:internal/deps/undici/undici:6619:13)
        at process.processImmediate (node:internal/timers:478:21)
        at process.callbackTrampoline (node:internal/async_hooks:130:17) {
      code: 'UND_ERR_CONNECT_TIMEOUT'

error saving to db when updating conversation NeonDbError: Error connecting to database: fetch failed
    at execute (/var/task/apps/web/.next/server/chunks/375.js:53:7255)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async lI (/var/task/apps/web/.next/server/app/haku/[id]/page.js:56:2168) {
  severity: undefined,
  code: undefined,
  detail: undefined,
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: undefined,
  line: undefined,
  routine: undefined,
  sourceError: TypeError: fetch failed
      at node:internal/deps/undici/undici:12502:13
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async execute (/var/task/apps/web/.next/server/chunks/375.js:53:7166)
      at async lI (/var/task/apps/web/.next/server/app/haku/[id]/page.js:56:2168) {
    [cause]: ConnectTimeoutError: Connect Timeout Error (attempted addresses: 3.126.212.11:443, 3.124.121.135:443)
        at onConnectTimeout (node:internal/deps/undici/undici:6635:28)
        at node:internal/deps/undici/undici:6587:50
        at Immediate._onImmediate (node:internal/deps/undici/undici:6619:13)
        at process.processImmediate (node:internal/timers:478:21)
        at process.callbackTrampoline (node:internal/async_hooks:130:17) {
      code: 'UND_ERR_CONNECT_TIMEOUT'

The call sometimes works and sometimes doesn't. For those that it doesn't, the request may be bigger than those that do, not sure.

16 Replies

conscious-sapphireOP•2y ago

It shouldn't be real timeout, as Vercel logs show that execution time for the latest fail was 20ms

conscious-sapphire•2y ago

Is this just on Vercel? Any issues connecting from your local machine, or using psql?

conscious-sapphireOP•2y ago

Haven’t noticed it on local machine at least, so feels like it’s only when deployed. As said, it only happens sporadically so I’m not 100 % on that.

conscious-sapphire•2y ago

Strange. Are you using the pooler URL?

conscious-sapphireOP•2y ago

Let me check, I'm using the Vercel integration I assume DATABASE_URL is the pooled one by default as there's DATABASE_URL_UNPOOLED , so in that case I'm using the pooled one

conscious-sapphire•2y ago

Yeah, that sounds right. Can you maybe share some timestamps of the errors? We can check logs on our end

conscious-sapphireOP•2y ago

Sure! Just a second - June 05 18:10:53.72 GMT+03:00 - June 05 16:58:09.42 GMT+03:00 - June 05 06:08:16.34 GMT+03:00 The project ID is quiet-dew-64775539 if that helps

conscious-sapphire•2y ago

Can you share the endpoint ID too, e.g ep-foo-bar-12345? I can't see logs between 1500-1600 UTC (June 05 18:10:53.72 GMT+03:00) for the one endpoint I checked OK, I found some logs for a different endpoint. No errors at those times though. Can you confirm the endpoint ID and I can double check. I see activity at 2024-06-05T15:10:57.070155Z and 2024-06-05T13:58:53.689920Z but no errors as I said

conscious-sapphireOP•2y ago

The fist and third of those was on endpoint round-mountain-a2cieklh but I can't find the endpoint for the second one, as it was already deleted as preview env If more data points help, it just happened on endpoint ep-icy-dream-a2jzgfk9 on June 06 21:41:47.43 GMT+03:00. I added a retry function to immediately retry, and the request immediately after succeeded.

conscious-sapphire•2y ago

No errors on this side. I see the log at 2024-06-06T18:41:47.448569Z and subsequent logs connecting to the database and performing the query. Are you using the latest driver. What region is your backend depoyed in?

conscious-sapphire•2y ago

Are you on the free tier. Is your database seeing a cold start at these times? Based on logs I don't think it's a cold start. I wonder if it's some DNS funny business https://github.com/nodejs/undici/issues/1531#issuecomment-1178806172

GitHub

fetch times out in under 5 seconds · Issue #1531 · nodejs/undici

Bug Description When trying to fetch a URL a fetch failed error is thrown with the code as code: 'UND_ERR_CONNECT_TIMEOUT' , This error is thrown at a request that barely takes 5 - 6 second...

conscious-sapphireOP•2y ago

Strange stuff 😄 Using latest versions of everything, both the app on Vercel and database on Neon are deployed to fra1, Pro tier on Vercel and Launch on Neon That particular query should never be a cold start, as it happens at the end of a chat completion to save the chat to the db. The database is used in the chat tools, so it's always active when the erroring query is executed

conscious-sapphireOP•2y ago

I found also https://github.com/vercel/next.js/issues/66373, so it's probably something with that and not the db.

GitHub

Issues · vercel/next.js

The React Framework. Contribute to vercel/next.js development by creating an account on GitHub.

conscious-sapphireOP•2y ago

And found a couple of suggestions for solutions there, https://github.com/vercel/next.js/issues/66373#issuecomment-2148546390 and https://github.com/vercel/vercel/issues/11692#issuecomment-2152859828. Thanks for help 🤓

GitHub

NextJS 14 returns fetch failed with UND_ERR_CONNECT_TIMEOUT error o...

Link to the code that reproduces this issue https://codesandbox.io/p/devbox/gifted-shirley-mzlvgy?file=%2Fapp%2Fapi%2Froute.js%3A1%2C1-39%2C1 To Reproduce From a client-side component, start a fect...

GitHub

Fetch failed with UND_ERR_CONNECT_TIMEOUT error on Next.js serverle...

TLDR; When executing a fetch request from a serverless function it sometimes fails returning a UND_ERR_CONNECT_TIMEOUT on a Nex.js production environment hosted on Vercel. I currently have a Next.j...

conscious-sapphire•2y ago

@mmurto wow, interesting. Did this help?

conscious-sapphireOP•2y ago

I haven’t tried yet, I’ll let a few warning logs accumulate, as I now have the retry functionality so it doesn’t affect users. I’ll let you know if the solutions there work later It’s super strange though, as I have a lot of other fetch calls, and none of them seem to be affected. It’s only this particular call and even it very sporadically.

Gaming

Programming

Strange timeouts on Vercel

Did you find this page helpful?