Diagnosing PGVector query variance
I'm running into a huge variance in response times (250ms -8000ms with an average case latency is fine at ~400ms.) in a Postgres query and trying to understand the source of the issue.
We have a Neon Postgres instance with a few tables that use
We use this for is a query that's a hybrid embedding and popularity search. The table isn't very big, it's about 35k records. The
That said, it doesn't even seem like it's a case of certain inputs being slower than others in doing the calculation. Anecdotally, we see single queries being really slow, then redoing that query again being fast. Right now our cache ratio is
Do you have any suggestions for next places to troubleshoot?
We have a Neon Postgres instance with a few tables that use
pgvector for embeddings and OpenAI text-embedding-3-small embeddings. We're hosted on Vercel and using Neon through their integration.We use this for is a query that's a hybrid embedding and popularity search. The table isn't very big, it's about 35k records. The
order by can't use the HSNW index because we're doing some math with it. Basically cosine distance + custom popularity score so we're in for some trouble. That said, it doesn't even seem like it's a case of certain inputs being slower than others in doing the calculation. Anecdotally, we see single queries being really slow, then redoing that query again being fast. Right now our cache ratio is
98.22%. Do you have any suggestions for next places to troubleshoot?
