winsoroaks
winsoroaks
Explore posts from servers
NNovuCCConvex Community
PostsComments
CCConvex Community
Created by winsoroaks on 4/29/2025 in #support-community
How to properly combine a Convex query with an action?
I'm trying to implement a query that needs to fetch some encrypted data and decrypt it using AWS KMS. However, I'm running into a TypeScript error where ctx.runAction is not available in the query context. Here's a simplified version of what I'm trying to do: 
export const queryEncryptedData = query({
  args: { id: v.string() },
  handler: async (ctx, args) => {
    // 1. First fetch the encrypted data
    const data = await ctx.db
      .query("myTable")
      .withIndex("by_id", q => q.eq("id", args.id))
      .first()
    if (!data) return undefined

    // 2. Then try to decrypt the sensitive fields
    // This is where we get the error:
    // Property 'runAction' does not exist on type 'QueryCtx'
    const decryptedItems = await Promise.all(
      data.items.map(async (item) => ({
        ...item,
        value: await ctx.runAction(internal.kms.decryptData, {
          data: item.encryptedValue?.data,
          key: item.encryptedValue?.key,
          iv: item.encryptedValue?.iv,
        })
      }))
    )

    return {
      items: data.items.map(item => ({
        ...item,
        // Need to return the decrypted value here
      }))
    }
  }
})
export const queryEncryptedData = query({
  args: { id: v.string() },
  handler: async (ctx, args) => {
    // 1. First fetch the encrypted data
    const data = await ctx.db
      .query("myTable")
      .withIndex("by_id", q => q.eq("id", args.id))
      .first()
    if (!data) return undefined

    // 2. Then try to decrypt the sensitive fields
    // This is where we get the error:
    // Property 'runAction' does not exist on type 'QueryCtx'
    const decryptedItems = await Promise.all(
      data.items.map(async (item) => ({
        ...item,
        value: await ctx.runAction(internal.kms.decryptData, {
          data: item.encryptedValue?.data,
          key: item.encryptedValue?.key,
          iv: item.encryptedValue?.iv,
        })
      }))
    )

    return {
      items: data.items.map(item => ({
        ...item,
        // Need to return the decrypted value here
      }))
    }
  }
})
is the only option to do useEffect in the front end component?
10 replies
CCConvex Community
Created by winsoroaks on 4/27/2025 in #support-community
How to properly mock Convex internal actions in tests
I'm trying to write tests for a function that uses AWS KMS decryption (fetchSomething). This function relies on internal.kms.decrypt action. I've tried several approaches but all attempts resulted in type errors or failed decryption. What's the correct way to mock an internal action in Convex tests? Example of what I'm trying to test: 
export const fetchSomething = action({
  args: {...},
  handler: async (ctx, args) => {

    // ... gets encrypted token ...
    const decryptedToken = await ctx.runAction(
      internal.kms.decrypt,
      { token: user.token }
    )
    // ... uses decrypted token ...

  }
})
export const fetchSomething = action({
  args: {...},
  handler: async (ctx, args) => {

    // ... gets encrypted token ...
    const decryptedToken = await ctx.runAction(
      internal.kms.decrypt,
      { token: user.token }
    )
    // ... uses decrypted token ...

  }
})
2 replies
CCConvex Community
Created by winsoroaks on 4/23/2025 in #support-community
how to test if schemaValidation = true works in prod without pushing
hi team! how can i test if schemaValidation = true works in prod without pushing? thanks!
7 replies
CCConvex Community
Created by winsoroaks on 4/22/2025 in #support-community
convex backup
hi team, does the convex backup and restore mean if convex db somehow goes down / dies / gets deleted by some wrong operations by the team, we can restore from the backup?
6 replies
CCConvex Community
Created by winsoroaks on 4/4/2025 in #support-community
any recommendation on how to store PII?
hi there, im planning to store some PII in the db. while i think convex's architecture is secure, im afraid of getting myself compromised and having attackers peek into my table. hence, im considering storing some sort of encryption key on a 3rd party provider (maybe aws or railway). does that make sense or im overengineering stuff?
7 replies
CCConvex Community
Created by winsoroaks on 12/29/2024 in #support-community
is there a way to persist env vars of the same preview deployment?
im not sure if im doing sth wrongly, i find that every time i push updated code to the same preview deployment, the env vars are wiped out. is there a way to persist them?
3 replies
CCConvex Community
Created by winsoroaks on 6/26/2024 in #support-community
tools to subscribe to websocket on the backend?
Hi team! I am planning to build an app that subscribes to some trading activities using convex, what’s the best way to do it? I currently just doing a while True loop, might not be the most efficient thing. Thanks! 😅
9 replies
CCConvex Community
Created by winsoroaks on 5/20/2024 in #support-community
is there a way to mock the ctx.db.system?
hi team! one of my mutation checks for the image size and store them in my table. 
const metadata = await ctx.db.system.get(s.storageId as Id<"_storage">)
if (!metadata) throw new ConvexError("not found")
size += metadata.size
const metadata = await ctx.db.system.get(s.storageId as Id<"_storage">)
if (!metadata) throw new ConvexError("not found")
size += metadata.size
is there a way to handle this from the convex-test package? thanks!
11 replies
CCConvex Community
Created by winsoroaks on 5/14/2024 in #support-community
queryWithUser is not a function.
(not urgent) hi team! im working on some unit tests. the convex-test library is awesome and fun! however, im running into a weird scenario (not related to the lib perhaps). 
// users.ts
import { LIMIT } from "./usage"

export const createUser = mutation({
  ...
  await ctx.db.insert("usage", {
    userId,
    used: 0,
    limit: LIMIT,
  })

...

const addUser = customCtx(async (ctx: QueryCtx) => {
  const userId = await getUserIdByTokenIdentifier(ctx)
  if (!userId) {
    throw new ConvexError("User ID not found")
  }
  return { userId }
})

export const queryWithUser = customQuery(query, addUser)
// users.ts
import { LIMIT } from "./usage"

export const createUser = mutation({
  ...
  await ctx.db.insert("usage", {
    userId,
    used: 0,
    limit: LIMIT,
  })

...

const addUser = customCtx(async (ctx: QueryCtx) => {
  const userId = await getUserIdByTokenIdentifier(ctx)
  if (!userId) {
    throw new ConvexError("User ID not found")
  }
  return { userId }
})

export const queryWithUser = customQuery(query, addUser)
and when i run vitest, im getting 
TypeError: queryWithUser is not a function
 ❯ convex/usage.ts:24:25
     22| })
     23|
     24| export const getUsage = queryWithUser({
       |                         ^
     25|   handler: async (ctx) => {
     26|     const usage = await ctx.db
TypeError: queryWithUser is not a function
 ❯ convex/usage.ts:24:25
     22| })
     23|
     24| export const getUsage = queryWithUser({
       |                         ^
     25|   handler: async (ctx) => {
     26|     const usage = await ctx.db
but when i hard code the LIMIT to be 100, it works. any idea what's wrong? 🙏
2 replies
CCConvex Community
Created by winsoroaks on 5/14/2024 in #support-community
query and update inside for-loop
(not urgent) hi team! im running into a weird situation. hopefully it's just that im blind to the logic implementation. i have the following code that "updates" the usage table with the image size the user uploaded 
await Promise.all(
  args.storageIds.map(async (s) => {
    const metadata = await ctx.db.system.get(s.storageId as Id<"_storage">)
    console.log(
      "🚀 ~ file: images.ts:43 ~ args.storageIds.map ~ metadata:",
      metadata.size
    ) // 93558, 68153, 68948

    const usage = await ctx.db
      .query("usage")
      .withIndex("by_userId", (q) => q.eq("userId", ctx.userId))
      .unique()
    console.log(
      "🚀 ~ file: images.ts:53 ~ args.storageIds.map ~ usage:",
      usage.used
    ) // 0, 0, 0, should be 0, 93558, 93558 + 68153
    await ctx.db.patch(usage._id, { used: usage.used + metadata.size })

    await ctx.db.insert("images", {
      userId: ctx.userId,
      uploadedImageId: s.storageId,
    })
  })
)
await Promise.all(
  args.storageIds.map(async (s) => {
    const metadata = await ctx.db.system.get(s.storageId as Id<"_storage">)
    console.log(
      "🚀 ~ file: images.ts:43 ~ args.storageIds.map ~ metadata:",
      metadata.size
    ) // 93558, 68153, 68948

    const usage = await ctx.db
      .query("usage")
      .withIndex("by_userId", (q) => q.eq("userId", ctx.userId))
      .unique()
    console.log(
      "🚀 ~ file: images.ts:53 ~ args.storageIds.map ~ usage:",
      usage.used
    ) // 0, 0, 0, should be 0, 93558, 93558 + 68153
    await ctx.db.patch(usage._id, { used: usage.used + metadata.size })

    await ctx.db.insert("images", {
      userId: ctx.userId,
      uploadedImageId: s.storageId,
    })
  })
)
im not sure why the usage returned is 0, 0, 0 where i'd expect it to be 0, 93558, 93558 + 68153. i managed to change the implementation to the following: 
let size = 0
await Promise.all(
  args.storageIds.map(async (s) => {
    const metadata = await ctx.db.system.get(s.storageId as Id<"_storage">)
    size += metadata.size
    await ctx.db.insert("images", {
      userId: ctx.userId,
      uploadedImageId: s.storageId,
    })
  })
)

const usage = await ctx.db
  .query("usage")
  .withIndex("by_userId", (q) => q.eq("userId", ctx.userId))
  .unique()
await ctx.db.patch(usage._id, { used: usage.used + size })
let size = 0
await Promise.all(
  args.storageIds.map(async (s) => {
    const metadata = await ctx.db.system.get(s.storageId as Id<"_storage">)
    size += metadata.size
    await ctx.db.insert("images", {
      userId: ctx.userId,
      uploadedImageId: s.storageId,
    })
  })
)

const usage = await ctx.db
  .query("usage")
  .withIndex("by_userId", (q) => q.eq("userId", ctx.userId))
  .unique()
await ctx.db.patch(usage._id, { used: usage.used + size })
which is def more efficient and worked. im just curious if there's some special case with my first implementation. thank you.
9 replies
CCConvex Community
Created by winsoroaks on 5/12/2024 in #support-community
running migration from dashboard
No description
10 replies
CCConvex Community
Created by winsoroaks on 5/11/2024 in #support-community
run migration on prod before pushing new schema?
i've added a new column to my table, after running a migration on local, 
// schema.ts
rank: v.optional(v.number()),

export const addRankToImages = migration({
  table: "images",
  migrateOne: async (ctx, image) => {
    if (!image.rank) {
      await ctx.db.patch(image._id, { rank: 0 })
    }
  },
})
// schema.ts
rank: v.optional(v.number()),

export const addRankToImages = migration({
  table: "images",
  migrateOne: async (ctx, image) => {
    if (!image.rank) {
      await ctx.db.patch(image._id, { rank: 0 })
    }
  },
})
can i run the migration on production before pushing the new schema? i also have rank an optional field now
2 replies
CCConvex Community
Created by winsoroaks on 5/8/2024 in #support-community
suggestions on improving my architecture? (not really convex specific)
No description
37 replies
CCConvex Community
Created by winsoroaks on 5/6/2024 in #support-community
another copy of javascript when adding tsconfig.json
i just started to add a tsconfig.json to my /convex dir and im getting a copy of js file for every ts file i wrote. this hasnt been the case in the past and i wonder if i've configured something wrongly? here's my tsconfig.json: 
{
  /* This TypeScript project config describes the environment that
   * Convex functions run in and is used to typecheck them.
   * You can modify it, but some settings required to use Convex.
   */
  "compilerOptions": {
    /* These settings are not required by Convex and can be modified. */
    "allowJs": true,
    "strict": true,
    "skipLibCheck": true,

    /* These compiler options are required by Convex */
    "target": "ESNext",
    "lib": ["ES2021", "dom"],
    "forceConsistentCasingInFileNames": true,
    "allowSyntheticDefaultImports": true,
    "module": "ESNext",
    "moduleResolution": "Node",
    "isolatedModules": true,
    "noEmit": true,
    "jsx": "react"
  },
  "include": ["./**/*"],
  "exclude": ["./_generated"]
}
{
  /* This TypeScript project config describes the environment that
   * Convex functions run in and is used to typecheck them.
   * You can modify it, but some settings required to use Convex.
   */
  "compilerOptions": {
    /* These settings are not required by Convex and can be modified. */
    "allowJs": true,
    "strict": true,
    "skipLibCheck": true,

    /* These compiler options are required by Convex */
    "target": "ESNext",
    "lib": ["ES2021", "dom"],
    "forceConsistentCasingInFileNames": true,
    "allowSyntheticDefaultImports": true,
    "module": "ESNext",
    "moduleResolution": "Node",
    "isolatedModules": true,
    "noEmit": true,
    "jsx": "react"
  },
  "include": ["./**/*"],
  "exclude": ["./_generated"]
}
thank you 🙂
7 replies
CCConvex Community
Created by winsoroaks on 5/3/2024 in #support-community
when is the best time to create a user entry in the db?
hi all! im curious what's the best practice here. assume that i have a to-do list app and using clerk for auth. if i allow the user to sign in via google, there's no clear way to do "user has signed up" -> "i need to register user in db."
when should i create an entry of the user in the db? im thinking of when the user has created a to-do list and clicked on the save button, i can check if the user has existed in the users db and create the entry. is this reasonable? thank you
34 replies
CCConvex Community
Created by winsoroaks on 4/24/2024 in #support-community
can i do auth or run a function on the server side?
hi all! i have the following code, 
"use client"

import { api } from "@/convex/_generated/api"
import { useConvexAuth, useQuery } from "convex/react"

import { Skeleton } from "@/components/ui/skeleton"

export function SideBar() {
  const { isLoading } = useConvexAuth()

  const username = useQuery(
    api.db.username,
    isLoading ? "skip" : {}
  )

  return <span className="capitalize">{username}</span>
}
"use client"

import { api } from "@/convex/_generated/api"
import { useConvexAuth, useQuery } from "convex/react"

import { Skeleton } from "@/components/ui/skeleton"

export function SideBar() {
  const { isLoading } = useConvexAuth()

  const username = useQuery(
    api.db.username,
    isLoading ? "skip" : {}
  )

  return <span className="capitalize">{username}</span>
}
and i've been running into the following error when i click on "logout" 
ConvexError: [CONVEX Q(db:username)] [Request ID: 26f4c2ab052920a5] Server Error
Uncaught ConvexError: Unauthenticated call to function requiring authentication
    at getClerkIdentity (../../convex/db/user.ts:18:6)
    at async handler (../../convex/db/user.ts:166:17)

  Called by client
ConvexError: [CONVEX Q(db:username)] [Request ID: 26f4c2ab052920a5] Server Error
Uncaught ConvexError: Unauthenticated call to function requiring authentication
    at getClerkIdentity (../../convex/db/user.ts:18:6)
    at async handler (../../convex/db/user.ts:166:17)

  Called by client
is it possible to move api.db.username to the server side for this component? or am i taking a wrong path? thanks!
3 replies
CCConvex Community
Created by winsoroaks on 4/5/2024 in #support-community
ConvexChatMessageHistory schema for langchain
Hi @michal, im reading the article: https://stack.convex.dev/ai-chat-using-langchain-and-convex and im trying to understand how the messages schema is constructured and its relationship with ConvexChatMessageHistory. im currently using a RunnableSequence and it doesn't save to memory. i have to manually do 
await memory.saveContext(
  {
    input: {
      sessionId,
      type: 'human',
      content: message
    }
  },
  {
    content: response.content
  }
)
await memory.saveContext(
  {
    input: {
      sessionId,
      type: 'human',
      content: message
    }
  },
  {
    content: response.content
  }
)
however, im running into the error below 
Uncaught Error: Uncaught Error: Failed to insert or update a document in table "messages" because it does not match the schema: Object contains extra field `session_id` that is not in the validator.
Path: .message.data
Object: {additional_kwargs: {}, content: "what is an avocado?", response_metadata: {}, session_id: "34a48a99-e910-43e9-af24-8a9dbfddcc79", type: "human"}
Validator: v.object({additional_kwargs: v.optional(v.any()), content: v.string(), name: v.optional(v.string()), response_metadata: v.optional(v.any()), role: v.optional(v.string())})
Uncaught Error: Uncaught Error: Failed to insert or update a document in table "messages" because it does not match the schema: Object contains extra field `session_id` that is not in the validator.
Path: .message.data
Object: {additional_kwargs: {}, content: "what is an avocado?", response_metadata: {}, session_id: "34a48a99-e910-43e9-af24-8a9dbfddcc79", type: "human"}
Validator: v.object({additional_kwargs: v.optional(v.any()), content: v.string(), name: v.optional(v.string()), response_metadata: v.optional(v.any()), role: v.optional(v.string())})
to fix this, i had to modify the schema to 
@@ -36,11 +36,14 @@ export default defineSchema({
     sessionId: v.string(),
     message: v.object({
       type: v.string(),
       data: v.object({
+        type: v.optional(v.string()),
         content: v.string(),
         role: v.optional(v.string()),
         name: v.optional(v.string()),
         additional_kwargs: v.optional(v.any()),
+        session_id: v.optional(v.any()),
         response_metadata: v.optional(v.any())
       })
     })
@@ -36,11 +36,14 @@ export default defineSchema({
     sessionId: v.string(),
     message: v.object({
       type: v.string(),
       data: v.object({
+        type: v.optional(v.string()),
         content: v.string(),
         role: v.optional(v.string()),
         name: v.optional(v.string()),
         additional_kwargs: v.optional(v.any()),
+        session_id: v.optional(v.any()),
         response_metadata: v.optional(v.any())
       })
     })
any idea why the input is having a diff structure when it tries to write to the db? thank you 🙂
2 replies
CCConvex Community
Created by winsoroaks on 4/4/2024 in #support-community
Adding deps to node.externalPackages didn't resolve the error
Hi team! im running in to the error below 
✘ [ERROR] Could not resolve "youtubei.js"

    node_modules/langchain/dist/document_loaders/web/youtube.js:2:26:
      2 │ import { Innertube } from "youtubei.js";
✘ [ERROR] Could not resolve "youtubei.js"

    node_modules/langchain/dist/document_loaders/web/youtube.js:2:26:
      2 │ import { Innertube } from "youtubei.js";
tried a few variations 
// convex.json
{
  "node": {
    "externalPackages": [
      "langchain" // "youtube-transcript", "youtubei.js"
    ]
  }
}
// convex.json
{
  "node": {
    "externalPackages": [
      "langchain" // "youtube-transcript", "youtubei.js"
    ]
  }
}
but didnt fix the issue 
Uncaught Failed to analyze ingest/load.js: Cannot find package 'youtube-transcript' imported from /tmp/external_deps/e9a05690-196f-40ed-88a7-cd46f4362a76/node_modules/langchain/dist/document_loaders/web/youtube.js
Uncaught Failed to analyze ingest/load.js: Cannot find package 'youtubei.js' imported from /tmp/external_deps/834ceee9-83db-4a8b-9907-c6805d171626/node_modules/langchain/dist/document_loaders/web/youtube.js
Uncaught Failed to analyze ingest/load.js: Cannot find package 'youtube-transcript' imported from /tmp/external_deps/e9a05690-196f-40ed-88a7-cd46f4362a76/node_modules/langchain/dist/document_loaders/web/youtube.js
Uncaught Failed to analyze ingest/load.js: Cannot find package 'youtubei.js' imported from /tmp/external_deps/834ceee9-83db-4a8b-9907-c6805d171626/node_modules/langchain/dist/document_loaders/web/youtube.js
any idea what's wrong? thank you.
11 replies
CCConvex Community
Created by winsoroaks on 4/2/2024 in #support-community
Uncaught Error: Failed to insert or update a document in table "messages" because it does not match
👋 was the ConvexVectorStore updated recently? im following the example from https://stack.convex.dev/ai-chat-using-langchain-and-convex and ran into the error above. was able to fix it by adding response_metadata: v.optional(v.any()) to the schema
2 replies
CCConvex Community
Created by winsoroaks on 3/10/2024 in #support-community
recommended logger for convex functions?
hi team, i've been using tslog as my logger for my front end stuff. apparently when i try to import it to my convex functions, im getting weird stuff, sth like some byte arrays? 
'\x1B[37m2024-03-10 06:36:32.902\x1B[39m\x1B[0m\x1B[0m\t\x1B[34m\x1B[1mINFO\x1B[22m\x1B[39m\x1B[0m\x1B[0m\t[\x1B[37m/user/client.js:1280\x1B[39m\x1B[0m\x1B[0m\x1B[1m\x1B[37m\x1B[39m\x1B[22m\x1B[0m\x1B[0m]\tUpdating ...'
'\x1B[37m2024-03-10 06:36:32.902\x1B[39m\x1B[0m\x1B[0m\t\x1B[34m\x1B[1mINFO\x1B[22m\x1B[39m\x1B[0m\x1B[0m\t[\x1B[37m/user/client.js:1280\x1B[39m\x1B[0m\x1B[0m\x1B[1m\x1B[37m\x1B[39m\x1B[22m\x1B[0m\x1B[0m]\tUpdating ...'
i think i can go with console.log, just wondering if there's some known limitation with my current approach. thanks!
2 replies