Integrating a Python endpoint in the t3 stack

shikishikichangchang · 2023-04-11T06:51:17.098Z

Is it possible to integrate a Python endpoint in the t3 stack? I need a Python endpoint because the package that I wanna use is an AI one and currently unavailable in JS/TS.

T

Tom•4/11/23, 7:04 AM

couldnt you just run it as a separate server and then call it from within a trpc endpoint?

T

Tom•4/11/23, 7:05 AM

not the most efficent but itll get the job done if you want to leverage trpc

C

cje•4/11/23, 7:28 AM

Write a GraphQL or OpenAPI Schema for it and make a request to it from your tRPC backend

J

jdsl•4/11/23, 7:55 AM

This might be what you are looking for if using Vercel for hosting:
https://vercel.com/docs/concepts/functions/serverless-functions/runtimes/python

Vercel Documentation

Using the Python Runtime with Serverless Functions

Learn how to use the Python runtime to compile Python Serverless Functions on Vercel.

S

shikishikichangchangOP•4/11/23, 8:35 AM

Yes, I can host a separate api for that endpoint but ideally I would like to keep everything in the same monorepo. Is it possible? Has anyone done it? I read online that some people got it working by putting the *.py files in /api (not /pages/api)

S

shikishikichangchangOP•4/11/23, 8:37 AM

I tried and couldnt get it working

S

shikishikichangchangOP•4/11/23, 8:37 AM

import type { ChildProcessWithoutNullStreams } from "child_process";
import { spawn } from "child_process";
import { trycatch } from "../../../utils/trycatch";
import { createTRPCRouter, protectedProcedure } from "../trpc";

export const pythonRouter = createTRPCRouter({
  test: protectedProcedure.query(async () => {
    return await trycatch({
      fn: () => {
        const python: ChildProcessWithoutNullStreams = spawn("python3", [
          "src/server/api/python/testing.py",
        ]);
        return new Promise<string>((resolve, reject) => {
          let output = "";

          python.stdout.on("data", (data: Buffer) => {
            output += data.toString();
          });

          python.stderr.on("data", (data: Buffer) => {
            reject(data.toString());
          });

          python.on("close", (code) => {
            if (code && code !== 0) {
              reject(`Process exited with code ${code}`);
            } else {
              resolve(output);
            }
          });
        });
      },
      errorMessages: ["Failed to run python code"],
    })();
  }),
});

import type { ChildProcessWithoutNullStreams } from "child_process";
import { spawn } from "child_process";
import { trycatch } from "../../../utils/trycatch";
import { createTRPCRouter, protectedProcedure } from "../trpc";

export const pythonRouter = createTRPCRouter({
  test: protectedProcedure.query(async () => {
    return await trycatch({
      fn: () => {
        const python: ChildProcessWithoutNullStreams = spawn("python3", [
          "src/server/api/python/testing.py",
        ]);
        return new Promise<string>((resolve, reject) => {
          let output = "";

          python.stdout.on("data", (data: Buffer) => {
            output += data.toString();
          });

          python.stderr.on("data", (data: Buffer) => {
            reject(data.toString());
          });

          python.on("close", (code) => {
            if (code && code !== 0) {
              reject(`Process exited with code ${code}`);
            } else {
              resolve(output);
            }
          });
        });
      },
      errorMessages: ["Failed to run python code"],
    })();
  }),
});

import type { ChildProcessWithoutNullStreams } from "child_process";
import { spawn } from "child_process";
import { trycatch } from "../../../utils/trycatch";
import { createTRPCRouter, protectedProcedure } from "../trpc";

export const pythonRouter = createTRPCRouter({
  test: protectedProcedure.query(async () => {
    return await trycatch({
      fn: () => {
        const python: ChildProcessWithoutNullStreams = spawn("python3", [
          "src/server/api/python/testing.py",
        ]);
        return new Promise<string>((resolve, reject) => {
          let output = "";

          python.stdout.on("data", (data: Buffer) => {
            output += data.toString();
          });

          python.stderr.on("data", (data: Buffer) => {
            reject(data.toString());
          });

          python.on("close", (code) => {
            if (code && code !== 0) {
              reject(`Process exited with code ${code}`);
            } else {
              resolve(output);
            }
          });
        });
      },
      errorMessages: ["Failed to run python code"],
    })();
  }),
});

import type { ChildProcessWithoutNullStreams } from "child_process";
import { spawn } from "child_process";
import { trycatch } from "../../../utils/trycatch";
import { createTRPCRouter, protectedProcedure } from "../trpc";

export const pythonRouter = createTRPCRouter({
  test: protectedProcedure.query(async () => {
    return await trycatch({
      fn: () => {
        const python: ChildProcessWithoutNullStreams = spawn("python3", [
          "src/server/api/python/testing.py",
        ]);
        return new Promise<string>((resolve, reject) => {
          let output = "";

          python.stdout.on("data", (data: Buffer) => {
            output += data.toString();
          });

          python.stderr.on("data", (data: Buffer) => {
            reject(data.toString());
          });

          python.on("close", (code) => {
            if (code && code !== 0) {
              reject(`Process exited with code ${code}`);
            } else {
              resolve(output);
            }
          });
        });
      },
      errorMessages: ["Failed to run python code"],
    })();
  }),
});

S

shikishikichangchangOP•4/11/23, 8:38 AM

I was able to get this code to run locally

S

shikishikichangchangOP•4/11/23, 8:38 AM

It basically calls a python file located next to itself

S

shikishikichangchangOP•4/11/23, 8:38 AM

but it fails to run on vercel

S

shikishikichangchangOP•4/11/23, 8:39 AM

I suspect the reason is that vercel doesnt have python installed

S

shikishikichangchangOP•4/11/23, 8:39 AM

I was wondering if it's possble to install all nextjs dependencies and pythons?

P

Patrick / Blaze•4/11/23, 8:40 AM

Take a look again at the link above: https://vercel.com/docs/concepts/functions/serverless-functions/runtimes/python

Vercel Documentation

Using the Python Runtime with Serverless Functions

Learn how to use the Python runtime to compile Python Serverless Functions on Vercel.

S

shikishikichangchangOP•4/11/23, 8:42 AM

I did

P

Patrick / Blaze•4/11/23, 8:42 AM

The page is answering your question: Yes its possible, yes you can install packages (see bottom of the page requirements.txt)

S

shikishikichangchangOP•4/11/23, 8:43 AM

Im asking if it's possible to install both nextJS and python

S

shikishikichangchangOP•4/11/23, 8:43 AM

together under pages/api

P

Patrick / Blaze•4/11/23, 8:44 AM

~~Yes, thats possible~~

P

Patrick / Blaze•4/11/23, 8:45 AM

Ah, maybe not. In their example repo it looks like this: https://github.com/vercel/examples/tree/main/python/hello-world

GitHub

examples/python/hello-world at main · vercel/examples

Enjoy our curated collection of examples and solutions. Use these patterns to build your own robust and scalable applications. - examples/python/hello-world at main · vercel/examples

P

Patrick / Blaze•4/11/23, 8:45 AM

So you might need another deployment.

P

Patrick / Blaze•4/11/23, 8:47 AM

Though based on this, its possible: https://stackoverflow.com/questions/63652016/python-serverless-function-vercel-next-js

Stack Overflow

Python Serverless Function Vercel - Next.js

I found out that I could use Python to create a serverless function inside a Next.js project. Once deployed to Vercel, it will get converted into a serverless function.
I went through the docs and ...

P

Patrick / Blaze•4/11/23, 8:47 AM

In short, if you're using another language to write a serverless function inside a Next.js project. Be sure to place it inside an api folder that sits in the root directory of the project (if there's none, create one).

In short, if you're using another language to write a serverless function inside a Next.js project. Be sure to place it inside an api folder that sits in the root directory of the project (if there's none, create one).

In short, if you're using another language to write a serverless function inside a Next.js project. Be sure to place it inside an api folder that sits in the root directory of the project (if there's none, create one).

In short, if you're using another language to write a serverless function inside a Next.js project. Be sure to place it inside an api folder that sits in the root directory of the project (if there's none, create one).

S

shikishikichangchangOP•4/11/23, 8:53 AM

That's what I did and failed haha

Sshikishikichangchang Yes, I can host a separate api for that endpoint but ideally I would like to kee...

S

shikishikichangchangOP•4/11/23, 8:54 AM

.

J

jdsl•4/11/23, 9:40 AM

Deployment failed or getting error hitting the endpoint?

J

jdsl•4/11/23, 9:40 AM

I'll test on one of my projects to see if I can get it to work

J

jdsl•4/12/23, 12:14 AM

Found some more info here:
https://github.com/vercel/vercel/discussions/6197
https://github.com/vercel/vercel/discussions/4023

Looks like while it is possible to mix & match, it's not as straight forward.
You have to move the api files to the root of your project

/api

/api

and it might cause a few issues with 3rd party packages that are expecting

pages/api

pages/api

pages/api

pages/api to exist.

S

shikishikichangchangOP•4/13/23, 12:01 AM

I read those two discussions previously too. Didn't help. Were you able to get it working?

J

jdsl•4/13/23, 1:03 AM

No, can't move my

/pages/api

/pages/api

/pages/api

/pages/api folder in the project I was gonna test since I have some third party packages I didn't want to mess with.
I might try later tonight on a fresh T3 and see if I can get it to work. I'll let you know what I learn.

S

shikishikichangchangOP•4/13/23, 2:49 AM

Thanks. Appreciate it

Integrating a Python endpoint in the t3 stack

Similar Threads

Similar Threads

Similar Threads