Exploring Effect/AI for Backend Development with TensorRT-LLM
Hey, we manage our own GPUs running open source LLMs and we're building the next iteration of our backend using Effect. We're just now diving into effect/ai and exploring it for our use case. We typically run
TensorRT-LLM / llgtrt, which expose an API that is similar to OpenAI.
