PLI

Portkey • LLMs in Prod

This is a space where industry practitioners deploying LLMs in production can share insights and learn from one another.

Join

PLI

Portkey • LLMs in Prod

This is a space where industry practitioners deploying LLMs in production can share insights and learn from one another.

Join

feature-releases

bulletin

Vrushank | Portkey

9/25/2024

Thrilled to share that we are partnering with MongoDB to help companies take their AI apps to produc

Thrilled to share that we are partnering with MongoDB to help companies take their AI apps to production! This partnership supercharges your AI dev cycle: 1️⃣ MongoDB becomes your one-stop shop for all your LLM logs, telemetry, and embeddings. On your cloud, under your control....

Vrushank | Portkey

9/12/2024

oh btw, OpenAI's new `o1-preview` and `o1-mini` models are already supported on Portkey.

oh btw, OpenAI's new o1-preview and o1-mini models are already supported on Portkey. Compared to gpt-4o, this model reflects for a long time, and answers questions like "how many r's in the word strawberry?", "how many words in your output?" etc exceptionally well. It works significatly better on maths, science, puzzle solving, and coding tasks....

Vrushank | Portkey

9/12/2024

**New Integration**: [Inference.net](https://docs.portkey.ai/docs/integrations/llms/inference.net) (

New Integration: Inference.net (wholesaler of LLM inference tokens) Portkey is now integrated with the Inference.net API, which is a "wholesaler" of LLM inference tokens for open source models like Llama 3. Inference.net is 50-90% cheaper than leading inference providers in the market, and could be especially useful for batch inference jobs!...

Vrushank | Portkey

9/11/2024

✨ New: Portkey's Guardrails can now detect & validate multiple JSON objects within code blocks and i

✨ New: Portkey's Guardrails can now detect & validate multiple JSON objects within code blocks and in plain text. We just shipped improved functionality for JSON Keys & JSON Schema Guardrail checks: :a_tick: Detect JSON within code blocks and in plain text...

Vrushank | Portkey

9/11/2024

New Contribution to Gateway by [James](https://github.com/jpulec):

New Contribution to Gateway by James: ✅ Sending multiple system prompts in your Anthropic calls. The OpenAI API supports this functionality, but the Anthropic API required the user to only send one system prompt in their request....

Vrushank | Portkey

9/10/2024

Just published docs for how to use Anthropic prompt caching with Portkey: https://docs.portkey.ai/do

Just published docs for how to use Anthropic prompt caching with Portkey: https://docs.portkey.ai/docs/integrations/llms/anthropic/prompt-caching Coming soon: Anthropic prompt caching support on the prompt playground! (thanks to @rickydickydoo)...

Vrushank | Portkey

9/6/2024

August was pretty special for us: Portkey crossed **2 Billion** total requests processed through our

August was pretty special for us: Portkey crossed 2 Billion total requests processed through our platform! (this number was 0 last year!) We're humbled to be production partners for thousands of leading AI companies around the world. Last month, we did 3 major releases (and about 40 other releases): Everything from Guardrails, to Conditional Router, to Tracing, and more!...

Vrushank | Portkey

9/4/2024

Just released: New Virtual Keys modal for adding and editing your virtual keys ✨

Vrushank | Portkey

8/27/2024

🔥 Day 0 support for the blazing fast [Cerebras Inference API](https://cerebras.ai/inference).

🔥 Day 0 support for the blazing fast Cerebras Inference API. Get an incredible throughput of 1800 tokens/sec with this API and route to it seamlessly with Portkey: https://docs.portkey.ai/docs/integrations/llms/cerebras...

Vrushank | Portkey

8/16/2024

@everyone We've made an important release to Portkey:

@everyone We've made an important release to Portkey: We are bringing Guardrails on the Gateway Portkey now incorporates 50+ state-of-the-art guardrails to help you enforce LLM behavior in real-time. And because it's Portkey, you can synchronously run Guardrails on your requests and route them with precision....

Vrushank | Portkey

8/2/2024

The new, benchmark-beating Gemini 1.5 Pro is now on Portkey!

The new, benchmark-beating Gemini 1.5 Pro is now on Portkey! https://x.com/PortkeyAI/status/1819250970724794507...

Vrushank | Portkey

7/25/2024

Mistral just released Mistral Large 2 — a 123B parameter model with a 128k context window.

Mistral just released Mistral Large 2 — a 123B parameter model with a 128k context window. ✅ Multilingual, highly efficient, and excels in coding, math, and reasoning. And surprisingly, it is on par with Meta’s Llama 3.1 405B!!...

Vrushank | Portkey

7/23/2024

Llama 3.1, works out of the box now with Portkey!

Llama 3.1, works out of the box now with Portkey! 🦙🦙🦙.🦙...

Vrushank | Portkey

7/19/2024

With the new Portkey release, all URL slugs look _✨nicer✨_.

With the new Portkey release, all URL slugs look ✨nicer✨. 🔊 Sound on...

Vrushank | Portkey

7/18/2024

OpenAI just dropped GPT-4o mini — a game-changer for AI accessibility! 🚀

OpenAI just dropped GPT-4o mini — a game-changer for AI accessibility! 🚀 ✅ 60% cheaper than GPT-3.5 Turbo ✅ Outperforms competitors on key benchmarks ✅ Multimodal: supports both text and vision...

Vrushank | Portkey

7/18/2024

Feature highlight - you can route to your **custom Azure URLs** (i.e. not the typical Azure OpenAI `

Feature highlight - you can route to your custom Azure URLs (i.e. not the typical Azure OpenAI {resource_name}.openai.azure.com/.. URLs) using Portkey's custom_host param! Here's a code snippet that shows how to do it - https://discord.com/channels/1143393887742861333/1263212287783600340/1263222644195721367...

Vrushank | Portkey

7/17/2024

Mistral just released Codestral Mamba, a revolutionary 7B parameter coding model based on the Mamba

Mistral just released Codestral Mamba, a revolutionary 7B parameter coding model based on the Mamba architecture. With linear time inference, 256k token context, and Apache 2.0 license, it's set to transform code generation. Try out the Codestral Mamba model using Portkey! 👇...

Vrushank | Portkey

7/1/2024

Portkey is now a PWA. You can install it easily as a standalone app, and keep it handy for all your

Portkey is now a PWA. You can install it easily as a standalone app, and keep it handy for all your LLM monitoring, prompts, and gateway needs!

Vrushank | Portkey

6/21/2024

Claude 3.5 Sonnet Now Live on Portkey!

Claude 3.5 Sonnet Now Live on Portkey! Available both on Anthropic API AND AWS Bedrock! https://x.com/jumbld/status/1804145028656144390...

Vrushank | Portkey

6/17/2024

Last one for the day (hopefully!)

Last one for the day (hopefully!) You can now Rename & Delete Configs!...

Previous Next

Gaming

Programming

Portkey • LLMs in Prod

This is a space where industry practitioners deploying LLMs in production can share insights and learn from one another.

Portkey • LLMs in Prod

This is a space where industry practitioners deploying LLMs in production can share insights and learn from one another.