Portkey • LLMs in Prod

PLI

Portkey • LLMs in Prod

This is a space where industry practitioners deploying LLMs in production can share insights and learn from one another.

Join

feature-releases

bulletin

Vrushank | Portkey
Vrushank | Portkey
View on Discord
5/3/2025

People! Gemini 2.5 Flash is available in the prompt playground now!!

People! Gemini 2.5 Flash is available in the prompt playground now!!
Vrushank | Portkey
Vrushank | Portkey
View on Discord
4/16/2025

Route to the latest models from OpenAI - `o3` & `o4-mini` through Portkey! Available on both the `/c

Route to the latest models from OpenAI - o3 & o4-mini through Portkey! Available on both the /chat/completions and /responses endpoints
No description
Vrushank | Portkey
Vrushank | Portkey
View on Discord
3/10/2025

We've redesigned the Portkey sidebar and put all the features under 3 core categories:

We've redesigned the Portkey sidebar and put all the features under 3 core categories: - Observability - View the Analytics & Logs for all your requests - AI Gateway - Manage providers, routing configs, and guardrails - Prompt Engineering - Develop, test, and deploy prod-ready prompts ...
Vrushank | Portkey
Vrushank | Portkey
View on Discord
2/1/2025

🔥 o3-mini is now available on Portkey! Use it across both OpenAI & Azure OpenAI seamlessly

🔥 o3-mini is now available on Portkey! Use it across both OpenAI & Azure OpenAI seamlessly ✅ Backward compatible with system role (you can continue using system role and we convert it to developer for the reasoning models) ✅ Backward compatible with max_tokens param. (we convert it to max_completion_tokens at the Gateway level)...
No description
Vrushank | Portkey
Vrushank | Portkey
View on Discord
1/28/2025

🚀 Just shipped: **Audit Logs**!!

🚀 Just shipped: Audit Logs!! As Gen AI systems scale in production, visibility becomes the cornerstone of responsible deployment. Portkey's audit logs is a complete system of record for your AI stack - tracking every prompt modification, API key creation, config change, and more across your entire Portkey org...
No description
Vrushank | Portkey
Vrushank | Portkey
View on Discord
1/17/2025

@everyone We are starting the webinar for LLMs in Prod report now! https://meet.google.com/epk-cctk-

@everyone We are starting the webinar for LLMs in Prod report now! https://meet.google.com/epk-cctk-wmd
Vrushank | Portkey
Vrushank | Portkey
View on Discord
12/28/2024

If you're building any AI app that uses RAG, you need to check out the Perplexity API — they have so

If you're building any AI app that uses RAG, you need to check out the Perplexity API — they have some of the most advanced search & retrieval features, all available for relatively low cost, and natively available on Portkey! Feature 1️⃣: Search Domain Filter Want your LLM to only cite scientific papers? Or avoid certain websites? Just send ```...
Vrushank | Portkey
Vrushank | Portkey
View on Discord
12/28/2024

🔐 Managing developer access just got easier on Portkey!

🔐 Managing developer access just got easier on Portkey! We now supports both SAML & OIDC for enterprise SSO – giving you the flexibility to use whichever authentication method works best for your team. Already using Okta, Azure AD, or Google Workspace? Set up SSO in minutes ⚡️...
No description
Vrushank | Portkey
Vrushank | Portkey
View on Discord
12/27/2024

The little things: Code blocks in Portkey logs now show their language type. Small upgrade, nicer ex

The little things: Code blocks in Portkey logs now show their language type. Small upgrade, nicer experience 🔍
No description
Vrushank | Portkey
Vrushank | Portkey
View on Discord
12/20/2024

We also mapped out the whole MCP ecosystem put and all available MCP servers, from reference impleme

We also mapped out the whole MCP ecosystem put and all available MCP servers, from reference implementations to community contributions, in one place - https://portkey.ai/mcp-servers
Vrushank | Portkey
Vrushank | Portkey
View on Discord
12/19/2024

@everyone Announcing Portkey's own MCP Client!

@everyone Announcing Portkey's own MCP Client! We're redefining how you can build an AI agent: Just write your task in natural language in Portkey prompts, and watch the agent execute it effortlessly! - 90+ MCP servers...
Vrushank | Portkey
Vrushank | Portkey
View on Discord
12/19/2024

The Open source Gateway now spins up a *mini console* that logs all of your requests in one place wi

The Open source Gateway now spins up a mini console that logs all of your requests in one place with stats on request latency and more 🔥 Update your Gateway package with ```sh npm update @portkey-ai/gateway...
No description
Vrushank | Portkey
Vrushank | Portkey
View on Discord
12/17/2024

New: Programatically create new virtual keys that refer to your Azure deployments.

New: Programatically create new virtual keys that refer to your Azure deployments. Docs: https://portkey.ai/docs/api-reference/admin-api/control-plane/virtual-keys/create-virtual-key...
Vrushank | Portkey
Vrushank | Portkey
View on Discord
12/16/2024

New on Portkey prompts: Move multiple prompts to another folder or delete them.

New on Portkey prompts: Move multiple prompts to another folder or delete them. Thanks to @Hammaad Siddeeqee for the suggestion!...
Vrushank | Portkey
Vrushank | Portkey
View on Discord
12/13/2024

@here Join in for our weekly office hours on https://discord.gg/J2mnVkWddu

@here Join in for our weekly office hours on https://discord.gg/J2mnVkWddu We'll discuss the latest updates to Portkey, customer success stories, and more!...
Vrushank | Portkey
Vrushank | Portkey
View on Discord
11/9/2024

**PROMPT FOLDERS ARE NOW LIVE**

PROMPT FOLDERS ARE NOW LIVE - Organize your prompts in folders - Move prompts from one folder to another ...
Vrushank | Portkey
Vrushank | Portkey
View on Discord
11/6/2024

Anthropic's prompt caching feature is now supported on the prompt playground!

Anthropic's prompt caching feature is now supported on the prompt playground! Set any message to be cached by just toggling the Cache Control setting in the UI. Special thanks to @rickydickydoo for championing this feature!...
No description
Vrushank | Portkey
Vrushank | Portkey
View on Discord
11/5/2024

Latest production-ready Gemini Pro & Flash models are now available on Portkey: https://new.portkey.

Latest production-ready Gemini Pro & Flash models are now available on Portkey: https://new.portkey.ai/announcements/gemini-002-models-now-available-on-portkey Key Improvements: * 50% reduced pricing on 1.5 Pro (for prompts <128K) * Increased rate limits: 2x higher on 1.5 Flash & 3x higher on 1.5 Pro...
Vrushank | Portkey
Vrushank | Portkey
View on Discord
10/9/2024

News that people here may like: We are participating in Hacktoberfest and giving away Airpods Pro +

News that people here may like: We are participating in Hacktoberfest and giving away Airpods Pro + some select Portkey swag to contributors! Just head over to the Gateway repo, look for issues tagged with hacktoberfest and start contributing! https://git.new/portkey...
No description
Vrushank | Portkey
Vrushank | Portkey
View on Discord
10/8/2024

The latest open source Llama 3.2 models - 1B & 3B Instruct and 11B & 90B Vision are now live on Port

The latest open source Llama 3.2 models - 1B & 3B Instruct and 11B & 90B Vision are now live on Portkey 🔥 ✅ Switch your OpenAI & Anthropic vision requests to the cheaper & faster Llama models instantly ✅ Mitigate API failures with production-grade fallbacks ✅ Create and deploy Llama prompt templates with ease...
No description
Next