Build reliable, secure, and production-ready AI apps easily.

pip install portkey-ai

💡 Features

🚪 AI Gateway:

• Unified API Signature: If you've used OpenAI, you already know how to use Portkey with any other provider.
• Interoperability: Write once, run with any provider. Switch between any model from any provider seamlessly.
• Automated Fallbacks & Retries: Ensure your application remains functional even if a primary service fails.
• Load Balancing: Efficiently distribute incoming requests among multiple models.
• Semantic Caching: Reduce costs and latency by intelligently caching results.

🔬 Observability:

• Logging: Keep track of all requests for monitoring and debugging.
• Requests Tracing: Understand the journey of each request for optimization.
• Custom Tags: Segment and categorize requests for better insights.

🚀 Quick Start

First, install the SDK & export Portkey API Key

Get Portkey API key here.

$ pip install portkey-ai
$ export PORTKEY_API_KEY=PORTKEY_API_KEY

Now, let's make a request with GPT-4

from portkey_ai import Portkey

# Construct a client with a virtual key
portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    virtual_key="VIRTUAL_KEY"
)

completion = portkey.chat.completions.create(
    messages = [{ "role": 'user', "content": 'Say this is a test' }],
    model = 'gpt-3.5-turbo'
)
print(completion)

Portkey fully adheres to the OpenAI SDK signature. This means that you can instantly switch to Portkey and start using Portkey's advanced production features right out of the box.

🤝 Supported Providers

	Provider	Support Status	Supported Endpoints
	OpenAI	✅ Supported	/completion, /embed
	Azure OpenAI	✅ Supported	/completion, /embed
	Anthropic	✅ Supported	/complete
	Anyscale	✅ Supported	/chat/completions
	Cohere	🚧 Coming Soon	generate, embed

---

📝 Full Documentation | 🛠️ Integration Requests |

🛠️ Contributing

Get started by checking out Github issues. Feel free to open an issue, or reach out if you would like to add to the project!