async-openai

Async Rust library for OpenAI

Overview

async-openai is an unofficial Rust library for OpenAI, based on OpenAI OpenAPI spec.

• Requests are retried with exponential backoff when rate limited.
• Ergonomic builder pattern for all request objects.
• SSE streaming.
• Granular feature flags to enable any types or apis.
• WASM.
• Middleware support with tower ecosystem.

+ OpenAI compatible providers

• Bring your own custom types for Request or Response objects.
• Customize path, query and headers per request or for all requests.
• Microsoft Azure OpenAI Service.

Feature Flags

What	APIs	Crate Feature Flags
Responses API	Responses, Conversations, Streaming events, Websocket Events	responses
Webhooks	Webhook Events	webhook
Platform APIs	Audio, Audio Streaming, Videos, Images, Image Streaming, Embeddings, Evals, Fine-tuning, Graders, Batch, Files, Uploads, Models, Moderations, Safety Alerts, Content Provenance Checks	audio, video, image, embedding, evals, finetuning, grader, batch, file, upload, model, moderation, safety, content-provenance-checks
Vector stores	Vector stores, Vector store files, Vector store file batches	vectorstore
ChatKit (Beta)	ChatKit	chatkit
Containers	Containers, Container Files	container
Skills	Skills	skill
Realtime	Realtime Calls, Client secrets, Client events, Server events	realtime
Chat Completions	Chat Completions, Streaming	chat-completion
Administration	Admin API Keys, Audit Logs, Certificates, Data Retention, Groups, Invites, Projects, Roles, Spend Alerts, Spend Limits, Usage, Users; And their nested APIs	administration
Legacy	Completions	completions

Usage

The library reads API key from the environment variable OPENAI_API_KEY.

# On macOS/Linux
export OPENAI_API_KEY='sk-...'

# On Windows Powershell
$Env:OPENAI_API_KEY='sk-...'

Other official environment variables supported are: OPENAI_ADMIN_KEY, OPENAI_BASE_URL, OPENAI_ORG_ID, OPENAI_PROJECT_ID

• Visit examples directory on how to use async-openai.
• Visit docs.rs/async-openai for docs.

Image Generation Example

use async_openai::{
    types::images::{CreateImageRequestArgs, ImageModel, ImageSize},
    Client,
};
use std::error::Error;

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    // create client, reads OPENAI_API_KEY environment variable for API key.
    let client = Client::new();

    let request = CreateImageRequestArgs::default()
        .model(ImageModel::GptImage2)
        .prompt("cats on sofa and carpet in living room")
        .n(2)
        .size(ImageSize::Auto)
        .user("async-openai")
        .build()?;

    let response = client.images().generate(request).await?;

    // Concurrently save each image in its own Tokio task.
    // Create directory if it doesn't exist.
    let paths = response.save("./data").await?;

    paths
        .iter()
        .for_each(|path| println!("Image file path: {}", path.display()));

    Ok(())
}

OpenAI Compatible Providers

Even though the scope of the crate is official OpenAI APIs, it is very configurable to work with compatible providers.

Bring Your Own Types

Enable methods whose input and outputs are generics with byot feature. It creates a new method with same name and _byot suffix.

For example, to use serde_json::Value as request and response type:

let response: Value = client
        .chat()
        .create_byot(json!({
            "messages": [
                {
                    "role": "developer",
                    "content": "You are a helpful assistant"
                },
                {
                    "role": "user",
                    "content": "What do you think about life?"
                }
            ],
            "model": "gpt-4o",
            "store": false
        }))
        .await?;

This can be useful in many scenarios:

• When shape of request/response in OpenAI-compatible APIs don't exactly match OpenAI.
• Extend existing types in this crate with new fields like extra_body (with serde flatten)
• To avoid typing verbose types.
• To escape deserialization errors on expected type and actual response mismatch.

*_byot methods require same trait bounds as regular methods.

Visit examples/bring-your-own-type directory to learn more.

References: Borrow Instead of Move

With byot use reference to request types

let response: Response = client
  .responses()
  .create_byot(&request).await?

Visit examples/borrow-instead-of-move to learn more.

Configurable Requests

Configure path, headers, and query parameters for a HTTP request.

Request Options

Use path(), .query(), .header(), .headers() on the API group. Path overrides the default path but all other methods are additive - adds to existing query or headers.

For demonstration:

client.
  .chat()
  // override default path
  .path("/v1/messages")
  // query can be a struct or a map too - additive
  .query(&[("limit", "10")])?
  // header for unique id for this API request - additive
  .header("x-request-id", "id123")?
  .list()
  .await?

Modifying all Requests

Use Config, OpenAIConfig etc. for configuring url, headers or query parameters globally for all requests.

Dynamic Dispatch

This allows you to use same code (say a fn) to call APIs on different OpenAI-compatible providers.

Create a client with Box or Arc wrapped configuration.

For example:

use async_openai::{Client, config::{Config, OpenAIConfig}};

// Use `Box` or `std::sync::Arc` to wrap the config
let config = Box::new(OpenAIConfig::default()) as Box<dyn Config>;
// create client
let client: Client<Box<dyn Config>> = Client::with_config(config);

// A function can now accept a `&Client<Box<dyn Config>>` parameter
// which can invoke any openai compatible api
fn chat_completion(client: &Client<Box<dyn Config>>) { 
    todo!() 
}

Rust Types

To only use Rust types from the crate - disable default features and use feature flag types.

There are granular feature flags like response-types, chat-completion-types, etc.

These granular types are enabled when the corresponding API feature is enabled - for example responses will enable response-types.

TLS backends

The crate exposes the underlying reqwest TLS options as Cargo features. Pick exactly one; disable default features when choosing anything other than rustls.

Webhooks

Support for webhook includes event types, signature verification, and building webhook events from payloads.

Middleware

Middleware is supported via Tower ecosystem, which can be enabled with middleware feature. See middleware for more detail.

Contributing

🎉 Thank you for taking the time to contribute and improve the project. I'd be happy to have you!

Please see contributing guide!

Complimentary Crates

• async-openai-wasm provides WASM support.
• openai-func-enums macros for working with function/tool calls.

License

This project is licensed under MIT license.