Custom Providers

Create custom LLM provider integrations for goose

While goose supports 30+ LLM providers out of the box, you may need to integrate with a custom or proprietary provider. Goose offers two approaches: declarative configuration for OpenAI-compatible APIs, and full Rust trait implementation for complete control.

When to Create a Custom Provider

Consider creating a custom provider when:

• Your organization has a proprietary LLM deployment
• You’re using an OpenAI-compatible API that isn’t built-in
• You need special authentication or request preprocessing
• You want to add custom retry logic or error handling
• You’re using a provider with non-standard streaming formats

Before building a custom provider, check if your provider is OpenAI or Anthropic API-compatible. If so, the declarative approach is much simpler.

Declarative Providers (Recommended)

For OpenAI-compatible APIs, create a JSON configuration file:

Basic Structure

Create ~/.config/goose/providers/my-provider.json:

{
  "name": "my-provider",
  "engine": "openai",
  "host": "https://api.myprovider.com/v1",
  "models": [
    {
      "name": "my-model-7b",
      "context_window": 8192,
      "supports_tools": true,
      "supports_vision": false
    },
    {
      "name": "my-model-70b",
      "context_window": 32768,
      "supports_tools": true,
      "supports_vision": true
    }
  ],
  "auth": {
    "type": "header",
    "header_name": "Authorization",
    "header_value": "Bearer ${MY_PROVIDER_API_KEY}"
  }
}

Configuration Fields

name · string · required

Unique identifier for the provider

engine · string · required

API format to use: openai, anthropic, or ollama

host · string · required

Base URL for API requests

models · array · required

List of available models

models[].name · string · required

Model identifier

models[].context_window · integer

Maximum context size in tokens

models[].supports_tools · boolean · default: "true"

Whether the model supports tool/function calling

models[].supports_vision · boolean · default: "false"

Whether the model can process images

auth · object

Authentication configuration

auth.type · string

Authentication method: header, query, or oauth

Real-World Examples

Together.ai

{
  "name": "together",
  "engine": "openai",
  "host": "https://api.together.xyz/v1",
  "models": [
    {
      "name": "meta-llama/Llama-3.3-70B-Instruct-Turbo",
      "context_window": 131072,
      "supports_tools": true
    },
    {
      "name": "Qwen/Qwen3.5-72B-Instruct-Turbo",
      "context_window": 32768,
      "supports_tools": true
    }
  ],
  "auth": {
    "type": "header",
    "header_name": "Authorization",
    "header_value": "Bearer ${TOGETHER_API_KEY}"
  }
}

Set your API key:

export TOGETHER_API_KEY="your-key-here"

OpenRouter

{
  "name": "openrouter",
  "engine": "openai",
  "host": "https://openrouter.ai/api/v1",
  "models": [
    {
      "name": "anthropic/claude-4.5-sonnet",
      "context_window": 200000,
      "supports_tools": true
    },
    {
      "name": "google/gemini-4-flash-latest",
      "context_window": 1000000,
      "supports_tools": true,
      "supports_vision": true
    }
  ],
  "auth": {
    "type": "header",
    "header_name": "Authorization",
    "header_value": "Bearer ${OPENROUTER_API_KEY}"
  },
  "extra_headers": {
    "HTTP-Referer": "https://github.com/block/goose",
    "X-Title": "Goose AI Agent"
  }
}

vLLM (Self-Hosted)

{
  "name": "vllm",
  "engine": "openai",
  "host": "http://localhost:8000/v1",
  "models": [
    {
      "name": "Qwen/Qwen3.5-32B-Instruct",
      "context_window": 32768,
      "supports_tools": true
    }
  ]
}

No API key needed for local deployment.

Using Your Provider

After creating the configuration file, use it:

export GOOSE_PROVIDER="my-provider"
export GOOSE_MODEL="my-model-70b"
goose session

Or configure interactively:

goose configure
# Select your provider from the list

Goose automatically discovers provider JSON files in ~/.config/goose/providers/.

Programmatic Provider Implementation

For providers with non-standard APIs or special requirements, implement the Provider trait in Rust:

Provider Trait Overview

#[async_trait]
pub trait Provider: Send + Sync {
    // Core methods
    async fn complete(
        &self,
        messages: &[Message],
        config: &ModelConfig,
    ) -> Result<Response>;

    async fn stream(
        &self,
        messages: &[Message],
        config: &ModelConfig,
    ) -> Result<Pin<Box<dyn Stream<Item = Result<StreamChunk>> + Send>>>;

    fn metadata(&self) -> &ProviderMetadata;

    // Optional methods
    fn supports_tools(&self) -> bool { true }
    fn supports_vision(&self) -> bool { false }
    async fn list_models(&self) -> Result<Vec<String>> { Ok(vec![]) }
}

Basic Implementation

Create a custom provider in crates/goose/src/providers/my_provider.rs:

use anyhow::Result;
use async_trait::async_trait;
use crate::providers::base::{Provider, ProviderMetadata, Response};
use crate::model::{Message, ModelConfig};

pub struct MyProvider {
    client: reqwest::Client,
    api_key: String,
    base_url: String,
}

impl MyProvider {
    pub fn new(api_key: String) -> Result<Self> {
        Ok(Self {
            client: reqwest::Client::new(),
            api_key,
            base_url: "https://api.myprovider.com/v1".to_string(),
        })
    }
}

#[async_trait]
impl Provider for MyProvider {
    async fn complete(
        &self,
        messages: &[Message],
        config: &ModelConfig,
    ) -> Result<Response> {
        let request_body = serde_json::json!({
            "model": config.model,
            "messages": messages,
            "temperature": config.temperature,
            "max_tokens": config.max_tokens,
        });

        let response = self.client
            .post(&format!("{}/chat/completions", self.base_url))
            .header("Authorization", format!("Bearer {}", self.api_key))
            .json(&request_body)
            .send()
            .await?;

        let json: serde_json::Value = response.json().await?;
        
        // Parse response and return
        Ok(Response {
            content: json["choices"][0]["message"]["content"]
                .as_str()
                .unwrap_or("") 
                .to_string(),
            usage: extract_usage(&json),
            model: config.model.clone(),
        })
    }

    async fn stream(
        &self,
        messages: &[Message],
        config: &ModelConfig,
    ) -> Result<Pin<Box<dyn Stream<Item = Result<StreamChunk>> + Send>>> {
        // Implement streaming if supported
        todo!("Streaming implementation")
    }

    fn metadata(&self) -> &ProviderMetadata {
        &ProviderMetadata {
            name: "my-provider".to_string(),
            display_name: "My Custom Provider".to_string(),
            supported_models: vec!["model-7b".to_string(), "model-70b".to_string()],
            requires_auth: true,
        }
    }
}

Register the Provider

Add your provider to the registry in crates/goose/src/providers/provider_registry.rs:

use crate::providers::my_provider::MyProvider;

pub fn create_provider(name: &str, config: &Config) -> Result<Box<dyn Provider>> {
    match name {
        "my-provider" => {
            let api_key = config.get_secret("MY_PROVIDER_API_KEY")?;
            Ok(Box::new(MyProvider::new(api_key)?))
        },
        // ... other providers
    }
}

Advanced Features

OAuth Authentication

For providers requiring OAuth:

use crate::oauth::{OAuthClient, OAuthConfig};

pub struct MyOAuthProvider {
    oauth_client: OAuthClient,
    // ...
}

impl MyOAuthProvider {
    pub async fn new(config: &OAuthConfig) -> Result<Self> {
        let oauth_client = OAuthClient::new(config).await?;
        Ok(Self { oauth_client })
    }

    async fn get_access_token(&self) -> Result<String> {
        self.oauth_client.get_token().await
    }
}

See the OAuth Providers guide for a complete implementation.

Retry Logic

Implement custom retry behavior:

use crate::providers::retry::{RetryConfig, retry_with_backoff};

impl MyProvider {
    async fn request_with_retry<F, T>(
        &self,
        f: F,
    ) -> Result<T>
    where
        F: Fn() -> BoxFuture<'static, Result<T>>,
    {
        let retry_config = RetryConfig {
            max_retries: 3,
            initial_delay_ms: 1000,
            max_delay_ms: 60000,
            exponential_base: 2.0,
        };
        
        retry_with_backoff(f, &retry_config).await
    }
}

Model Discovery

Allow dynamic model listing:

#[async_trait]
impl Provider for MyProvider {
    async fn list_models(&self) -> Result<Vec<String>> {
        let response = self.client
            .get(&format!("{}/models", self.base_url))
            .header("Authorization", format!("Bearer {}", self.api_key))
            .send()
            .await?;

        let json: serde_json::Value = response.json().await?;
        let models = json["models"]
            .as_array()
            .unwrap_or(&vec![])
            .iter()
            .filter_map(|m| m["id"].as_str().map(String::from))
            .collect();

        Ok(models)
    }
}

Testing Your Provider

Unit Tests

Test provider functionality:

#[cfg(test)]
mod tests {
    use super::*;

    #[tokio::test]
    async fn test_complete() {
        let provider = MyProvider::new("test-key".to_string()).unwrap();
        let messages = vec![Message::user("Hello")];
        let config = ModelConfig::default();

        let response = provider.complete(&messages, &config).await;
        assert!(response.is_ok());
    }
}

Integration Tests

Test with goose:

export MY_PROVIDER_API_KEY="your-key"
export GOOSE_PROVIDER="my-provider"
goose session

Next Steps

Provider Interface

Complete guide to implementing the Provider trait

Declarative Providers

JSON configuration reference for OpenAI-compatible providers

OAuth Providers

Implement OAuth authentication flows

Providers Overview

API reference for the Provider trait