> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/block/goose/llms.txt
> Use this file to discover all available pages before exploring further.

# Custom Providers

> Create custom LLM provider integrations for goose

While goose supports 30+ LLM providers out of the box, you may need to integrate with a custom or proprietary provider. Goose offers two approaches: declarative configuration for OpenAI-compatible APIs, and full Rust trait implementation for complete control.

## When to Create a Custom Provider

Consider creating a custom provider when:

* Your organization has a proprietary LLM deployment
* You're using an OpenAI-compatible API that isn't built-in
* You need special authentication or request preprocessing
* You want to add custom retry logic or error handling
* You're using a provider with non-standard streaming formats

<Info>
  Before building a custom provider, check if your provider is OpenAI or Anthropic API-compatible. If so, the declarative approach is much simpler.
</Info>

## Declarative Providers (Recommended)

For OpenAI-compatible APIs, create a JSON configuration file:

### Basic Structure

Create `~/.config/goose/providers/my-provider.json`:

```json theme={null}
{
  "name": "my-provider",
  "engine": "openai",
  "host": "https://api.myprovider.com/v1",
  "models": [
    {
      "name": "my-model-7b",
      "context_window": 8192,
      "supports_tools": true,
      "supports_vision": false
    },
    {
      "name": "my-model-70b",
      "context_window": 32768,
      "supports_tools": true,
      "supports_vision": true
    }
  ],
  "auth": {
    "type": "header",
    "header_name": "Authorization",
    "header_value": "Bearer ${MY_PROVIDER_API_KEY}"
  }
}
```

### Configuration Fields

<ParamField path="name" type="string" required>
  Unique identifier for the provider
</ParamField>

<ParamField path="engine" type="string" required>
  API format to use: `openai`, `anthropic`, or `ollama`
</ParamField>

<ParamField path="host" type="string" required>
  Base URL for API requests
</ParamField>

<ParamField path="models" type="array" required>
  List of available models
</ParamField>

<ParamField path="models[].name" type="string" required>
  Model identifier
</ParamField>

<ParamField path="models[].context_window" type="integer">
  Maximum context size in tokens
</ParamField>

<ParamField path="models[].supports_tools" type="boolean" default="true">
  Whether the model supports tool/function calling
</ParamField>

<ParamField path="models[].supports_vision" type="boolean" default="false">
  Whether the model can process images
</ParamField>

<ParamField path="auth" type="object">
  Authentication configuration
</ParamField>

<ParamField path="auth.type" type="string">
  Authentication method: `header`, `query`, or `oauth`
</ParamField>

### Real-World Examples

<Tabs>
  <Tab title="Together.ai">
    ```json theme={null}
    {
      "name": "together",
      "engine": "openai",
      "host": "https://api.together.xyz/v1",
      "models": [
        {
          "name": "meta-llama/Llama-3.3-70B-Instruct-Turbo",
          "context_window": 131072,
          "supports_tools": true
        },
        {
          "name": "Qwen/Qwen3.5-72B-Instruct-Turbo",
          "context_window": 32768,
          "supports_tools": true
        }
      ],
      "auth": {
        "type": "header",
        "header_name": "Authorization",
        "header_value": "Bearer ${TOGETHER_API_KEY}"
      }
    }
    ```

    Set your API key:

    ```bash theme={null}
    export TOGETHER_API_KEY="your-key-here"
    ```
  </Tab>

  <Tab title="OpenRouter">
    ```json theme={null}
    {
      "name": "openrouter",
      "engine": "openai",
      "host": "https://openrouter.ai/api/v1",
      "models": [
        {
          "name": "anthropic/claude-4.5-sonnet",
          "context_window": 200000,
          "supports_tools": true
        },
        {
          "name": "google/gemini-4-flash-latest",
          "context_window": 1000000,
          "supports_tools": true,
          "supports_vision": true
        }
      ],
      "auth": {
        "type": "header",
        "header_name": "Authorization",
        "header_value": "Bearer ${OPENROUTER_API_KEY}"
      },
      "extra_headers": {
        "HTTP-Referer": "https://github.com/block/goose",
        "X-Title": "Goose AI Agent"
      }
    }
    ```
  </Tab>

  <Tab title="vLLM (Self-Hosted)">
    ```json theme={null}
    {
      "name": "vllm",
      "engine": "openai",
      "host": "http://localhost:8000/v1",
      "models": [
        {
          "name": "Qwen/Qwen3.5-32B-Instruct",
          "context_window": 32768,
          "supports_tools": true
        }
      ]
    }
    ```

    No API key needed for local deployment.
  </Tab>
</Tabs>

### Using Your Provider

After creating the configuration file, use it:

```bash theme={null}
export GOOSE_PROVIDER="my-provider"
export GOOSE_MODEL="my-model-70b"
goose session
```

Or configure interactively:

```bash theme={null}
goose configure
# Select your provider from the list
```

Goose automatically discovers provider JSON files in `~/.config/goose/providers/`.

## Programmatic Provider Implementation

For providers with non-standard APIs or special requirements, implement the `Provider` trait in Rust:

### Provider Trait Overview

```rust theme={null}
#[async_trait]
pub trait Provider: Send + Sync {
    // Core methods
    async fn complete(
        &self,
        messages: &[Message],
        config: &ModelConfig,
    ) -> Result<Response>;

    async fn stream(
        &self,
        messages: &[Message],
        config: &ModelConfig,
    ) -> Result<Pin<Box<dyn Stream<Item = Result<StreamChunk>> + Send>>>;

    fn metadata(&self) -> &ProviderMetadata;

    // Optional methods
    fn supports_tools(&self) -> bool { true }
    fn supports_vision(&self) -> bool { false }
    async fn list_models(&self) -> Result<Vec<String>> { Ok(vec![]) }
}
```

### Basic Implementation

Create a custom provider in `crates/goose/src/providers/my_provider.rs`:

```rust theme={null}
use anyhow::Result;
use async_trait::async_trait;
use crate::providers::base::{Provider, ProviderMetadata, Response};
use crate::model::{Message, ModelConfig};

pub struct MyProvider {
    client: reqwest::Client,
    api_key: String,
    base_url: String,
}

impl MyProvider {
    pub fn new(api_key: String) -> Result<Self> {
        Ok(Self {
            client: reqwest::Client::new(),
            api_key,
            base_url: "https://api.myprovider.com/v1".to_string(),
        })
    }
}

#[async_trait]
impl Provider for MyProvider {
    async fn complete(
        &self,
        messages: &[Message],
        config: &ModelConfig,
    ) -> Result<Response> {
        let request_body = serde_json::json!({
            "model": config.model,
            "messages": messages,
            "temperature": config.temperature,
            "max_tokens": config.max_tokens,
        });

        let response = self.client
            .post(&format!("{}/chat/completions", self.base_url))
            .header("Authorization", format!("Bearer {}", self.api_key))
            .json(&request_body)
            .send()
            .await?;

        let json: serde_json::Value = response.json().await?;
        
        // Parse response and return
        Ok(Response {
            content: json["choices"][0]["message"]["content"]
                .as_str()
                .unwrap_or("") 
                .to_string(),
            usage: extract_usage(&json),
            model: config.model.clone(),
        })
    }

    async fn stream(
        &self,
        messages: &[Message],
        config: &ModelConfig,
    ) -> Result<Pin<Box<dyn Stream<Item = Result<StreamChunk>> + Send>>> {
        // Implement streaming if supported
        todo!("Streaming implementation")
    }

    fn metadata(&self) -> &ProviderMetadata {
        &ProviderMetadata {
            name: "my-provider".to_string(),
            display_name: "My Custom Provider".to_string(),
            supported_models: vec!["model-7b".to_string(), "model-70b".to_string()],
            requires_auth: true,
        }
    }
}
```

### Register the Provider

Add your provider to the registry in `crates/goose/src/providers/provider_registry.rs`:

```rust theme={null}
use crate::providers::my_provider::MyProvider;

pub fn create_provider(name: &str, config: &Config) -> Result<Box<dyn Provider>> {
    match name {
        "my-provider" => {
            let api_key = config.get_secret("MY_PROVIDER_API_KEY")?;
            Ok(Box::new(MyProvider::new(api_key)?))
        },
        // ... other providers
    }
}
```

### Advanced Features

#### OAuth Authentication

For providers requiring OAuth:

```rust theme={null}
use crate::oauth::{OAuthClient, OAuthConfig};

pub struct MyOAuthProvider {
    oauth_client: OAuthClient,
    // ...
}

impl MyOAuthProvider {
    pub async fn new(config: &OAuthConfig) -> Result<Self> {
        let oauth_client = OAuthClient::new(config).await?;
        Ok(Self { oauth_client })
    }

    async fn get_access_token(&self) -> Result<String> {
        self.oauth_client.get_token().await
    }
}
```

See the [OAuth Providers guide](/development/oauth-providers) for a complete implementation.

#### Retry Logic

Implement custom retry behavior:

```rust theme={null}
use crate::providers::retry::{RetryConfig, retry_with_backoff};

impl MyProvider {
    async fn request_with_retry<F, T>(
        &self,
        f: F,
    ) -> Result<T>
    where
        F: Fn() -> BoxFuture<'static, Result<T>>,
    {
        let retry_config = RetryConfig {
            max_retries: 3,
            initial_delay_ms: 1000,
            max_delay_ms: 60000,
            exponential_base: 2.0,
        };
        
        retry_with_backoff(f, &retry_config).await
    }
}
```

#### Model Discovery

Allow dynamic model listing:

```rust theme={null}
#[async_trait]
impl Provider for MyProvider {
    async fn list_models(&self) -> Result<Vec<String>> {
        let response = self.client
            .get(&format!("{}/models", self.base_url))
            .header("Authorization", format!("Bearer {}", self.api_key))
            .send()
            .await?;

        let json: serde_json::Value = response.json().await?;
        let models = json["models"]
            .as_array()
            .unwrap_or(&vec![])
            .iter()
            .filter_map(|m| m["id"].as_str().map(String::from))
            .collect();

        Ok(models)
    }
}
```

## Testing Your Provider

### Unit Tests

Test provider functionality:

```rust theme={null}
#[cfg(test)]
mod tests {
    use super::*;

    #[tokio::test]
    async fn test_complete() {
        let provider = MyProvider::new("test-key".to_string()).unwrap();
        let messages = vec![Message::user("Hello")];
        let config = ModelConfig::default();

        let response = provider.complete(&messages, &config).await;
        assert!(response.is_ok());
    }
}
```

### Integration Tests

Test with goose:

```bash theme={null}
export MY_PROVIDER_API_KEY="your-key"
export GOOSE_PROVIDER="my-provider"
goose session
```

## Next Steps

<CardGroup cols={2}>
  <Card title="Provider Interface" icon="code" href="/development/provider-interface">
    Complete guide to implementing the Provider trait
  </Card>

  <Card title="Declarative Providers" icon="file-code" href="/development/declarative-providers">
    JSON configuration reference for OpenAI-compatible providers
  </Card>

  <Card title="OAuth Providers" icon="key" href="/development/oauth-providers">
    Implement OAuth authentication flows
  </Card>

  <Card title="Providers Overview" icon="plug" href="/api/providers/overview">
    API reference for the Provider trait
  </Card>
</CardGroup>