> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/block/goose/llms.txt
> Use this file to discover all available pages before exploring further.

# Custom Providers

> Creating and configuring custom AI providers

## Overview

Goose supports two approaches to adding custom providers:

1. **Declarative Providers** - JSON configuration files for OpenAI/Anthropic/Ollama-compatible APIs
2. **Rust Implementation** - Full `Provider` trait implementation for custom logic

This guide focuses on declarative providers, which cover most use cases.

## Declarative Providers

Declarative providers let you add OpenAI-compatible APIs without writing Rust code. They're perfect for:

* Commercial API providers (Together, Groq, Fireworks, etc.)
* Self-hosted LLM servers (vLLM, text-generation-inference)
* OpenRouter and other aggregator services
* Custom proxy endpoints

### Provider Engines

Choose the engine that matches your API format:

* **`openai`** - OpenAI Chat Completions API format
* **`anthropic`** - Anthropic Messages API format
* **`ollama`** - Ollama API format

Most providers use the OpenAI format.

## Configuration Structure

```json theme={null}
{
  "name": "custom_provider_id",
  "engine": "openai",
  "display_name": "My Custom Provider",
  "description": "Description of the provider",
  "api_key_env": "CUSTOM_PROVIDER_API_KEY",
  "base_url": "https://api.example.com",
  "models": [
    {
      "name": "model-name-1",
      "context_limit": 128000
    }
  ],
  "requires_auth": true,
  "supports_streaming": true,
  "timeout_seconds": 600,
  "headers": {
    "X-Custom-Header": "value"
  },
  "base_path": "v1/chat/completions"
}
```

### Configuration Fields

<ParamField path="name" type="string" required>
  Unique identifier for the provider (auto-generated from display\_name)
</ParamField>

<ParamField path="engine" type="string" required>
  API format: `openai`, `anthropic`, or `ollama`
</ParamField>

<ParamField path="display_name" type="string" required>
  Human-readable name shown in UIs
</ParamField>

<ParamField path="description" type="string">
  Optional description of the provider
</ParamField>

<ParamField path="api_key_env" type="string">
  Environment variable name for the API key (required if `requires_auth` is true)
</ParamField>

<ParamField path="base_url" type="string" required>
  Base URL of the API endpoint
</ParamField>

<ParamField path="models" type="array" required>
  List of available models with their context limits
</ParamField>

<ParamField path="requires_auth" type="boolean" default="true">
  Whether the provider requires authentication
</ParamField>

<ParamField path="supports_streaming" type="boolean" default="true">
  Whether the provider supports streaming responses
</ParamField>

<ParamField path="timeout_seconds" type="number" default="600">
  Request timeout in seconds
</ParamField>

<ParamField path="headers" type="object">
  Optional custom headers to include in requests
</ParamField>

<ParamField path="base_path" type="string">
  Optional API path (defaults based on engine)
</ParamField>

## Creating Custom Providers

### Method 1: Configuration File

Create a JSON file in `~/.config/goose/custom_providers/`:

```bash theme={null}
mkdir -p ~/.config/goose/custom_providers
cat > ~/.config/goose/custom_providers/groq.json <<EOF
{
  "name": "custom_groq",
  "engine": "openai",
  "display_name": "Groq",
  "description": "Fast LLM inference from Groq",
  "api_key_env": "GROQ_API_KEY",
  "base_url": "https://api.groq.com/openai/v1",
  "models": [
    {
      "name": "llama-3.3-70b-versatile",
      "context_limit": 32768
    },
    {
      "name": "mixtral-8x7b-32768",
      "context_limit": 32768
    }
  ],
  "requires_auth": true,
  "supports_streaming": true
}
EOF

# Set the API key
export GROQ_API_KEY="your-api-key"

# Use the provider
goose session start --provider custom_groq --model llama-3.3-70b-versatile
```

### Method 2: Programmatic Creation (Rust)

```rust theme={null}
use goose::config::declarative_providers::{
    create_custom_provider,
    CreateCustomProviderParams,
};
use goose::providers::base::ModelInfo;

let params = CreateCustomProviderParams {
    engine: "openai_compatible".to_string(),
    display_name: "Groq".to_string(),
    api_url: "https://api.groq.com/openai/v1".to_string(),
    api_key: "your-api-key".to_string(),
    models: vec![
        "llama-3.3-70b-versatile".to_string(),
        "mixtral-8x7b-32768".to_string(),
    ],
    supports_streaming: Some(true),
    headers: None,
    requires_auth: true,
    catalog_provider_id: None,
    base_path: None,
};

let config = create_custom_provider(params)?;
```

The API key is automatically stored in the system keyring.

## Example Providers

### Together.ai

```json theme={null}
{
  "name": "custom_together",
  "engine": "openai",
  "display_name": "Together AI",
  "api_key_env": "TOGETHER_API_KEY",
  "base_url": "https://api.together.xyz",
  "models": [
    {
      "name": "meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo",
      "context_limit": 8192
    },
    {
      "name": "mistralai/Mixtral-8x7B-Instruct-v0.1",
      "context_limit": 32768
    }
  ],
  "base_path": "v1/chat/completions"
}
```

### Fireworks.ai

```json theme={null}
{
  "name": "custom_fireworks",
  "engine": "openai",
  "display_name": "Fireworks AI",
  "api_key_env": "FIREWORKS_API_KEY",
  "base_url": "https://api.fireworks.ai",
  "models": [
    {
      "name": "accounts/fireworks/models/llama-v3p3-70b-instruct",
      "context_limit": 131072
    }
  ],
  "base_path": "inference/v1/chat/completions"
}
```

### OpenRouter

```json theme={null}
{
  "name": "custom_openrouter",
  "engine": "openai",
  "display_name": "OpenRouter",
  "api_key_env": "OPENROUTER_API_KEY",
  "base_url": "https://openrouter.ai/api",
  "models": [
    {
      "name": "anthropic/claude-sonnet-4",
      "context_limit": 200000
    },
    {
      "name": "openai/gpt-4o",
      "context_limit": 128000
    }
  ],
  "base_path": "v1/chat/completions",
  "headers": {
    "HTTP-Referer": "https://github.com/block/goose",
    "X-Title": "Goose Agent"
  }
}
```

### Self-Hosted vLLM

```json theme={null}
{
  "name": "custom_vllm",
  "engine": "openai",
  "display_name": "Local vLLM",
  "base_url": "http://localhost:8000",
  "models": [
    {
      "name": "meta-llama/Meta-Llama-3-70B-Instruct",
      "context_limit": 8192
    }
  ],
  "requires_auth": false,
  "base_path": "v1/chat/completions"
}
```

### Anthropic-Compatible Provider

For providers using Anthropic's API format:

```json theme={null}
{
  "name": "custom_anthropic_compatible",
  "engine": "anthropic",
  "display_name": "Anthropic-Compatible Provider",
  "api_key_env": "CUSTOM_API_KEY",
  "base_url": "https://api.example.com",
  "models": [
    {
      "name": "claude-compatible-model",
      "context_limit": 200000
    }
  ],
  "headers": {
    "anthropic-version": "2023-06-01"
  }
}
```

## Managing Custom Providers

### List Providers

```bash theme={null}
# List all providers including custom ones
goose provider list
```

```rust theme={null}
use goose::providers::providers;

let available_providers = providers();
for provider_name in available_providers {
    println!("Available: {}", provider_name);
}
```

### Update Provider

```rust theme={null}
use goose::config::declarative_providers::{
    update_custom_provider,
    UpdateCustomProviderParams,
};

let params = UpdateCustomProviderParams {
    id: "custom_groq".to_string(),
    engine: "openai_compatible".to_string(),
    display_name: "Groq (Updated)".to_string(),
    api_url: "https://api.groq.com/openai/v1".to_string(),
    api_key: "new-api-key".to_string(),
    models: vec!["llama-3.3-70b-versatile".to_string()],
    supports_streaming: Some(true),
    headers: None,
    requires_auth: true,
    catalog_provider_id: None,
    base_path: None,
};

update_custom_provider(params)?;
```

### Delete Provider

```bash theme={null}
rm ~/.config/goose/custom_providers/groq.json
```

Or programmatically:

```rust theme={null}
use goose::config::declarative_providers::{
    delete_custom_provider,
    custom_providers_dir,
};

delete_custom_provider("custom_groq")?;
```

### Refresh Providers

After adding/removing provider files:

```rust theme={null}
use goose::providers::refresh_custom_providers;

refresh_custom_providers().await?;
```

## Advanced Configuration

### Custom Headers

Add provider-specific headers:

```json theme={null}
{
  "headers": {
    "X-API-Version": "2024-01",
    "X-Custom-Header": "value"
  }
}
```

### Base Path Customization

Override the default API path:

```json theme={null}
{
  "engine": "openai",
  "base_url": "https://api.example.com",
  "base_path": "custom/v2/chat/completions"
}
```

### No Authentication

For providers that don't require auth:

```json theme={null}
{
  "requires_auth": false,
  "api_key_env": ""
}
```

### Multiple Model Configurations

```json theme={null}
{
  "models": [
    {
      "name": "small-model",
      "context_limit": 4096,
      "input_token_cost": 0.0000005,
      "output_token_cost": 0.0000015
    },
    {
      "name": "large-model",
      "context_limit": 32768,
      "input_token_cost": 0.000003,
      "output_token_cost": 0.000015
    }
  ]
}
```

## Implementing Provider Trait (Advanced)

For full control, implement the `Provider` trait:

```rust theme={null}
use goose::providers::base::{Provider, ProviderDef, ProviderMetadata};
use async_trait::async_trait;

pub struct MyCustomProvider {
    api_client: ApiClient,
    model: ModelConfig,
}

#[async_trait]
impl Provider for MyCustomProvider {
    fn get_name(&self) -> &str {
        "my_custom"
    }
    
    async fn stream(
        &self,
        model_config: &ModelConfig,
        session_id: &str,
        system: &str,
        messages: &[Message],
        tools: &[Tool],
    ) -> Result<MessageStream, ProviderError> {
        // Custom implementation
        todo!()
    }
    
    fn get_model_config(&self) -> ModelConfig {
        self.model.clone()
    }
}

impl ProviderDef for MyCustomProvider {
    type Provider = Self;
    
    fn metadata() -> ProviderMetadata {
        ProviderMetadata::new(
            "my_custom",
            "My Custom Provider",
            "Custom provider implementation",
            "default-model",
            vec!["model-1"],
            "https://docs.example.com",
            vec![],
        )
    }
    
    fn from_env(
        model: ModelConfig,
        _extensions: Vec<ExtensionConfig>,
    ) -> BoxFuture<'static, Result<Self::Provider>> {
        Box::pin(async move {
            // Initialize from environment
            Ok(MyCustomProvider {
                api_client: /* ... */,
                model,
            })
        })
    }
}
```

Then register it:

```rust theme={null}
use goose::providers::provider_registry::register_provider;

register_provider::<MyCustomProvider>();
```

## Troubleshooting

### Provider Not Found

Check that the JSON file is in the correct location:

```bash theme={null}
ls ~/.config/goose/custom_providers/
```

### Invalid Configuration

Validate your JSON:

```bash theme={null}
jq . ~/.config/goose/custom_providers/your-provider.json
```

### Authentication Errors

Verify the API key environment variable:

```bash theme={null}
echo $YOUR_PROVIDER_API_KEY
```

### Connection Errors

Test the API endpoint:

```bash theme={null}
curl -H "Authorization: Bearer $API_KEY" \
  https://api.example.com/v1/models
```

## See Also

* [Provider Overview](/api/providers/overview)
* [Provider Development](/development/provider-interface)
* [Declarative Providers Guide](/development/declarative-providers)
* [Configuration Reference](/api/core/config)