Providers Overview

Understanding and working with AI model providers in Goose

What are Providers?

Providers are the abstraction layer in Goose that connects to different AI model services. Each provider implements the Provider trait, which standardizes how Goose communicates with various AI backends like Anthropic, OpenAI, Databricks, and Ollama.

Provider Trait

All providers implement the Provider trait, which defines the core interface:

pub trait Provider: Send + Sync {
    fn get_name(&self) -> &str;
    
    async fn stream(
        &self,
        model_config: &ModelConfig,
        session_id: &str,
        system: &str,
        messages: &[Message],
        tools: &[Tool],
    ) -> Result<MessageStream, ProviderError>;
    
    async fn complete(
        &self,
        model_config: &ModelConfig,
        session_id: &str,
        system: &str,
        messages: &[Message],
        tools: &[Tool],
    ) -> Result<(Message, ProviderUsage), ProviderError>;
    
    fn get_model_config(&self) -> ModelConfig;
    
    async fn fetch_supported_models(&self) -> Result<Vec<String>, ProviderError>;
}

Key Methods

stream()

The primary method for streaming responses from the AI model. Returns a MessageStream that yields partial messages as they’re generated.

complete()

A convenience method that collects the entire stream into a single message and usage information. Most implementations delegate to stream() and collect the results.

get_model_config()

Returns the current model configuration including model name, temperature, and other parameters.

fetch_supported_models()

Queries the provider’s API to get a list of available models.

Provider Metadata

Each provider defines metadata through the ProviderDef trait:

pub struct ProviderMetadata {
    pub name: String,
    pub display_name: String,
    pub description: String,
    pub default_model: String,
    pub known_models: Vec<ModelInfo>,
    pub model_doc_link: String,
    pub config_keys: Vec<ConfigKey>,
}

Configuration Keys

Providers specify their required configuration through ConfigKey:

pub struct ConfigKey {
    pub name: String,          // e.g., "ANTHROPIC_API_KEY"
    pub required: bool,        // Whether this key is required
    pub secret: bool,          // Should be stored securely
    pub default: Option<String>,
    pub oauth_flow: bool,      // Uses OAuth instead of API key
    pub primary: bool,         // Show prominently in setup UI
}

Model Configuration

The ModelConfig struct controls model behavior:

pub struct ModelConfig {
    pub model_name: String,
    pub temperature: Option<f32>,
    pub top_p: Option<f32>,
    pub max_tokens: Option<i32>,
    pub context_limit: Option<usize>,
    // ... other parameters
}

Usage and Pricing

Providers track token usage through ProviderUsage:

pub struct ProviderUsage {
    pub model: String,
    pub usage: Usage,
}

pub struct Usage {
    pub input_tokens: Option<i32>,
    pub output_tokens: Option<i32>,
    pub total_tokens: Option<i32>,
}

Built-in Providers

Goose includes several built-in providers:

• Anthropic - Claude models (Sonnet, Opus, Haiku)
• OpenAI - GPT-4, GPT-4o, and other OpenAI models
• Databricks - Models on Databricks AI Gateway
• Ollama - Local open-source models
• Azure OpenAI - OpenAI models hosted on Azure
• Google Vertex AI - Google’s AI models
• AWS Bedrock - Amazon’s model marketplace

Custom Providers

You can create custom providers in two ways:

1. Declarative Providers - JSON configuration files for OpenAI-compatible APIs
2. Rust Implementation - Implement the Provider trait for full control

See the Custom Providers guide for details.

Provider Registry

Providers are registered at startup and can be retrieved by name:

use goose::providers::{create, providers};

// Get available provider names
let provider_names = providers();

// Create a provider instance
let provider = create("anthropic", model_config, extensions).await?;

Error Handling

Providers use the ProviderError enum for error handling:

pub enum ProviderError {
    Authentication(String),
    RateLimited { retry_after: Option<Duration> },
    ContextLengthExceeded(String),
    RequestFailed(String),
    ExecutionError(String),
    UsageError(String),
}

Retry Configuration

Providers support configurable retry logic:

pub struct RetryConfig {
    pub max_retries: usize,
    pub initial_interval_ms: u64,
    pub backoff_multiplier: f64,
    pub max_interval_ms: u64,
}

Next Steps

• Explore specific provider implementations:
  • Anthropic Provider
  • OpenAI Provider
  • Databricks Provider
  • Ollama Provider
• Learn how to create custom providers