Loading

Using different models in Elastic Agent Builder

Elastic Agent Builder uses large language models (LLMs) to power agent reasoning and decision-making. By default, agents use an Elastic Managed LLM.

You can use additional models by configuring a connector.

Refer to select a different model to learn how to switch configured models in the UI.

By default, Elastic Agent Builder uses the Elastic Managed LLM connector to interface with models running on the Elastic Inference Service.

This managed service requires zero setup and no additional API key management.

Learn more about the Elastic Managed LLM connector and pricing.

By default, Elastic Agent Builder uses an Elastic Managed LLM. To use a different model, select a configured connector and set it as the default.

  1. Search for GenAI Settings in the global search field
  2. Select your preferred connector from the Default AI Connector dropdown
  3. Save your changes
  1. Find connectors under Alerts and Insights / Connectors in the global search bar
  2. Select Create Connector and select your model provider
  3. Configure the connector with your API credentials and preferred model
  4. Expand Additional settings and select chat_completion as the task type Additional settings expanded showing chat_completion task type selected
  5. Search for GenAI Settings in the global search field
  6. Select your new connector from the Default AI Connector dropdown under Custom connectors
  7. Save your changes

For detailed instructions on creating connectors, refer to Connectors.

Learn more about preconfigured connectors.

You can connect a locally hosted LLM to Elastic using the OpenAI connector. This requires your local LLM to be compatible with the OpenAI API format.

Refer to the OpenAI connector documentation for detailed setup instructions.

To create connectors programmatically, refer to the Connectors API documentation.

Elastic Agent Builder requires models with strong reasoning and tool-calling capabilities. State-of-the-art models perform significantly better than smaller or older models.

Agent Builder relies on advanced LLM capabilities including:

  • Function calling: Models must accurately select appropriate tools and construct valid parameters from natural language requests
  • Multi-step reasoning: Agents need to plan, execute, and adapt based on tool results across multiple iterations
  • Structured output: Models must produce properly formatted responses that the agent framework can parse

While Elastic offers LLM connectors for many different vendors and models, not all LLMs are robust enough to be used with Elastic Agent Builder.

The following models are known to work well with Elastic Agent Builder. These categories represent a spectrum from maximum reasoning capability to maximum throughput. Choose based on your latency, cost, and complexity requirements.

Category Model examples Use cases Trade-offs
Extended reasoning - Gemini 3 Pro
- Claude 4.5 Opus		 Open-ended exploration, multi-step planning, and complex analysis Higher latency and cost; best for latency-insensitive, batch, or async workflows
Balanced performance - GPT-5.2
- Claude 4.5 Sonnet		 General-purpose agents requiring reliable tool orchestration and data retrieval and synthesis Moderate cost; suitable for real-time and interactive use
High throughput GPT-OSS-120B Latency-sensitive pipelines and high-concurrency scenarios with well-scoped tasks Lower reasoning depth; smaller context window. Ideal for air-gapped deployments
Tip

For agents working with large documents or conversation histories, consider models with extended context windows. For example, Claude 4.5 Sonnet and Gemini 3 Pro support up to 1M tokens. Check your model provider's documentation for specific context limits.

Smaller or less capable models may produce errors like:

Error: Invalid function call syntax
		
Error executing agent: No tool calls found in the response.

While any chat-completion-compatible connector can technically be configured, we strongly recommend using state-of-the-art models for reliable agent performance.

Note

Smaller or "mini" model variants are not recommended for Elastic Agent Builder as they lack the necessary capabilities for reliable agent workflows.