Model configuration in Elastic Agent Builder

Elastic Agent Builder uses large language models (LLMs) to power agent reasoning and decision-making.

For model performance for Observability and Elastic Security, refer to Configure access to LLMs. The Recommended models section focuses on Elastic Agent Builder.

For Elastic Cloud Serverless projects and Elastic Cloud Hosted deployments, Elastic Agent Builder uses Elastic Managed LLMs running on the Elastic Inference Service (EIS). This managed service requires zero setup.

Use the model selector in the chat interface to switch between available models. The selector displays all configured models, including preconfigured models (on Elastic Cloud Hosted and Elastic Cloud Serverless) and any custom inference endpoints or connectors you set up.

To learn more, refer to select a different model.

×

When you call an agent through the Converse API, you can override the agent's configured model for that request. Pass one of the following in the request body:

• inference_id: the ID of an inference endpoint.
• connector_id: the ID of a connector.

Both parameters route the request to the same underlying model and are mutually exclusive. If you send both, the request fails with a 400 error. If you omit both, the agent uses its default model.

For an example request, refer to Route a request to a specific model.

For more information about these settings, refer to Manage access to AI features.

To give you faster responses at a lower token cost, Elastic Agent Builder automatically routes some low-effort, latency-sensitive operations to a fast model instead of your default model. A fast model is a smaller, high-throughput model that returns results more quickly than a frontier reasoning model.

This routing is automatic and cannot be turned off. You do not need to configure anything to benefit from it. Your default model still handles agent reasoning, tool selection, and response synthesis.

Elastic Agent Builder uses the fast model for well-scoped tasks that do not need deep reasoning, such as ES|QL query generation and conversation title generation.

On Elastic Cloud Serverless projects and Elastic Cloud Hosted deployments, the fast model is preconfigured through Elastic Managed LLMs on the Elastic Inference Service (EIS), with no setup. Claude Haiku 4.5 is one example. The latency improvement is most noticeable on Elastic Cloud Serverless.

On Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed deployments, there is no preconfigured fast model unless you connect to EIS through Cloud Connect. Without it, fast model routing falls back to your default model.

Elastic Agent Builder uses a recommended Elastic Managed LLM for fast routing. When more than one recommended fast model is available, such as Claude Haiku 4.5 and Google Gemini Flash, it uses the first one. To prefer a specific model, open the Feature settings page, go to the Agent Builder section, and order the models under Fast models so your preferred one is at the top. To find this page, refer to Change the default model.

In 9.5, only recommended Elastic Managed LLMs are used for fast routing. Assigning a custom or non-recommended model has no effect on the fast model.

×

If no fast model is available, Elastic Agent Builder uses your default model instead. Fast model routing is an optimization, so a missing fast model reduces the latency benefit but does not cause errors.

You can configure additional models for Elastic Agent Builder in two ways:

Configure an inference endpoint for your model provider. The endpoint must support the chat_completion task type.

Choose based on how the model is hosted:

• Elastic-managed models: Add an endpoint in Elastic inference.
• Third-party providers (such as OpenAI, Anthropic, or Amazon Bedrock): Add an endpoint in External inference.

If you prefer not to use the UI, you can also create endpoints with the inference APIs. See Create models programmatically.

To use additional models that aren't preconfigured, create a connector for your model provider.

1. Find connectors under Alerts and Insights / Connectors in the global search bar.

2. Select Create Connector and select your model provider.

3. Configure the connector with your API credentials and preferred model.

4. Expand Additional settings and select chat_completion as the task type.

   
   ×

You can create either an inference endpoint or a connector through the API:

• Create an inference endpoint with the inference APIs.
• Create a connector with the Connectors API. Works on all deployments.

You can connect a locally hosted LLM to Elastic using the OpenAI connector. This requires your local LLM to be compatible with the OpenAI API format.

For detailed setup instructions, refer to the OpenAI connector documentation.

Elastic Agent Builder requires models with strong reasoning and tool-calling capabilities. State-of-the-art models perform significantly better than smaller or older models.

Agent Builder relies on advanced LLM capabilities including:

• Function calling: Models must accurately select appropriate tools and construct valid parameters from natural language requests.
• Multi-step reasoning: Agents need to plan, execute, and adapt based on tool results across multiple iterations.
• Structured output: Models must produce properly formatted responses that the agent framework can parse.

While Elastic Agent Builder can be configured to use LLMs from many different vendors, not all LLMs are robust enough for reliable agent workflows.

The following models are known to work well with Elastic Agent Builder. These categories represent a spectrum from maximum reasoning capability to maximum throughput. Choose based on your latency, cost, and complexity requirements.

Smaller or less capable models may produce errors like:

Error: Invalid function call syntax
		

Error executing agent: No tool calls found in the response.
		

Although any chat-completion-compatible model can technically be configured, use state-of-the-art models for reliable agent performance.

• Elastic Inference Service (EIS)
• External inference
• Manage access to AI features
• Configure access to LLMs
• LLM performance matrix for Observability
• LLM performance matrix for Elastic Security
• Limitations and known issues
• Get started
• Connectors