LLM tracing in Elastic APM: prompts, responses, and token counts in the span view

Elastic APM now does LLM tracing in the trace view. The GenAI tab in the span flyout has the whole conversation, so you can read the system prompt, the user messages and the model response, and copy any of them. Every GenAI span row in the waterfall shows input and output token counts, so in an agentic trace with twenty LLM calls you can find the span using the most tokens without opening any of them. Your LLM calls are now in the same waterfall as your database queries and HTTP spans.

Both features follow the OTel GenAI semantic conventions and work with any OTel-instrumented provider. If your framework already emits OTel GenAI span attributes, there is nothing to change.

How OTel GenAI spans are structured

A GenAI span stores everything as span attributes. A typical chat span includes:

• gen_ai.provider.name: the provider (openai, anthropic, aws.bedrock, etc.); gen_ai.system is supported as a fallback for older instrumentation.
• gen_ai.operation.name: the operation type (chat, embeddings, etc.).
• gen_ai.request.model: the model being called.
• gen_ai.usage.input_tokens: tokens consumed by the prompt.
• gen_ai.usage.output_tokens: tokens generated in the response.
• gen_ai.input.messages, gen_ai.output.messages: conversation messages.
• gen_ai.system_instructions: the system prompt.

Both features read from these attributes:

How to read LLM prompts and responses in the GenAI tab

When any gen_ai.* attribute is present on a span, the span flyout shows a dedicated GenAI tab next to Metadata.

The Details section shows model metadata from the span attributes: operation type, request model, provider, input and output token counts, response model, and response ID. The Conversation section shows the full exchange, populated from gen_ai.system_instructions (system prompt), gen_ai.input.messages (user messages), and gen_ai.output.messages (model response), each with a copy button so you can pull the exact prompt or response out of the trace without scraping text from a formatted table.

All raw span attributes remain accessible on the Metadata tab.

The GenAI tab is also available in the span flyout in Discover, so you can inspect LLM prompts and responses directly alongside your log and trace data without switching to the APM view.

What instrumentation does LLM tracing require?

No Kibana-side configuration is needed. The GenAI tab appears automatically when any gen_ai.* attribute is present on a span. Full Conversation support requires the OTel GenAI v1.37.0 span-attribute model: gen_ai.input.messages, gen_ai.output.messages, and gen_ai.system_instructions.

Frameworks that emit the older span-events model (gen_ai.user.message, gen_ai.assistant.message, gen_ai.choice) will show the Details metadata section but will not populate the Conversation section. For a current list of compatible instrumentations, see the OTel GenAI semantic conventions.

To verify, open the span in Discover, check that gen_ai.input.messages and gen_ai.output.messages are present, and confirm the Conversation section renders.

If your application already sends APM data to Elastic from a GenAI workload, open any GenAI span in the trace view and check for the GenAI tab.

LLM token usage in the trace waterfall

Token count badges now appear on each GenAI span row in the waterfall, so you can scan the full trace without drilling in. In agentic traces with ten or twenty LLM calls, this lets you identify which span is driving token consumption before opening any span.

Each row shows input and output token count badges sourced from gen_ai.usage.input_tokens and gen_ai.usage.output_tokens. The row label is the span name, which instrumentation frameworks typically set to something like chat gpt-4o-mini.

Waterfall-level token counts are most useful in agentic traces where a chain of LLM calls uses different models or the same model with varying context sizes.

Which LLM providers does Elastic APM support for GenAI tracing?

Elastic APM's LLM tracing works with any OTel-instrumented provider: the GenAI tab and waterfall token counts use the same OTel attribute schema regardless of which provider your application uses. Provider is read from gen_ai.provider.name, falling back to gen_ai.system for older instrumentation.

The OTel GenAI semantic conventions that enable this provider detection are currently in a Development lifecycle. Check the release notes before upgrading instrumentation.

How to enable LLM tracing in Elastic APM

Availability: Both features are available as a Technical Preview on Elastic Serverless and will be available as a Technical Preview in Elastic Stack 9.6.

To try these features:

1. Instrument your GenAI application with an OTel SDK that follows the OTel GenAI semantic conventions (v1.37.0 or later for full Conversation support).
2. Send traces to Elastic Observability using OTLP, the Elastic APM agent, or an EDOT SDK.
3. Open the APM section in Kibana, navigate to a service that makes LLM calls, and open the trace waterfall for any transaction.

The GenAI tab appears on any span with at least one gen_ai.* attribute set; token count badges appear when gen_ai.usage.input_tokens or gen_ai.usage.output_tokens are present.

If you don't have a GenAI application to test with, the otel-genai-chat-app repository is a minimal OpenAI chat app pre-instrumented with EDOT. Set OPENAI_API_KEY and follow the EDOT commands in the README to send traces to Elastic and see both features in action.

What's next for LLM observability in Elastic APM

We're exploring cost estimation per span (estimated spend based on model pricing and token counts, surfaced in the waterfall) and tool call rendering (structured display of tool/function call inputs and outputs for agentic spans).

If you are building GenAI applications and want early access or to share feedback, reach out through the Elastic community forums or open an issue in the kibana repository.