Fleet and Elastic Agent 8.18.3

Details

There is a known issue where Elastic Agent remains in an Upgrade scheduled state when a scheduled Elastic Agent upgrade is cancelled. Attempting to restart the upgrade on the UI returns an error: The selected agent is not upgradeable: agent is already being upgraded..

Impact

Until this issue is fixed in a later patch version, you can call the Upgrade an agent endpoint of the Kibana Fleet API with the force parameter set to true to force-upgrade the Elastic Agent:

curl --request POST \
  --url https://<KIBANA_HOST>/api/fleet/agents/<AGENT_ID>/upgrade \
  --user "<SUPERUSER_NAME>:<SUPERUSER_PASSWORD>" \
  --header 'Content-Type: application/json' \
  --header 'kbn-xsrf: true' \
  --data '{"version": "<VERSION>","force": true}'

To force-upgrade multiple Elastic Agents, call the Bulk upgrade agents endpoint of the Kibana Fleet API with the force parameter set to true:

curl --request POST \
  --url https://<KIBANA_HOST>/api/fleet/agents/bulk_upgrade \
  --user "<SUPERUSER_NAME>:<SUPERUSER_PASSWORD>" \
  --header 'Content-Type: application/json' \
  --header 'kbn-xsrf: true' \
  --data '{"version": "<VERSION>","force": true,"agents":["<AGENT_IDS>"]}'

Details

On May 2, 2025 a known issue was discovered that the .fleet-agents index template was missing a mapping for the local_metadata.complete attribute. This may cause agent checkins to be rejected and the agents to appear as offline.

In this Fleet’s logs this will appear as:

elastic fail 400: document_parsing_exception: [1:209] object mapping for [local_metadata] tried to parse field [local_metadata] as object, but found a concrete value
Eat bulk checkin error; Keep on truckin'

And in the Elastic Agent logs it will appear as:

"log.level":"error","@timestamp":"2025-04-22:12:35:25.295Z","message":"Eat bulk checkin error; Keep on truckin'","component":{"binary":"fleet-server","dataset":"elastic_agent.fleet_server","id":"fleet-server-es-containerhost","type":"fleet-server"},"log":{"source":"fleet-server-es-containerhost"},"service.type":"fleet-server","error.message":"elastic fail 400: document_parsing_exception: [1:209] object mapping for [local_metadata] tried to parse field [local_metadata] as object, but found a concrete value","ecs.version":"1.6.0","service.name":"fleet-server","ecs.version":"1.6.0"

This attribute was added to the template in versions: 8.17.11 8.18.3, and 8.19.3.

Further investigation revealed that the .fleet-agents index template was not correctly applied due to an unchanged _meta.managed_index_mappings_version number. This change also affects other attributes as well, such as upgrade_attempts, namespaces, unprivileged, and unhealthy_reason. If there is an error related to any of these attributes, there will be a similar error message in the logs.

Impact

Updating to a version with a fixed _meta.managed_index_mappings_version will correctly apply the new index template. The fixed versions are 8.18.8, 8.19.4, 9.0.8, 9.1.4.