Loading

Connect your self-managed cluster to AutoOps

ECK ECE Self-Managed

To use AutoOps with your ECE, ECK, or self-managed cluster, you first need to create an Elastic Cloud account or log in to your existing account. An installation wizard will then guide you through the steps of installing Elastic Agent to send metrics from your cluster to AutoOps in Elastic Cloud.

Complete the steps in the following subsections to connect your cluster to AutoOps. The connection process takes about 10 minutes.

Note

If you have an Elasticsearch cluster set up for local development or testing, you can connect it to AutoOps using Docker. Refer to Connect your local development cluster to AutoOps.

Ensure your system meets the following requirements before proceeding:

Note

Users can only belong to one Elastic Cloud organization at a time. If you want to create or join another organization, you must leave the previous one or use a different email address.

The following steps describe how to connect your ECE, ECK, or self-managed cluster to AutoOps.

If you already have an Elastic Cloud account:

  1. Log in to Elastic Cloud.
  2. On your home page, in the Connected clusters section, select Connect self-managed cluster.
  3. On the Connect your self-managed cluster page, in the AutoOps section, select Connect.
  4. Go through the installation wizard as detailed in the following sections.

If you don’t have an existing Elastic Cloud account:

  1. Sign up for an account.
  2. Follow the prompts on your screen to create an organization.
  3. Go through the installation wizard as detailed in the following sections.

This is the first step of the installation wizard. Your cluster ships metrics to AutoOps with the help of Elastic Agent.

Select one of the following methods to install Elastic Agent:

  • Kubernetes
  • Docker
  • Linux
Important

Using AutoOps for your ECE, ECK, and self-managed clusters requires a new, dedicated Elastic Agent. You must install an agent even if you already have an existing one for other purposes.

To learn more about how AutoOps securely gathers data from your cluster, refer to our FAQ.

Depending on your selected installation method, you may have to provide the following information to create the installation command:

  • Elasticsearch endpoint URL: Enter the URL for the Elasticsearch cluster you want to monitor by connecting to AutoOps.
  • Preferred authentication method: Choose one of the following:

With this authentication method, you need to create an API key to grant access to your cluster. Complete the following steps:

  1. From your Elastic Cloud home page, select a deployment.
  2. Go to Stack management > API keys and select Create API key.
  3. In the flyout, enter a name for your key and select User API key.
  4. Enable Control security privileges and enter the following script:
{
 "autoops": {
   "cluster": [
     "monitor",
     "read_ilm",
     "read_slm"
   ],
   "indices": [
     {
       "names": [
         "*"
       ],
       "privileges": [
         "monitor",
         "view_index_metadata"
       ],
       "allow_restricted_indices": true
     }
   ],
   "applications": [],
   "run_as": [],
   "metadata": {},
   "transient_metadata": {
     "enabled": true
   }
 }
}
		
  1. Select Create API key.
  2. When prompted to copy the key, select Beats from the dropdown.
  3. Copy the key and save it for later. You will need it when you install the agent.

With this authentication method, you need the username and password of a user with the necessary privileges to grant access to your cluster. There are two ways to set up a user with the these privileges:

  • (Recommended) From your Elastic Cloud home page, select a deployment and go to Developer tools. In Console, run the following command:
POST /_security/role/autoops
{
  "cluster": [
    "monitor",
    "read_ilm",
    "read_slm"
  ],
  "indices": [
    {
      "names": [
        "*"
      ],
      "privileges": [
        "monitor",
        "view_index_metadata"
      ],
      "allow_restricted_indices": true
    }
  ],
  "applications": [],
  "run_as": [],
  "metadata": {
    "description": "Allows Elastic agent to pull cluster metrics for AutoOps."
  },
  "transient_metadata": {
    "enabled": true
  }
}
		
  • Alternatively, manually assign the following privileges in your account:

    Setting Privileges
    Cluster privileges monitor, read_ilm, and read_slm
    Index privileges Indices: *
    monitor, view_index_metadata
Note

If you manually assign privileges, you won't be able to allow Elastic Agent to access restricted indices.

  • System architecture: Select the system architecture of the machine running the agent.

  • Metrics storage location: Select where to store your metrics data from the list of available AWS regions.

    Region Name
    us-east-2 Ohio
    eu-west-1 Ireland
    ap-northeast-1 Tokyo

    More regions are coming soon.

The wizard will generate an installation command based on your configuration. Depending on your installation method, the following command formats are available:

  • Kubernetes
    • YAML
  • Docker
    • Docker
    • Docker compose
  • Linux
Tip

To ensure optimum resource usage, we recommend installing the agent on a different machine from the one where your cluster is running.

Complete the following steps to run the command:

  1. Copy the command.

  2. Paste it into a text editor and update the placeholder values in the following environment variables:

    Environment variable Description
    AUTOOPS_OTEL_URL The Elastic Cloud URL to which Elastic Agent ships data. The URL is generated based on the CSP and region you pick.
    This URL shouldn't be edited.
    AUTOOPS_ES_URL The URL Elastic Agent uses to communicate with Elasticsearch.
    AUTOOPS_ES_API_KEY The API key for API key authentication to access the cluster. It combines the ${id}:${api_key} values.
    This variable shouldn't be used with AUTOOPS_ES_USERNAME and AUTOOPS_ES_PASSWORD.
    AUTOOPS_ES_USERNAME The username for basic authentication to access the cluster.
    This variable should be used with AUTOOPS_ES_PASSWORD.
    AUTOOPS_ES_PASSWORD The password for basic authentication to access the cluster.
    This variable should be used with AUTOOPS_ES_USERNAME.
    ELASTIC_CLOUD_CONNECTED_MODE_API_KEY The Elastic Cloud API Key used to register the cluster.
    This key shouldn't be edited.
    AUTOOPS_TEMP_RESOURCE_ID The temporary ID for the current installation wizard.
  3. Run the command from the machine where you want to install the agent.

  4. Return to the wizard and select I have run the command.

It might take a few minutes for your cluster details to be validated and the first metrics to be shipped to AutoOps.

If the connection is unsuccessful, an error message will appear with a possible reason for the failure and recommended next steps. For a list of these errors, refer to Potential errors.

Sometimes, an exact reason for the failure cannot be determined. In this case, explore additional resources or contact us.

To uninstall the agent, refer to Uninstall Elastic Agent.

If the connection is successful, AutoOps will start analyzing your metrics and reporting on any issues found. Depending on the size of your cluster, this process can take up to 30 minutes.

After the account is ready, the Open AutoOps button will appear in the wizard. Select it to launch AutoOps.

Learn more about AutoOps.

After you've completed the setup, you can access AutoOps for your cluster at any time.

  1. Log in to Elastic Cloud.
  2. In the Connected clusters section, locate the cluster you want to work on.
  3. In the Services column, select AutoOps.

To connect more clusters, we recommend repeating the steps to connect to AutoOps.

You can use the same installation command to connect multiple clusters, but each cluster needs a separate, dedicated Elastic Agent.

Complete the following steps to disconnect your cluster from your Cloud organization. You need the Organization owner role to perform this action.

  1. Based on your installation method, complete the steps to stop Elastic Agent from shipping metrics to Elastic Cloud.
  2. Log in to Elastic Cloud.
  3. On the Connected clusters page or the Connected clusters section of the home page, locate the cluster you want to disconnect.
  4. From that cluster’s actions menu, select Disconnect cluster.
  5. Enter the cluster’s name in the field that appears and then select Disconnect cluster.
Warning

If you disconnect a cluster, its connection to your Elastic Cloud account will end and all metrics and AutoOps data will be permanently deleted.