LOOKUP JOIN

Stack GA 9.1.0 Serverless

LOOKUP JOIN enables you to add data from another index, AKA a 'lookup' index, to your ES|QL query results, simplifying data enrichment and analysis workflows.

Refer to the high-level landing page for an overview of the LOOKUP JOIN command, including use cases, prerequisites, and current limitations.

Syntax

FROM <source_index>
| LOOKUP JOIN <lookup_index> ON <field_name>
		

FROM <source_index>
| LOOKUP JOIN <lookup_index> ON <field_name1>, <field_name2>, <field_name3>
		

FROM <source_index>
| LOOKUP JOIN <lookup_index> ON <left_field1> >= <lookup_field1> AND <left_field2> == <lookup_field2>
		

Parameters

<lookup_index>

The name of the lookup index. This must be a specific index name - wildcards, aliases, and remote cluster references are not supported. Indices used for lookups must be configured with the lookup index mode.

<field_name> or <field_name1>, <field_name2>, <field_name3> or <left_field1> >= <lookup_field1> AND <left_field2> == <lookup_field2>

The join condition. Can be one of the following:

• A single field name
• A comma-separated list of field names Stack Planned
• An expression with one or more join conditions linked by AND. Each condition compares a field from the left index with a field from the lookup index using binary operators (==, >=, <=, >, <, !=). Each field name in the join condition must exist in only one of the indexes. Use RENAME to resolve naming conflicts. Stack Planned Serverless Preview

If using join on a single field or a field list, the fields used must exist in both your current query results and in the lookup index. If the fields contains multi-valued entries, those entries will not match anything (the added fields will contain null for those rows).

Description

The LOOKUP JOIN command adds new columns to your ES|QL query results table by finding documents in a lookup index that share the same join field value as your result rows.

For each row in your results table that matches a document in the lookup index based on the join fields, all fields from the matching document are added as new columns to that row.

If multiple documents in the lookup index match a single row in your results, the output will contain one row for each matching combination.

Supported types

Examples

IP Threat correlation: This query would allow you to see if any source IPs match known malicious addresses.

FROM firewall_logs
| LOOKUP JOIN threat_list ON source.IP
		

To filter only for those rows that have a matching threat_list entry, use WHERE ... IS NOT NULL with a field from the lookup index:

FROM firewall_logs
| LOOKUP JOIN threat_list ON source.IP
| WHERE threat_level IS NOT NULL
		

Host metadata correlation: This query pulls in environment or ownership details for each host to correlate with your metrics data.

FROM system_metrics
| LOOKUP JOIN host_inventory ON host.name
| LOOKUP JOIN ownerships ON host.name
		

Service ownership mapping: This query would show logs with the owning team or escalation information for faster triage and incident response.

FROM app_logs
| LOOKUP JOIN service_owners ON service_id
		

LOOKUP JOIN is generally faster when there are fewer rows to join with. ES|QL will try and perform any WHERE clause before the LOOKUP JOIN where possible.

The following two examples will have the same results. One has the WHERE clause before and the other after the LOOKUP JOIN. It does not matter how you write your query, our optimizer will move the filter before the lookup when possible.

FROM employees
| EVAL language_code = languages
| WHERE emp_no >= 10091 AND emp_no < 10094
| LOOKUP JOIN languages_lookup ON language_code
		

FROM employees
| EVAL language_code = languages
| LOOKUP JOIN languages_lookup ON language_code
| WHERE emp_no >= 10091 AND emp_no < 10094