Alienvault OTX Integration


Version	1.36.0 (View all)
Managed integration release status	GA
Subscription level What's this?	Basic
Developed by What's this?	Elastic
Ingestion method(s)	API
Minimum Kibana version(s)	9.3.5 8.19.16

Important - Managed integrations on Elastic Cloud Hosted

You can use this integration as an Elastic Managed integration on Elastic Cloud Hosted deployments running this version or later.

This integration is for Alienvault OTX. It retrieves indicators for all pulses subscribed to a specific user account on OTX

Agentless Enabled Integration

Agentless integrations allow you to collect data without having to manage Elastic Agent in your cloud. They make manual agent deployment unnecessary, so you can focus on your data instead of the agent that collects it. For more information, refer to Agentless integrations and the Agentless integrations FAQ. Agentless deployments are only supported in Elastic Serverless and Elastic Cloud environments. This functionality is in beta and is subject to change. Beta features are not subject to the support SLA of official GA features.

Configuration

To use this package, it is required to have an account on Alienvault OTX. Once an account has been created, and at least 1 pulse has been subscribed to, the API key can be retrieved from your user profile dashboard. In the top right corner there should be an OTX KEY.

Logs

Threat

Retrieves all the related indicators over time, related to your pulse subscriptions on OTX.

Field	Description	Type
@timestamp	Event timestamp.	date
cloud.image.id	Image ID for the cloud instance.	keyword
data_stream.dataset	Data stream dataset name.	constant_keyword
data_stream.namespace	Data stream namespace.	constant_keyword
data_stream.type	Data stream type.	constant_keyword
event.dataset	Event dataset	constant_keyword
event.module	Event module	constant_keyword
host.containerized	If the host is a container.	boolean
host.os.build	OS build information.	keyword
host.os.codename	OS codename, if any.	keyword
input.type	Type of Filebeat input.	keyword
log.flags	Flags for the log file.	keyword
log.offset	Offset of the entry in the log file.	long
otx.content	Extra text or descriptive content related to the indicator.	keyword
otx.description	A description of the indicator.	keyword
otx.id	The ID of the indicator.	keyword
otx.indicator	The value of the indicator, for example if the type is domain, this would be the value.	keyword
otx.title	Title describing the indicator.	keyword
otx.type	The indicator type, can for example be "domain, email, FileHash-SHA256".	keyword
threat.feed.dashboard_id	Dashboard ID used for Kibana CTI UI	constant_keyword
threat.feed.name	Display friendly feed name	constant_keyword
threat.indicator.file.hash.pehash	The file's pehash, if available.	keyword
threat.indicator.first_seen	The date and time when intelligence source first reported sighting this indicator.	date
threat.indicator.last_seen	The date and time when intelligence source last reported sighting this indicator.	date
threat.indicator.modified_at	The date and time when intelligence source last modified information for this indicator.	date

		{
    "@timestamp": "2025-12-23T02:57:10.418Z",
    "agent": {
        "ephemeral_id": "1ed024a0-03a1-4af1-81cc-61327fde650f",
        "id": "95bc2845-dad2-4e11-9388-12706c5c9db1",
        "name": "elastic-agent-88018",
        "type": "filebeat",
        "version": "8.19.4"
    },
    "data_stream": {
        "dataset": "ti_otx.threat",
        "namespace": "88824",
        "type": "logs"
    },
    "ecs": {
        "version": "8.11.0"
    },
    "elastic_agent": {
        "id": "95bc2845-dad2-4e11-9388-12706c5c9db1",
        "snapshot": false,
        "version": "8.19.4"
    },
    "event": {
        "agent_id_status": "verified",
        "category": [
            "threat"
        ],
        "created": "2025-12-23T02:57:10.418Z",
        "dataset": "ti_otx.threat",
        "ingested": "2025-12-23T02:57:13Z",
        "kind": "enrichment",
        "original": "{\"count\":2,\"next\":\"http://bf11511981f4:8080/api/v1/indicators/export?types=domain%2CIPv4%2Chostname%2Curl%2CFileHash-SHA256\\u0026modified_since=2020-11-29T01%3A10%3A00+00%3A00\\u0026page=2\",\"previous\":null,\"results\":{\"content\":\"\",\"description\":null,\"id\":1251,\"indicator\":\"info.3000uc.com\",\"title\":null,\"type\":\"hostname\"}}",
        "type": [
            "indicator"
        ]
    },
    "input": {
        "type": "httpjson"
    },
    "otx": {},
    "tags": [
        "preserve_original_event",
        "forwarded",
        "otx-threat"
    ],
    "threat": {
        "indicator": {
            "type": "domain-name",
            "url": {
                "domain": "info.3000uc.com"
            }
        }
    }
}
		
	

Pulses Subscribed (Recommended)

Retrieves all indicators from subscribed pulses on OTX from API /api/v1/pulses/subscribed using Filebeat's CEL input. The following subscriptions are included by this API:

All pulses by users you are subscribed to
All pulses you are directly subscribed to
All pulses you have created yourself
All pulses from groups you are a member of

Indicators of Comprosie (IoC) Expiration

Pulses Subscribed datastream also supports IoC expiration by using latest transform. Below are the steps on how it is handled:

All the indicators are retrieved into source indices named logs-ti_otx.pulses_subscribed-* using CEL input and processed via ingest pipelines. These indicators have a property named expiration which is either a null value or a timestamp such as "2023-09-07T00:00:00". When the value is null or if the timestamp value is less than current timestamp now(), the indicator is not expired, and hence is still active.
A latest transform is continuosly run on source indices. The purpose of this transform is to:
- Move only the active indicators from source indices into destination indices named logs-ti_otx_latest.pulses_subscribed-<NUMBER> where NUMBER indicates index version.
- Delete expired indicators based on the expiration timestamp value.
All the active indicators can be retrieved using destination index alias logs-ti_otx_latest.pulses_subscribed which points to the latest destination index version.

Note: Do not use the source indices logs-ti_otx.pulses_subscribed-*, because when the indicators expire, the source indices will contain duplicates. Always use the destination index alias: logs-ti_otx_latest.pulses_subscribed to query all active indicators.

Field	Description	Type
@timestamp	Event timestamp.	date
data_stream.dataset	Data stream dataset name.	constant_keyword
data_stream.namespace	Data stream namespace.	constant_keyword
data_stream.type	Data stream type.	constant_keyword
event.dataset	Event dataset	constant_keyword
event.module	Event module	constant_keyword
input.type	Type of Filebeat input.	keyword
labels.is_ioc_transform_source	Indicates whether an IOC is in the raw source data stream, or the in latest destination index.	constant_keyword
log.flags	Flags for the log file.	keyword
log.offset	Offset of the entry in the log file.	long
otx.content		keyword
otx.count		integer
otx.created		date
otx.description		keyword
otx.expiration		date
otx.id	The ID of the indicator.	keyword
otx.indicator		keyword
otx.is_active		integer
otx.prefetch_pulse_ids		boolean
otx.pulse.adversary		keyword
otx.pulse.attack_ids		keyword
otx.pulse.author_name		keyword
otx.pulse.created		date
otx.pulse.description		keyword
otx.pulse.extract_source		keyword
otx.pulse.id		keyword
otx.pulse.industries		keyword
otx.pulse.malware_families		keyword
otx.pulse.modified		date
otx.pulse.more_indicators		boolean
otx.pulse.name		keyword
otx.pulse.public		integer
otx.pulse.references		keyword
otx.pulse.revision		integer
otx.pulse.targeted_countries		keyword
otx.pulse.tlp		keyword
otx.role		keyword
otx.t		double
otx.t2		double
otx.t3		double
otx.title		keyword
threat.feed.dashboard_id	Dashboard ID used for Kibana CTI UI	constant_keyword
threat.feed.name	Display friendly feed name	constant_keyword
threat.indicator.file.hash.pehash	The file's pehash, if available.	keyword
threat.indicator.first_seen	The date and time when intelligence source first reported sighting this indicator.	date
threat.indicator.last_seen	The date and time when intelligence source last reported sighting this indicator.	date
threat.indicator.modified_at	The date and time when intelligence source last modified information for this indicator.	date

		{
    "@timestamp": "2023-08-08T05:05:15.000Z",
    "agent": {
        "ephemeral_id": "c12b4f72-265e-41f0-96e0-103c81de7908",
        "id": "8299ae35-ee0e-4107-9acb-1b6acfdda1fb",
        "name": "docker-fleet-agent",
        "type": "filebeat",
        "version": "8.13.0"
    },
    "data_stream": {
        "dataset": "ti_otx.pulses_subscribed",
        "namespace": "32586",
        "type": "logs"
    },
    "ecs": {
        "version": "8.11.0"
    },
    "elastic_agent": {
        "id": "8299ae35-ee0e-4107-9acb-1b6acfdda1fb",
        "snapshot": false,
        "version": "8.13.0"
    },
    "event": {
        "agent_id_status": "verified",
        "category": [
            "threat"
        ],
        "dataset": "ti_otx.pulses_subscribed",
        "ingested": "2024-08-02T06:03:28Z",
        "kind": "enrichment",
        "original": "{\"content\":\"\",\"count\":2,\"created\":\"2023-08-08T05:05:15\",\"description\":\"\",\"expiration\":null,\"id\":3454375108,\"indicator\":\"pinup-casino-tr.site\",\"is_active\":1,\"prefetch_pulse_ids\":false,\"pulse_raw\":\"{\\\"adversary\\\":\\\"\\\",\\\"attack_ids\\\":[\\\"T1531\\\",\\\"T1059\\\",\\\"T1566\\\"],\\\"author_name\\\":\\\"SampleUser\\\",\\\"created\\\":\\\"2023-08-22T09:43:18.855000\\\",\\\"description\\\":\\\"\\\",\\\"extract_source\\\":[],\\\"id\\\":\\\"64e38336d783f91d6948a7b1\\\",\\\"industries\\\":[],\\\"malware_families\\\":[\\\"WHIRLPOOL\\\"],\\\"modified\\\":\\\"2023-08-22T09:43:18.855000\\\",\\\"more_indicators\\\":false,\\\"name\\\":\\\"Sample Pulse\\\",\\\"public\\\":1,\\\"references\\\":[\\\"https://www.cisa.gov/news-events/analysis-reports/ar23-230a\\\"],\\\"revision\\\":1,\\\"tags\\\":[\\\"cisa\\\",\\\"backdoor\\\",\\\"whirlpool\\\",\\\"malware\\\"],\\\"targeted_countries\\\":[],\\\"tlp\\\":\\\"white\\\"}\",\"role\":null,\"t\":0,\"t2\":0.0050694942474365234,\"t3\":2.7960586547851562,\"title\":\"\",\"type\":\"domain\"}",
        "type": [
            "indicator"
        ]
    },
    "input": {
        "type": "cel"
    },
    "otx": {
        "count": 2,
        "created": "2023-08-08T05:05:15.000Z",
        "expiration": "2023-08-13T05:05:15.000Z",
        "id": "3454375108",
        "is_active": 1,
        "prefetch_pulse_ids": false,
        "pulse": {
            "attack_ids": [
                "T1531",
                "T1059",
                "T1566"
            ],
            "author_name": "SampleUser",
            "created": "2023-08-22T09:43:18.855Z",
            "description": "",
            "extract_source": [],
            "id": "64e38336d783f91d6948a7b1",
            "industries": [],
            "malware_families": [
                "WHIRLPOOL"
            ],
            "modified": "2023-08-22T09:43:18.855Z",
            "more_indicators": false,
            "name": "Sample Pulse",
            "public": 1,
            "references": [
                "https://www.cisa.gov/news-events/analysis-reports/ar23-230a"
            ],
            "revision": 1,
            "targeted_countries": [],
            "tlp": "white"
        },
        "t": 0,
        "t2": 0.0050694942474365234,
        "t3": 2.7960586547851562
    },
    "tags": [
        "preserve_original_event",
        "forwarded",
        "otx-pulses_subscribed",
        "cisa",
        "backdoor",
        "whirlpool",
        "malware"
    ],
    "threat": {
        "indicator": {
            "provider": "OTX",
            "type": "domain-name",
            "url": {
                "domain": "pinup-casino-tr.site"
            }
        }
    }
}
		
	

Data retention

Threat indicators are re-collected across polling intervals, and the latest transform keeps the active, deduplicated view in its destination index. The source data streams therefore hold repeated copies of the same indicators. The threat data stream does not ship a retention.

The package bounds the growth of these source data streams with a retention that depends on the deployment type:

Data stream	Self-managed and Elastic Cloud Hosted (ILM policy)	Serverless (data stream lifecycle)
`logs-ti_otx.pulses_subscribed-*`	`logs-ti_otx.pulses_subscribed-default_policy`: roll over after 2d, delete 3d after rollover	delete 5d after rollover

On self-managed and Elastic Cloud Hosted deployments the ILM policy applies. The data stream lifecycle shipped with the package is not used there. ILM counts the delete age from the rollover of a backing index, so a document can remain for up to the rollover age plus the delete age. On Serverless, ILM is not available and the data stream lifecycle applies instead. It also works per backing index: Elasticsearch rolls the write index over automatically on age, size, or document count, and deletes a backing index once the retention has passed since it rolled over. A document therefore stays for the retention plus up to one rollover interval. The rollover age is derived from the retention and is an implementation detail that Elasticsearch may change. Where the package installs a transform, the transform's destination indices are not affected by either.

To keep data for a different period:

Self-managed and Elastic Cloud Hosted: edit the ILM policy in Kibana under Stack Management → Index Lifecycle Policies, or with PUT _ilm/policy/<policy name>. A package upgrade reinstalls the package's ILM policies, so check your change after upgrading.
Serverless: set the retention on the data stream, for example PUT _data_stream/logs-ti_otx.pulses_subscribed-default/_lifecycle with the body {"data_retention": "90d"}. Replace default with your namespace.

Changelog

Version	Details	Minimum Kibana version
1.36.0	Enhancement (View pull request) Document the retention applied to source data streams on stateful and Serverless deployments, and how to change it.	9.3.5 8.19.16
1.35.0	Enhancement (View pull request) Remove redundant ECS field definitions and the catch-all dynamic template from the transforms, relying on the `ecs@mappings` component template.	9.3.5 8.19.16
1.34.0	Enhancement (View pull request) Enable request trace log removal.	9.3.5 8.19.16
1.33.0	Enhancement (View pull request) Add tags to ingest pipeline processors and add `preserve_original_event` to pipeline-level `on_failure` handlers.	9.3.5 8.19.16
1.32.2	Enhancement (View pull request) Set agentless deployment mode `release` field to `ga`.	9.3.5 8.19.16
1.32.1	Bug fix (View pull request) Fix transform destination to use keyword for data_stream.namespace, allowing multi-namespace deployments.	9.3.5 8.19.16
1.32.0	Enhancement (View pull request) Enable Agentless deployment.	9.3.5 8.19.16
1.31.0	Enhancement (View pull request) Use num_failure_retries instead of unattended mode for transform failure recovery.	9.1.4 9.0.7 8.19.4
1.30.1	Bug fix (View pull request) Remove duplicate security-solution-default tag references	9.1.4 9.0.7 8.19.4
1.30.0	Enhancement (View pull request) Prevent updating fleet health status to degraded when pagination completes.	9.1.4 9.0.7 8.19.4
1.29.0	Enhancement (View pull request) Allow transforms to run in unattended mode.	9.0.0 8.13.0
1.28.0	Enhancement (View pull request) Update Kibana constraint to support 9.0.0.	9.0.0 8.13.0
1.27.1	Bug fix (View pull request) Updated SSL description to be uniform and to include links to documentation.	8.13.0
1.27.0	Enhancement (View pull request) Do not remove `event.original` in main ingest pipeline.	8.13.0
1.26.0	Enhancement (View pull request) Add "preserve_original_event" tag to documents with `event.kind` set to "pipeline_error".	8.13.0
1.25.3	Bug fix (View pull request) Fix labels.is_ioc_transform_source values	8.13.0
1.25.2	Bug fix (View pull request) Add missing fields in transform	8.13.0
1.25.1	Bug fix (View pull request) Fix ECS date mapping on threat fields.	8.13.0
1.25.0	Enhancement (View pull request) Update the kibana constraint to ^8.13.0. Modified the field definitions to remove ECS fields made redundant by the ecs@mappings component template.	8.13.0
1.24.1	Bug fix (View pull request) Fix type-mapping inconsistency for `otx.id` field.	8.12.0
1.24.0	Enhancement (View pull request) Set sensitive values as secret.	8.12.0
1.23.2	Enhancement (View pull request) Changed owners	8.10.3
1.23.1	Bug fix (View pull request) Fix IOC expiration duration character casting.	8.10.3
1.23.0	Enhancement (View pull request) Append hash and IP values to related.* fields	8.10.3
1.22.0	Enhancement (View pull request) Limit request tracer log count to five.	8.10.3
1.21.0	Enhancement (View pull request) ECS version updated to 8.11.0.	8.10.3
1.20.0	Enhancement (View pull request) Improve 'event.original' check to avoid errors if set.	8.10.3
1.19.0	Enhancement (View pull request) Add Pulses Subscribed datastream to support IoC expiration Enhancement (View pull request) Add DLM policy. Add owner.type to package manifest.	8.10.3
1.18.0	Enhancement (View pull request) ECS version updated to 8.10.0.	8.7.1
1.17.0	Enhancement (View pull request) The format_version in the package manifest changed from 2.11.0 to 3.0.0. Removed dotted YAML keys from package manifest. Added 'owner.type: elastic' to package manifest.	8.7.1
1.16.0	Enhancement (View pull request) Add tags.yml file so that integration's dashboards and saved searches are tagged with "Security Solution" and displayed in the Security Solution UI.	8.7.1
1.15.0	Enhancement (View pull request) Update package-spec to 2.10.0.	8.7.1
1.14.0	Enhancement (View pull request) Update package to ECS 8.9.0.	8.7.1
1.13.0	Enhancement (View pull request) Document duration units.	8.7.1
1.12.0	Enhancement (View pull request) Document valid duration units.	8.7.1
1.11.0	Enhancement (View pull request) Ensure event.kind is correctly set for pipeline errors.	8.7.1
1.10.0	Enhancement (View pull request) Update package to ECS 8.8.0.	8.7.1
1.9.0	Enhancement (View pull request) Add a new flag to enable request tracing	8.7.1
1.8.0	Enhancement (View pull request) Update package to ECS 8.7.0.	8.0.0
1.7.1	Enhancement (View pull request) Honor `preserve_original_event` setting.	8.0.0
1.7.0	Enhancement (View pull request) Update package to ECS 8.6.0.	8.0.0
1.6.1	Enhancement (View pull request) Add support to drop empty documents	8.0.0
1.6.0	Enhancement (View pull request) Update package to ECS 8.5.0.	8.0.0
1.5.0	Enhancement (View pull request) Update package to ECS 8.4.0	8.0.0
1.4.2	Bug fix (View pull request) Fix proxy URL documentation rendering.	8.0.0
1.4.1	Enhancement (View pull request) Update categories to include `threat_intel`.	8.0.0
1.4.0	Enhancement (View pull request) Update package to ECS 8.3.0.	8.0.0
1.3.2	Enhancement (View pull request) Update readme file to add documentation link	8.0.0
1.3.1	Enhancement (View pull request) Update package descriptions	8.0.0
1.3.0	Enhancement (View pull request) Update to ECS 8.2	8.0.0
1.2.2	Enhancement (View pull request) Add field mapping for event.created	8.0.0
1.2.1	Enhancement (View pull request) Add documentation for multi-fields	8.0.0
1.2.0	Enhancement (View pull request) Update to ECS 8.0	8.0.0
1.1.0	Enhancement (View pull request) Adding threat.feed fields and dashboards	8.0.0
1.0.3	Bug fix (View pull request) Change test public IPs to the supported subset	8.0.0
1.0.2	Enhancement (View pull request) Bump minimum version	8.0.0
1.0.1	Enhancement (View pull request) Update title and description.	—
1.0.0	Enhancement (View pull request) Initial release	—