CPU-based indexing can also degrade search performance when indexing and search happen simultaneously, especially in online systems, like ecommerce platforms and ad-serving pipelines. This happens because index builds consume significant CPU resources, leaving fewer cycles available for queries and driving up search latency and latency variance. Even after indexing completes, latency can remain high due to index fragmentation. Customers often run a forced merge to consolidate segments, but it can be slow on CPUs, further delaying recovery to the expected search latency by hours to days.

Elasticsearch has introduced GPU-accelerated vector indexing powered by NVIDIA cuVS. In this blog, we show how Elasticsearch achieves up to 7x faster vector indexing throughput by offloading hierarchical navigable small world (HNSW) index construction to GPUs, indexing 138 million vectors in under 10 minutes using eight NVIDIA RTX PRO 6000 GPUs.

Indexing over 100M Vectors in under 10 minutes

Figure 1 shows vector indexing throughput on 138 million 1,024-dimensional vectors from the MS MARCO dataset, representing roughly 4 TB of multimodal documents (Figure 1). The benchmark was run with Rally, Elasticsearch’s macro-benchmarking framework. The test ran on a server with 8 NVIDIA RTX Pro 6000 GPUs with a 2 socket AMD EPYC 9555. GPUs were turned on for the GPU test and turned off for the CPU test. The result was that Elasticsearch achieved 7x higher indexing throughput, lowering time to index 138M vectors from 1 hour on CPUs to under 10 minutes on GPUs.

Does GPU vector indexing change recall?

Index acceleration isn’t helpful if indexes built on GPU have lower search quality compared to CPUs. The throughput-versus-recall curves for GPU-built and CPU-built indexes overlap, showing that GPU indexing delivers the same recall profile as Elasticsearch’s CPU-based indexing path, with minimal accuracy or search-time performance penalty (Figure 2).

Search latency under concurrent indexing load

Offloading indexing to the GPU improved search latency on the CPU by 6x when running indexing and search simultaneously (Figure 3) because indexing on GPUs frees CPU resources for faster search latency. This allows a single Elasticsearch cluster to support real-time ingestion and low-latency retrieval at the same time, without forcing a tradeoff between freshness and search performance.

Force merge time for HNSW indexes on GPU versus CPU

Additionally, GPU-accelerated indexing in Elasticsearch reduced forced merge time from roughly four hours to five minutes, helping restore low-latency search in real-time (Figure 4). Note: Force merge was used to merge each shard into four segments.

Here are the full results for GPU versus CPU vector indexing in Elasticsearch:

Metric

CPU

GPU

Change

Index build time, 138 million vectors

~1 hour

Under 10 min

7x throughput

p90 search latency under indexing load

Baseline

6x lower

6x

Force merge, four segments per shard

~4 hours

~5 min

~48x

Recall at target

95%

95%

Unchanged

What GPU vector indexing means for Elasticsearch users

Elasticsearch’s integration of NVIDIA cuVS brings GPU-accelerated vector indexing to production AI search with over 84% reduction in search latency to deliver faster ingest with a 640% increase in indexing throughput with a lower CPU overhead and limited infrastructure tradeoffs. This unlocks a new class of applications, such as faster RAG pipelines, semantic search, multimodal retrieval, and fraud detection at scale.

Get started with GPU vector indexing in Elasticsearch

Install Elasticsearch with GPU-accelerated vector indexing by following these instructions. Replicate the above benchmarks by running rally msmarco track with parameters in the GitHub issue. Learn more about GPU-accelerated vector indexing, search, and preprocessing by visiting NVIDIA cuVS. 

Acknowledgments

The authors would like to thank Chris Hegarty, Gilad Gal, and Mayya Sharipova from Elastic, as well as Nathan Stephens from NVIDIA, for their contributions to this article.

Recap

First, let’s bring you up to speed. Elasticsearch has established itself as a powerful vector database, offering a rich set of features and strong performance for large-scale similarity search. With capabilities such as scalar quantization, Better Binary Quantization (BBQ), SIMD vector operations, and more disk-efficient algorithms like DiskBBQ, it already provides efficient and flexible options for managing vector workloads.

By integrating NVIDIA cuVS as a callable module for vector search tasks, we aim to deliver significant gains in vector indexing performance and efficiency to better support large-scale vector workloads.

The challenge

One of the toughest challenges in building a high-performance vector database is constructing the vector index - the HNSW graph. Index building quickly becomes dominated by millions or even billions of arithmetic operations as every vector is compared against many others. In addition, index lifecycle operations, such as compaction and merges, can further increase the overall compute overhead of indexing. As data volumes and associated vector embeddings grow exponentially, accelerated computing GPUs, built for massive parallelism and high-throughput math, are ideally positioned to handle these workloads.

Enter the Elasticsearch-GPU Plugin

NVIDIA cuVS is an open-source CUDA-X library for GPU-accelerated vector search and data clustering that enables fast index building and embedding retrieval for AI and recommendation workloads.

Elasticsearch uses cuVS through cuvs-java, an open-source library developed by the community and maintained by NVIDIA. The cuvs-java library is lightweight and builds on the cuVS C API using Panama Foreign Function to expose cuVS features in an idiomatic Java way, while remaining modern and performant.

The cuvs-java library is integrated into a new Elasticsearch plugin; therefore, vector indexing on the GPU can occur on the same Elasticsearch node and process, without the need to provision any external code or hardware. During index building, if the cuVS library is installed and a GPU is present and configured, Elasticsearch will use the GPU to accelerate the vector indexing process. The vectors are given to the GPU, which constructs a CAGRA graph. This graph is then converted to the HNSW format, making it immediately available for vector search on the CPU. The final format of the built graph is the same as what would be built on the CPU; this allows Elasticsearch to leverage GPUs for high-throughput vector indexing when the underlying hardware supports it, while freeing CPU power for other tasks (concurrent search, data processing, etc.).

Index build acceleration

As part of integrating GPU acceleration into Elasticsearch, several enhancements were made to cuvs-java, focusing on efficient data input/output and function invocation. A key enhancement is the use of cuVSMatrix to transparently model vectors, whether they reside on the Java heap, off-heap, or in GPU memory. This enables data to move efficiently between memory and the GPU, avoiding unnecessary copies of potentially billions of vectors.

Thanks to this underlying zero-copy abstraction, both transferring to GPU memory and retrieving the graph can occur directly. During indexing, vectors are first buffered in memory on the Java heap, then sent to the GPU to construct the CAGRA graph. The graph is subsequently retrieved from the GPU, converted into HNSW format, and persisted to disk.

At merge time, the vectors are already stored on disk, bypassing the Java heap entirely. Index files are memory-mapped, and data is transferred directly into GPU memory. The design also easily accommodates different bit-widths, such as float32 or int8, and naturally extends to other quantization schemes.

Drumroll…so, how does it perform?

Before we get into the numbers, a bit of context is helpful. Segment merging in Elasticsearch typically runs automatically in the background during indexing, which makes it difficult to benchmark in isolation. To obtain reproducible results, we used force-merge to explicitly trigger segment merging in a controlled experiment. Since force-merge performs the same underlying merge operations as background merging, its performance serves as a useful indicator of expected improvements, even though the exact gains may differ in real-world indexing workloads.

Now, let’s see the numbers.

Our initial benchmark results are very promising. We ran the benchmark on an AWS g6.4xlarge instance with locally attached NVMe storage. A single node of Elasticsearch was configured to use the default, optimal number of indexing threads (8 - one for each physical core), and to disable merge throttling (which is less applicable with fast NVMe disks).

For the dataset, we used 2.6 million vectors with 1,536 dimensions from the OpenAI Rally vector track, encoded as base64 strings, and indexed as float32 hnsw. In all scenarios, the constructed graphs achieve recall levels of up to 95%. Here’s what we found:

• Indexing Throughput: By moving graph construction to the GPU during in-memory buffer flushes, we increase throughput by ~12x.

• Force-merge: After indexing completes, the GPU continues to accelerate segment merging, speeding up the force-merge phase by ~7x.

• CPU usage: Offloading graph construction to the GPU significantly reduces both average and peak CPU utilization. The graphs below illustrate CPU usage during indexing and merging, highlighting how much lower it is when these operations run on the GPU. Lower CPU utilization during GPU indexing frees up CPU cycles that can be redirected to improve search performance.

• Recall: Accuracy remains effectively the same between CPU and GPU runs, with the GPU-built graph reaching marginally higher recall.

Comparing along another dimension: Price

The earlier comparison intentionally used identical hardware, with the only difference being whether the GPU was used during indexing. That setup is useful for isolating raw compute effects, but we can also look at the comparison from a cost perspective.

At roughly the same hourly price as the GPU-accelerated configuration, one can provision a CPU-only setup with approximately twice the comparable CPU and memory resources: 32 vCPUs (AMD EPYC) and 64 GB of RAM, allowing to double the number of indexing threads to 16.

To keep the comparison fair and consistent, we ran this CPU-only experiment on an AWS g6.8xlarge instance, with the GPU explicitly disabled. This allowed us to hold all other hardware characteristics constant while evaluating the cost–performance trade-off of GPU acceleration versus CPU-only indexing.

The more powerful CPU instance does show improved performance compared to the benchmarks in the above section, as you would expect. However, when we compare this more powerful CPU instance against the original GPU-accelerated results, the GPU still delivers substantial performance gains: ~5x improvement in indexing throughput, and ~6x in force merge, all while building graphs that achieve recall levels of up to 95%.

Conclusion

In end-to-end scenarios, GPU acceleration with NVIDIA cuVS delivers nearly a 12x improvement in indexing throughput and a 7x decrease in force-merge latency, with significantly lower CPU utilization. This shows that vector indexing and merge workloads benefit significantly from GPU acceleration. On a cost-adjusted comparison, GPU acceleration continues to yield substantial performance gains, with approximately 5x higher indexing throughput and 6x faster force-merge operations.

GPU-accelerated vector indexing is currently planned for Tech Preview in Elasticsearch 9.3, which is scheduled to be released early in 2026.

Stay tuned for more.