Elasticsearch Updates & Release Notes

Features and enhancements

FIPS

• Upgrade Bouncy Castle FIPS to 1.0.2.7 and 2.0.2 #156296

Infra/Core

• Upgrade ASM to 9.10.1 #156213
• Upgrade the bundled JDK to 26.0.2 #156203

Mapping

• Upgrade to Lucene 10.5.1 #156643

Security

• Upgrade Jackson to 2.21.5 for ES|QL and Arrow #156368

Fixes

Aggregations

• Fix exponential histogram min/max not being fully respected in rank estimation #156268 (issue: #156057)

Downsampling

• Fix duplicate field in the cumulative histogram reset document #156287 (issue: #156276)

ES|QL

• Account for t-digest percentiles memory in the request circuit breaker #155586
• Fix TS STATS aggregate aliases that collide with grouping keys #155312 (issue: #153507)
• Fix TS queries that name a time bucket after the timestamp field #155623
• Fix incorrect results from Parquet predicate pushdown on multi-valued fields #156604
• Integrate circuit breaker into BestBucketsDeferringCollector #155600 (issue: #148516)
• Replace an IN subquery evaluated as a false filter with an empty local relation #155648 (issue: #155563)

Inference

• Fix Anthropic translation to the unified schema #155314
• Fix Google Vertex AI chat completion tool call argument serialization for non-string values #156665 (issue: #156644)
• Fix BroadcastMessageAction request serialization #156603
• Translate unified tool-calling message blocks to the Anthropic format #154607

Infra/Core

• Fix IllegalStateException when a truncated string cuts off in the middle of a JSON object #156382 (issue: #156289)

Machine Learning

• Fail closed on an incomplete TorchScript pre-load state hook scan #3149
• Fail gracefully when restoring a categorizer with an out-of-range token ID #3143

Search

• Add a limit on aggregation nesting depth #155743
• Count failed _msearch sub-search responses against the circuit breaker #156683
• Ensure batched queries always return a response #155908
• Exclude non-metadata _type from default search hit fields #155706 (issue: #110438)
• Fail the request instead of the node on deeply nested aggregations #155745
• Fix NullPointerException in LongComparator caused by null search_after values #132434 (issue: #132370)
• Fix nested and mode-based sorts on integer fields returning incorrect values on indices created before 8.19 or in 9.0.x #155580 (issue: #155243)
• Keep scroll contexts on transient search queue rejection #155697

Snapshot/Restore

• Ensure Error propagates out of S3BlobContainer #156765

Transform

• Preserve headers when copying a transform configuration #156191

Vector Search

• Fix DOT_PRODUCT and COSINE score clamping in the BBQ vector scorer #156015 (issues: #156010, #156011, #156009, #156012)
• Fix search failures on indices without vector fields when many field patterns are requested #156466
• Fix the auto_calibrate index option being rejected by a leftover experimental flag #156623
• Score nested kNN inner_hits like the query phase #156332 (issue: #138496)

Original source

- Elastic Stack: Planned

Features and enhancements

FIPS:

• Update Bouncy Castle FIPS TLS libraries #155937

Infra/Core:

• Upgrade lz4-java to 1.11.1 #155598

Machine Learning:

• Add EuroBERT and Jina v5 ops to the graph validation allowlist #3015
• Downgrade log severity for a batch of recoverable errors #2889
• Harden pytorch_inference with TorchScript model graph validation #3008 (issue: #2890)
• Improve adherence to memory limits for the bucket gatherer #2848
• Improve error handling for quantiles state documents #2894
• Improve handling of invalid JSON state documents #2895
• Improve messaging for OOM process termination #2841
• Report the actual memory usage of the autodetect process #2846
• Update the PyTorch library to version 2.7.1 #2863

Security:

• Upgrade Apache HttpComponents 5.x to address CVEs #153057
• Upgrade Apache HttpCore 5 to 5.4.3 to fix header DoS CVEs #153467
• Upgrade Bouncy Castle to 1.85 #155819

Vector Search:

• Cache little-endian float layout in the DiskBBQ scalar corrections loop #154373

Fixes

Analysis:

• Trip the request breaker on term and phrase suggesters based on backing array estimates #154837

Codec:

• Fix false positives in boolean queries with must_not clauses on non-indexed fields #155936 (issue: #155653)

ES|QL:

• Allow garbage collection of closed search contexts in ES|QL #155418
• Apply PropagateEmptyRelation to AbstractSubqueryJoin #154768 (issue: #154755)
• Fix BUCKET with very large bucket counts #153392 (issue: #153389)
• Fix FROM_BASE64 generating invalid non-UTF-8 strings. Previously it could create binary values that ES|QL does not support in other operations; only valid UTF-8 is supported, and invalid cases now return null with a warning. #154955
• Fix MV_SORT incorrectly allowing geospatial types, which are not sortable #154417
• Fix exponential histogram merging with zero bucket overlap #155459 (issue: #153665)

ILM:

• Support dynamically enabling history indexes in ILMHistoryTemplateRegistry #155602 (issue: #141807)

Infra/CLI:

• Upgrade JANSI to 2.4.3 to fix terminal detection on zlib-ng based systems (EL10) #155853 (issue: #154510)

Machine Learning:

• Fix flaky CIoManagerTest/testFileIoGood test #3017
• Fix flaky concurrent LFU cache count invariant under lock timeouts #3090
• Mark the ML controller non-dumpable before accepting commands #3081
• Reject TorchScript custom state hooks before load and forbid inductor::_reinterpret_tensor #3078
• Reject non-native ABIs in the ML seccomp filter (socketcall/getuid collision) #3080

PromQL:

• Support date_nanos timestamp indices #153996 (issue: #146923)

Search:

• Clamp merge scheduler thread count when it exceeds max merge count #155693 (issues: #96594, #155678)
• Limit dot-separated identifiers per version part #154572
• Parse scroll ID from a bounded byte array #154542
• Use readArraySize in readImmutableMap and readImmutableOpenMap #154525

Vector Search:

• Fix EUCLIDEAN score clamping order in the BBQ vector scorer #155630 (issue: #155568)

Original source

- Elastic Stack: Generally available since 9.4

Features and enhancements

FIPS

• Update Bouncy Castle FIPS TLS libraries #155937

Infra/Core

• Upgrade lz4-java to 1.11.1 #155598

Infra/Plugins

• Add policy_base_id field to fleet-agents and fleet-policies index templates #154521

Machine Learning

• Add EuroBERT and Jina v5 ops to the graph validation allowlist #3015
• Downgrade log severity for a batch of recoverable errors #2889
• Harden pytorch_inference with TorchScript model graph validation #3008 (issue: #2890)
• Improve adherence to memory limits for the bucket gatherer #2848
• Improve error handling for quantiles state documents #2894
• Improve handling of invalid JSON state documents #2895
• Improve messaging for OOM process termination #2841
• Report the actual memory usage of the autodetect process #2846
• Restrict file system access for PyTorch models #2851
• Update the PyTorch library to version 2.7.1 #2863

Security

• Add security stats to the monitoring mapping #155552
• Upgrade Apache HttpComponents 5.x to address CVEs #153057
• Upgrade Apache HttpCore 5 to 5.4.3 to fix header DoS CVEs #153467
• Upgrade Bouncy Castle to 1.85 #155819
• Upgrade Jackson to 2.18.9 #154569
• Upgrade Log4j to 2.26.1 #154115

Vector Search

• Cache little-endian float layout in the DiskBBQ scalar corrections loop #154373

Fixes

Analysis

• Add a check against self-referencing filters #154793
• Trip the request breaker on term and phrase suggesters based on backing array estimates #154837

ES|QL

• Allow garbage collection of closed search contexts in ES|QL #155418
• Avoid an integer overflow loop in the REPEAT function #154295
• Fix FROM_BASE64 generating invalid non-UTF-8 strings. Previously it could create binary values that ES|QL does not support in other operations; only valid UTF-8 is supported, and invalid cases now return null with a warning. #154955
• Fix MV_SORT incorrectly allowing geospatial types, which are not sortable #154417
• Fix a cause of lost ES|QL warnings by preserving thread context #154016
• Fix a concurrency issue in InferenceOperator#pollNextRequest for failed responses #154972 (issue: #154866)

ILM

• Support dynamically enabling history indexes in ILMHistoryTemplateRegistry #155602 (issue: #141807)

Infra/CLI

• Upgrade JANSI to 2.4.3 to fix terminal detection on zlib-ng based systems (EL10) #155853 (issue: #154510)

Machine Learning

• Fix double decrement of the pending inference request count #154577 (issue: #154483)
• Fix flaky CIoManagerTest/testFileIoGood test #3017
• Fix flaky concurrent LFU cache count invariant under lock timeouts #3090
• Mark the ML controller non-dumpable before accepting commands #3081
• Reject TorchScript custom state hooks before load and forbid inductor::_reinterpret_tensor #3078
• Reject non-native ABIs in the ML seccomp filter (socketcall/getuid collision) #3080

Mapping

• Fix wildcard queries on keyword fields whose normalizer rewrites wildcard characters #153582 (issue: #150699)

Ranking

• Add a hardcoded limit on nested retrievers #154814
• Fix an error caused by empty reranker results #155099

Search

• Apply max_regex_length limit to intervals regexp and wildcard rules #155231
• Bound numeric string length before parsing #154689
• Clamp merge scheduler thread count when it exceeds max merge count #155693 (issues: #96594, #155678)
• Ensure cancelled _field_caps tasks are removed from the tasks API #153831 (issue: #137475)
• Limit dot-separated identifiers per version part #154572
• Parse scroll ID from a bounded byte array #154542
• Use readArraySize in readImmutableMap and readImmutableOpenMap #154525

Transform

• Retry transient failures during restarts #153298

Vector Search

• Fix EUCLIDEAN score clamping order in the BBQ vector scorer #155630 (issue: #155568)

Original source

Highlights

Verify overwrite protection in repository analysis
The safety of the snapshot algorithm relies on the repository preventing certain objects from being overwritten. From version 9.5.0 onwards, Elasticsearch will verify that the repository implements overwrite protection correctly during repository analysis.

Batched execution in the query phase
Searches that target multiple shards on the same data node now batch shards into a single round-trip per data node, with partial reductions performed on the data nodes. This reduces transport layer overhead and spreads the load of reductions across multiple nodes. The enhancement is controlled by a setting search.batched_query_phase, which is enabled by default in 9.5.0.

Improved reindex resilience and new reindex management APIs
We're making multiple changes to the resilience and ease-of-use of the reindex API. Firstly, asynchronous reindex operations will now survive the graceful shutdown of the node where they are running. This makes long-running operations much more reliable, especially in a managed cluster. Secondly, reindex operations will now normally use the point-in-time API instead of the scroll API for paginated search. This is the preferred approach for deep pagination, and allows reindex to benefit from recent improvements to PIT resilience (where available). Finally, we are adding dedicated new APIs to monitor and manage reindex operations, providing a better user-experience than the older tasks APIs. Read more about these changes in the blog at https://www.elastic.co/search-labs/blog/elasticsearch-reindex-node-relocation-pit-serverless.

Data stream lifecycle can move data to the frozen tier
Data stream lifecycle (DLM) now supports the frozen tier. Set a frozen_after value in a data stream's lifecycle, and DLM moves aging backing indices to the frozen tier as partially-mounted searchable snapshots once they pass that age, reducing storage cost while keeping the data fully searchable. frozen_after sits alongside data_retention and downsampling in the lifecycle configuration, with no ILM policy required. DLM writes frozen data to a new cluster-level default snapshot repository. This capability requires an Enterprise license and is not available in Elastic Cloud Serverless.

Support for flattened fields in ES|QL
ES|QL now supports the flattened field type. Fields mapped as flattened were previously unsupported and could not be referenced in queries. They can now be loaded and, together with the new FIELD_EXTRACT ES|QL function, queried by sub-field.FIELD_EXTRACT(<flattened field>, "<sub-field>") extracts the value of a single sub-field from a flattened object and returns it as a keyword. The second argument is the literal name of the sub-field exactly as it is stored, for example FIELD_EXTRACT(attributes, "host.name"). The dot is part of the key, so the same dotted form addresses both originally-flat and originally-nested sub-fields. When a sub-field holds multiple values, the result is a multi-valued keyword.Both the flattened type support and the FIELD_EXTRACT function are in Technical Preview.

date_range field type available as tech preview
The date_range field type is now available in ES|QL as a technical preview. This unlocks the range-specific functions TO_DATE_RANGE, TO_RANGE, RANGE_MIN, RANGE_MAX, RANGE_CONTAINS, RANGE_INTERSECTS, and RANGE_WITHIN, as well as support for date_range fields in generic functions such as COUNT, PRESENT, ABSENT, and the MV_* family.

Add native support for metric temporality
TSDB now supports metric temporality natively: Counters and histograms can be ingested through the OTLP endpoint with both cumulative and delta temporality. The temporality will be stored as an additional metric dimension and ES|QL TS queries will interpret the data automatically, taking the temporality into account. There is no new query syntax for this functionality. Existing queries will continue to work as expected. The temporality is also respected and preserved during downsampling. Metrics ingested using other means can use the new index.time_series.temporality_field index setting to define the dimension field storing the temporality.

PromQL: Add basic support for native (exponential) histograms
You can now query exponential_histogram fields with PromQL syntax for native histograms. The supported functions are increase(), sum(), histogram_quantile(), histogram_avg(), histogram_count() and histogram_sum().

ES|QL Data Federation
ES|QL Data Federation (experimental) is now available. This feature enables you to create and manage external data sources (Amazon S3 initially) and query datasets using ES|QL.This is an experimental feature and is not enabled by default. To try it out, set esql.federation.enabled: true in your Elasticsearch configuration and xpack.dataFederation.enabled: true in your Kibana configuration.

Columnar index mode (Tech Preview)
Two new index modes, columnar and logsdb_columnar, are now available as a Tech Preview. With columnar mode enabled, Elasticsearch becomes a fully columnar store for search and analytics, offering a significantly smaller storage footprint and the building blocks for faster analytical queries in future releases.Fields are stored once, as doc values only — no inverted index or BKD tree is created by default, which eliminates redundant index structures and significantly reduces the storage footprint. Doc values skippers (compact skip lists with min/max metadata) are enabled by default for number based fields, allowing large blocks of documents to be skipped without scanning individual values. Doc value skippers will be enabled for all applicable fields at a later point. Text fields remain to be indexed by default, but are no longer mapped by default.Mappings are always flat: object and passthrough fields are auto-flattened to leaf fields at mapping time. The original document source is not stored; the columnar source is generated on the fly from doc values at query time. A pre-built columnar representation can be stored on disk using the new columnar_stored source mode (optional for Enterprise licenses).New multi-value semantics: the original ordering of multiple values per field per document (e.g., in arrays) is preserved by default. Optionally, fields in mappings can be configured to only allow one value per document.Single-value enforcement (doc_values.multi_value: false): reject documents that provide more than one value for a field, enabling tighter columnar encoding and stronger schema guarantees.Required-value enforcement (doc_values.nullability: false): reject documents that omit a field or supply an explicit null.There are two columnar index modes:

• logsdb_columnar adds logging-oriented defaults on top of the columnar foundation: a default @timestamp mapping, and index sorting on host.name and @timestamp when those fields are present.
• columnar provides the same column-oriented storage without use-case-specific defaults, suitable for general-purpose indices and data streams.
  Improved columnar storage: An optimized doc values format is used by default, further reducing storage footprint, especially when combined with index sorting.

Create backing indices for backfilling past timestamps in TSDB (opt-in)
Bulk requests that create documents in a time series data stream no longer fail outright when a document's @timestamp falls outside the boundaries of all existing backing indices. If the timestamp is still within the data stream's eligible write window and the new data_stream.past_tsdb_index_creation_enabled cluster setting is enabled (defaults to false), Elasticsearch now creates the necessary backing indices needed to cover it before indexing, instead of rejecting the write. Timestamps outside the eligible window, or in the future, are still rejected.

ES95 is the default TSDB doc values codec
ES95 is now the default doc values codec for new time series indices. On host metrics data it reduces total doc values storage by about 30% versus ES819, driven by @timestamp (up to 90% smaller) and floating-point gauges and counters via adaptive floating-point encoding, at no indexing or query cost. Existing indices keep their codec; opt out with index.time_series.es95_codec.enabled: false.

Release support for match with non-mapped expressions
The match function and : operator can now work with expressions that no longer need to represent a mapped field in an Elasticsearch index.The match function no longer needs to be pushed down as a Lucene query to the shard. When matching on an expression that does not represent an Elasticsearch mapped field, but has the text data type, we evaluate the match function on-the-fly by analyzing the value of the expression for each row and checking whether any of the analyzed tokens match the analyzed tokens of the given query string.

Features and enhancements

Allocation:

• Early publish DesiredBalance when newly created replicas are assigned #149645
• Expose DesiredBalanceStats as metrics #146928

Analysis:

• Allow appending synonyms to existing rules via append=true (default false) #146870 (issue: #146864)
• Allow multiple synonym sets per filter using the synonyms API #147157 (issue: #144941)

Audit:

• Allow audit logging to be turned on/off without server restart #147333

Authentication:

• Change federated feature flag to cluster config #153098
• Introduce feature flag for keyless auth #152684
• Provide default jwt_audience #152362

Authorization:

• [Evaluations] Register .evaluation-* as hidden indices with viewer/editor access #148880

CCS:

• Support -remote:index-style exclusion syntax #148497

CRUD:

• Expand index pressure on update expansion #147151
• Minimize the amount of memory used by CommitReferencesInfo #147976

Codec:

• Update to lucene 10.5 #151959
• Use IOContext to detect random access for bloom filters #147224
• Write skipindex to a separate file in TSDB Codec #147238

Data streams:

• Adding dynamic template for *.lifecycle.last_activity fields #149505
• Allow deleting a backing index via modify data streams API #151137
• Data stream lifecycle (DLM) can move data to the frozen tier #150457
• Throttle concurrent downsampling operations in data stream lifecycle. A new dynamic cluster setting, data_streams.lifecycle.downsampling.max_indices_in_progress (default 10), caps the number of backing indices per data stream that data stream lifecycle will submit to downsampling in a single run. Indices that are deferred are recorded in the error store with a warning and will be retried in a subsequent run. #150802 (issue: #150624)
• Updating ecs@mappings to include new flattened gen_ai fields #148674
• [otel-data] Add HTTP body size, severity aliases #152622

Distributed:

• Improve allocation explain API for canRemain #152121 (issue: #131603)
• Make INDICES_RECOVERY_MAX_CONCURRENT_OUTGOING_RECOVERIES setting dynamic #151138
• Support outbound peer recovery throttling #149319

Downsampling:

• Assign downsampling to least loaded candidate node #148220

ES|QL:

• Add Arrow output support for date_nanos and date_range data types #150479 (issue: #146350)
• Add ENRICH to bytes counting #150677
• Add IP_LOCATION command #149421 (issues: #150231, #132489)
• Add range_intersects and range_contains functions #148713
• Add a BY subclause to CHANGE_POINT command #148473
• Add analyzer option in ES|QL TOP_SNIPPETS function #147369
• Add order option to TOP_SNIPPETS #145946
• Add release hook to Vector and Block #150017
• Avoid tracking group ids for intermediate input #141466
• Bulk-grow TopN UTF-8 string encoding #149022
• Cache FilterOperator/EvalOperator toString #148354
• Chunk time-series aggregation output #151670 (issue: #147286)
• Constant-RHS fast path for MOD and DIV evaluators #148678
• Count read bytes for Lucene operators #150036
• De-snapshot column metadata for BUCKET #152998 (issue: #148508)
• ESQL planning and streaming changes for bulk keyword lookup #148937
• ESQL: Push down unrelated filters past MV_EXPAND #144979 (issue: #144636)
• ES|QL - Add timeout to inference operations COMPLETION, RERANK and TEXT_EMBEDDING #146666 (issue: #146528)
• ES|QL JSON_EXTRACT function is GA #152957
• ES|QL query approximation support for FORK #148412
• ES|QL: unify SET and request-body parameters into one settings framework #149206 (issue: #143239)
• Enable ES|QL EMBEDDING function for calculating dense vector embeddings on multimodal input #146801
• Extend FIRST and EARLIEST aggregations to support additional types including spatial types, version and unsigned_long #149307 (issue: #141596)
• Extend FIRST and EARLIEST to support dense_vector, exponential_histogram and tdigest #149716
• Extend LAST and LATEST to support the same field types as FIRST and EARLIEST #152154 (issue: #149478)
• Fast paths for prefix/suffix/contains LIKE patterns (SIMD substring search for contains) #148680
• Gate field_extract pushdown, use RECHECK #154135
• Highlighting in TOP_SNIPPETS #144742
• Implement DEDUP command #147574 (issue: #133322)
• Implement bucket metadata #146077 (issue: #138146)
• Improve LongLongSwissHash for high cardinality #147078
• Make FUSE command GA #151885 (issue: #123389)
• Make date_range tech preview #151949
• Make the worker queue dynamic #151103
• Make views REST API available in serverless #151431
• Optimize field_extract on flattened fields #148879
• Optimize field_extract on flattened fields #148891
• Ordinal fast path for BytesRefTopNBlockHash #148333
• Parallel TopN Operator #150148
• Prune constant sort keys from TopN #147769 (issue: #143518)
• Prune non-top-N groups during aggregation #148256
• Prune redundant stats groupings #150030
• REPLACE fast-path review fixes #149167
• Release to_text conversion function #152523
• Release flattened type and field_extract #151841
• Release support for match with non-mapped expressions #152941
• Rewrite SUM(X+c) to SUM(X) + c*COUNT(X) #145510 (issue: #140470)
• Route (LONG, BYTES_REF) STATS through an adaptive composite BlockHash #152010
• SIMD contains for literal LIKE patterns #149026
• Share numeric TopN thresholds #149926
• Speed up REPLACE on constant regex #149033
• Support ST_UNION, ST_INTERSECTION, ST_DIFFERENCE, ST_SYMDIFFERENCE #149092
• Support full-text search functions for unmapped fields with LOAD and return null for partially unmapped fields that cannot be implicitly converted #152200 (issue: #144121)
• Support keyword/text in TOP(field,...,outputField) #151767 (issue: #151751)
• Unmapped fields/full text null warning #152973 (issue: #151403)
• Unmapped_fields="load" for views, subqueries and fork #151750 (issue: #142033)
• Unmatched DROP wildcard no-op if unmapped #153090 (issue: #143226)
• Use atomic reference counting for blocks and vectors #153410 (issues: #153394, #153411, #154241, #153752)
• FieldExtract double-sided and single-sided range query pushdown #149710
• RoundTo Block Loader Optimization #144234

Engine:

• Avoid storing in BlobTranslogFile totalOps for active shards when it is equal to zero #148992
• Log BCC upload progress and BCC queueing time #149484
• Merging is now more aggressive by default, especially for segments under 16MB. #133946 (issues: #120624, #129764, #130328)
• Reduce the amount of memory used by TranslogReplicator#BlobTranslogFile #148881
• Wire internal refresh listeners on SearchEngine #149889

Geo:

• Support BufferParameters in ST_BUFFER #147615

ILM:

• ILM & DLM create snapshots with partial: true to prevent blocking cluster recovery actions #153774 (issue: #153643)

Indices APIs:

• Rollover ignoring active shards can ignore reroute #149859 (issue: #149730)

Inference:

• Add chat_completion task type to the Anthropic inference service #148539 (issue: #147188)
• Add support for audio, video and pdf inputs for embedding task #147329 (issue: #146432)
• Add timeout support to the update inference endpoint API #152179 (issue: #140501)
• Change EIS rerank request format to use objects for query and documents #150172
• Determine inference timeout based on task type #146089
• Enable EMBEDDING task for OpenAiService #145833
• Enable multiple items per content object for Elastic service #148340
• Support jina-embeddings-v5-omni input format for Jina integration #148551
• Support multimodal (image) rerank for the Elastic Inference Service #152012
• Support the Google Vertex AI global endpoint for inference endpoints #151054 (issue: #150990)
• Update the rerank endpoint for the Elastic inference service #152869
• [Inference API] Add product use case and origin to telemetry #148140
• [Inference API] Add reasoning fields to Elastic Service TaskSettings #152230
• [Inference API] Add stack version and production environment to APM attributes #147633
• [Inference API] Adding OpenAI OAuth2 implementation #150132
• [Inference API] Adding deployment type to Inference API telemetry #149098
• [Inference API] Apply Retry-After header if present in EIS responses #147985
• [Inference API] Region policy for inference requests #153093

Infra/Core:

• Ability to configure a number of replicas for system indices #149892 (issue: #99720)
• Wire OTel SDK self-monitoring metrics into the span exporter #150026

Infra/Logging:

• Introduce params support for query logging (ESQL & SQL) #150266

Infra/Metrics:

• APM OTel SDK: flush metrics and traces under a single shared timeout #149220 (issue: #149072)
• Add missing OTel resource information #150354
• Add stack trace suppression for OTel traces #150626
• Buffer metrics if APM server returns an error #146401
• ES-14041: Add regression test suite for trace/span export #146095
• ES-14041: Lock APM-agent trace contract in regression test #147729
• ES-14041: Port trace/span export from APM agent to OTel SDK #146096
• Flush OTel SDK metrics at shutdown #145839
• Flush OTel traces at shutdown #145939
• Set OTel span status ERROR for HTTP 5xx responses #151593
• Setting for trace sampling and batch processor #150876

Infra/Plugins:

• Add config.name mapping to fleet-agents index #148703
• Add effective_config_hash keyword mapping to fleet-agents index #146975
• [Fleet] Add policy_base_id field to fleet-agents and fleet-policies index templates #154521

Infra/REST API:

• Support form encoded REST bodies for authenticated requests #149005

Infra/Scripting:

• Add Painless allocation counter mechanics #151085
• Add Painless allocation-limit setting scaffolding #151017
• Add Painless cancellation checks to fetch, agg-reduce, and bulk-write script contexts #149754
• Emit Painless array runtime-size allocation pre-checks #151687
• Emit Painless compile-time allocation pre-checks #151339
• Emit Painless static-type string concat allocation pre-checks #152326
• Painless: cancellation_aware augmentation infrastructure #150092
• Painless: script_aware Iterable/Collection/Map augmentations #151016
• Painless: script_aware Stream.collect(Collector) and primitive forEachRemaining #151337
• Painless: script_aware String search + regex limit-factor for replaceAll/replaceFirst #151086
• Painless: script_aware wrappers for native iteration methods #151059
• Painless: script_aware wrappers for stream terminal ops #151080
• Painless: honor search timeout in score scripts #148260
• Wire search-timeout cancellation to Painless script contexts #149346

Ingest Node:

• Add AI bots to user-agent regex #148665
• Add max_field_bytes to attachment processor and ingest.attachment.max_field_size node setting #148493
• Adding support for IPinfo Plus database #149885

Logs:

• Enable OTLP logs and traces by default #147811
• Optimize sorted numeric field data loading for dense fields #150138

Machine Learning:

• Add ECS event.ingested to ML anomaly detection result documents #148583 (issue: #144836)
• Add EuroBERT and Jina v5 ops to graph validation allowlist #3015
• Add byte_level_bpe ML tokenization with merges #147217
• Add maintenance task to close idle AD jobs #144904
• Better error handling regarding quantiles state documents #2894
• Better handling of invalid JSON state documents #2895
• Better messaging regarding OOM process termination #2841
• Create settings for platform architecture #147496
• Downgrade log severity for a batch of recoverable errors #2889
• Expose datafeed authorization.cloud_api_key.id on GET for CPS operator visibility #150473
• Harden pytorch_inference with TorchScript model graph validation #3008 (issue: #2890)
• Improve adherence to memory limits for the bucket gatherer #2848
• Lookup join and Inline stats support for query approximation #145980
• Report the actual memory usage of the autodetect process #2846
• Update the PyTorch library to version 2.7.1 #2863

Mapping:

• Add doc_values mapping attribute to RoutingFieldMapper #146576
• Cache FieldInfo instances per shard Directory #148865
• Columnar index modes available as Tech Preview #152481
• Improve DocumentParser#parseObjectDynamic() if subobjects are disabled and dynamic templates #150553
• Release semantic field #152528 (issue: #143287)
• Restrict doc values multi_value/nullability parameters to columnar mode #152111
• Store semantic_text input in doc values #152387
• Support nested fields under subobjects:false #152357
• Wire columnar index mode through synthetic source test framework #152231

PromQL:

• Add GET /_prometheus/api/v1/metadata endpoint #146044 (issue: #146045)
• Add TS_COLLAPSE command #148690
• ES|QL PromQL: implicit type coercion for counter/gauge #149985 (issue: #140035)
• Fix PromQL round(to_nearest) floating point accuracy #149972
• Implement GET /_prometheus/api/v1/status/buildinfo #150235
• Implement PromQL set operator: top-level or (UNION) #151486
• PromQL: Add support for histogram_count, histogram_sum and histogram_avg #151502
• PromQL: Add support for increase on histograms #152065
• PromQL: Add support for sum aggregation on exponential histograms #152082
• PromQL: Support histogram_quantile for exponential histograms #152171 (issue: #150074)
• Route Prometheus remote write via data_stream labels #148910
• Support form-encoded POST on PromQL HTTP query and discovery routes when security and HTTP TLS are enabled #148974
• Use MetricsInfo for Prometheus labels API #150948

Recovery:

• Remove RecoveryMonitor #144762 (issue: #93544)
• ThrottlingRecoveryService can throttle recoveries on data node #149317

Reindex:

• Improved reindex resilience and new reindex management APIs #148786
• Reindex parse exception has document index and ID #145146 (issue: #48432)
• Reject reindexes with wrong parameters #145290 (issue: #37855)
• Track remote reindex HTTP response bytes in REQUEST circuit breaker #149389

Search:

• Add PIT context relocation metrics to stateless recovery #150604
• Add cache-miss wait time to search response #151576
• Add circuit breaker to TransportMultiSearchAction #150115
• Add diagnostic fields to reader context lifecycle logs #147552 (issue: #112680)
• Add inference flag to field caps #152102 (issues: #147575, #124567)
• Add points memory estimate to tier heap estimate #150545
• Add reader-heap circuit breaker for the stateless search engine #149062
• Add stats parameter support to Count API #140143 (issue: #67528)
• Align bfloat16 vector data to minimize page crossings #147225
• Allow semantic text use with embedding models #144796
• Blob store prefetching: Allow for all nodes #150175
• Caching: Prefetch from blob storage, if local prefetching fails #147964
• Collapse SearchHit.readFrom and SearchHits.readFrom signatures #147155
• Enable the adaptive replica selection formula adjustment #152793
• Enable the batched query phase #148622
• Estimate PointRangeQuery memory in the circuit breaker #151211
• Expose _clusters metadata in open point-in-time response for CCS #149466 (issue: #84400)
• FSDirectoryFactory: fix read advice for slices inside CFS #147222
• Generalize per-fetch directory metrics to all pluggable metrics #153433
• Honor search timeout by not retrying on replicas #150075 (issue: #149508)
• Include index and shard id in shard failure exception message #147566 (issue: #113489)
• Include target indices and query in profiled search results #145230 (issue: #143783)
• Make synonym rule limit configurable at the cluster level and increase the limit from 10,000 to 100,000 #146087
• Migrate rank-eval SearchHits from unpooled to pooled #146673
• Pool ClientHit PIT hits via ref-counting #146868
• Ref count EQL _source (search hits and events) #146182
• Reject pathPrefix containing consecutive slashes #147541 (issue: #117048)
• Reserve CB memory for automaton construction #148722 (issue: #147428)
• Scrolling ref counts hits #144426
• Stop carrying shard request with each shard search result #146931 (issue: #146684)
• Stop retrying shard requests on replicas for non-retriable errors #149506 (issue: #72349)
• Synonyms search_after pagination #146965
• Track search phase request/result bytes at coordinator #151987
• Upgrade qa/vector opentelemetry dependency for security vulnerability #152225
• Use ref-counted SearchHits in data frame analytics paths #145361
• Wait for shard readiness before executing search on data node #146378 (issue: #139672)
• CompletionSuggestion: ref-count hits, released by SearchResponse #144351
• ExpandSearchPhase: fix inner hits lifecycle to use pooled hits #144342
• SearchHitRowSet uses pooled SearchHits #145235

Searchable Snapshots:

• Azure event loop default threads #151913
• Cache miss no longer blocked by fetch #146207

Security:

• Add EncryptionService for encrypt/decrypt operations using PEK #147418
• Add KibanaCasesImplicitPrivilegesProvider contributing implicit index privileges for Kibana Cases #152714
• Add x-pack-kibana plugin contributing implicit index privileges for Kibana Alerting V2 #148331
• Add automatic key rotation for primary encryption key #148203
• Add destructive reset endpoint for project encryption key #150466
• Add encryption at rest for primary encryption key #148568
• Add primary encryption key generation and distribution #145990
• Bump ES|QL/Arrow Jackson to 2.21.4 #153005
• Implicit Index Privileges SPI #147176
• Move encryption key management into a dedicated x-pack-encryption module #149668
• Reuse resolved privilege automata in implicit SPI #152938
• Support wildcard application names in implicit privilege SPI #148121
• Surface implicit privileges in get-role API #147781
• Upgrade the version of log4j to 2.26.1 #154115
• [Encryption] Add encryption password to auto-configuration #151522
• [Encryption] Graceful degradation when no password configured #151405
• [Encryption] Move PEK password wrap/unwrap to disk write/read only #151609
• [Encryption] Remove PEK Feature Flag #152290
• [kibana_system] add significant_events-* data access for kibana_system user #150465

Snapshot/Restore:

• Add data_access_tier and metadata_access_tier settings to Azure repository #149345
• Add data_storage_class and metadata_storage_class settings for GCP repository #151058
• Add azure snapshot repository access tier settings #150396
• Add distinct storage class settings for S3 #148920
• Add resumable write buffer size client setting for GCS repository #150838
• Adding a check to verify that BlobContainer::blobExists works as expected in the repository analysis tool #146595 (issue: #119303)
• Adding a check to verify that BlobContainer::listBlobsByPrefix works as expected #146440
• S3HttpFixture: track and expose storage class on blobs #149069
• Verify overwrite protection in repository analysis #143701 (issue: #134632)

Suggesters:

• Return 400 instead of 500 for suggest requests across indices with different analysis chains. #150608

TSDB:

• Add TO_COUNTER() function and ::counter cast operator #149687
• Add TO_GAUGE() function and ::gauge cast operator #149713
• Add native support for metric temporality #152091
• Add raw histogram mapping hint #148938
• Create backing indices for backfilling past timestamps in TSDB (opt-in) #152716 (issue: #149856)
• Handle OTLP explicit-bounds histograms without buckets in tdigest and exponential histogram conversion paths #149591
• Make ES95 the default TSDB doc values codec #152861
• Two-phase iterator for TSDB numeric range doc values #152515

Transform:

• Add Defer Flag #145693 (issue: #140456)
• Integrate with UIAM tokens #149637

Vector Search:

• Add AVX-512 f32 dot product and squared L2 kernels #145674
• Advise MADV_RANDOM on blob cache regions backing vector data files #150066
• BF16 bulk AVX-512 K-unroll for sequential paths #147672
• DiskBBQ - add symmetric 1-bit OSQ vector scorer #149339
• DiskBBQ - wire auto calibration at merge time #152894
• DiskBBQ Bulk collect knn docs to improve query latency #146588
• DiskBBQ: switches from striped dibit to packed #149344 (issue: #144608)
• Improve float & byte comparisons for RankVectors #147507
• Native BBQ apply corrections (all distance types) #145567
• Report vectordb_document usage in _xpack/usage #152780
• Tuned AVX-512 int4 dot product implementations (~20% gain) #148287
• Use panama bulk methods for bulk vector scoring #150906
• [DiskBBQ] Version DiskBBQ format to ES950 #153012
• [Native] AVX-512 BBQ vector operations #146069
• [Native] head+spread prefetch in amd64 bulk kernels #148945
• [Native] i7u bulk AVX-512 shared-b #147999
• [Native] i8 bulk AVX-512 shared-b #147865
• DenseVectorQuery: lazy filter evaluation and use bulk scorer #147504 (issue: #137042)

Original source

Features and enhancements

Authentication:

• Upgrade OpenTelemetry in ms-graph-authz [#151968]

ES|QL:

• Implement watchdog timeout for GROK [#152851]

Inference:

• Update commons-text library version to match commons-lang3 [#152092]

Infra/Core:

• Upgrade jackson [#152163]

Machine Learning:

• Add EuroBERT and Jina v5 ops to graph validation allowlist [#3015]
• Downgrade log severity for a batch of recoverable errors [#2889]
• Harden pytorch_inference with TorchScript model graph validation [#3008] (issue: [#2890])
• Improve adherence to memory limits for the bucket gatherer [#2848]
• Improve error handling for quantiles state documents [#2894]
• Improve handling of invalid JSON state documents [#2895]
• Improve messaging for OOM process termination [#2841]
• Report the actual memory usage of the autodetect process [#2846]
• Restrict file system access for PyTorch models [#2851]
• Update the PyTorch library to version 2.7.1 [#2863]

Fixes

Aggregations:

• Avoid OOMs by accounting for memory during cardinality agg reduction phase [#152773] (issue: [#150290])
• Fix NPE when aggregation param scripts return null [#152152] (issue: [#136639])

Authentication:

• Fix Clone API Key silently dropping fields that follow a null expiration [#152874]

Authorization:

• Fix data stream privilege checking [#148179]
• Improve simulate bulk action authz [#152148]

ES|QL:

• Avoid dense run-length allocation in HLL [#152035]
• Call Project.output() only when Project.expressionsResolved() is true [#151697] (issue: [#149681])
• Fix UnsupportedOperationException for partially unmapped keyword fields with multi-fields [#150676] (issue: [#150667])
• Fix errors and warnings with spatial functions that parse invalid BBOX [#152877] (issue: [#152876])
• Resolve prefixed searchable-snapshot backing indices to their data stream in METRICS_INFO/TS_INFO [#153128]
• Resolve NoSuchElementException in ViewUnionAll analyzer retries [#152867]
• Treat empty list query parameters as null [#152098] (issue: [#152271])

Geo:

• Fix infinite loop in GeoLineDecomposer dateline crossing checks [#152096] (issue: [#152066])

Inference:

• Translate tools and tool_choice to Anthropic format for Google Model Garden Anthropic chat completion [#152427]

Ingest Node:

• Include ingest metadata in self reference checks [#152932]

Machine Learning:

• Automatically repair ML anomaly results aliases pointing at a .reindexed-v7 index [#147688] (issue: [#147686])
• Extend AD results template for reindexed indices and generalize heal beyond job_id mapping [#153755] (issue: [#147686])
• Fix ML jobs stuck in starting state on Serverless trial projects [#150362]
• Fix flaky CIoManagerTest/testFileIoGood test [#3017]
• Retry datafeed STARTED state write on system reassignment [#151399]
• Roll reindexed ML state indices in daily maintenance [#149555]

Mapping:

• Fix flattened field array-offset corruption with preserve_leaf_arrays: exact [#153043] (issue: [#153014])

Network:

• Fix remote write with audit request bodies [#153581] (issue: [#152713])

Search:

• Add more defensive protections when parsing query_string [#152385]
• Apply search timeout to the DFS phase query rewrite [#153479]
• Apply search timeout to the query rewrite step [#153082] (issue: [#104187])
• Fork ensureShardSearchActive callbacks off the refresh thread [#152509] (issue: [#97280])
• Return a 400 (Bad Request) for invalid tokens in a script query [#152476]
• Trip request breaker on big completion suggest size/shardSize [#152552]
• Trivial hardening of data URI validation [#152664]

Snapshot/Restore:

• Fix snapshot creation on CIFS shares in case of access denied exception [#153219] (issue: [#152053])

TSDB:

• Ignore OTLP number data points without a value [#152733]

Transform:

• Tighten retry startup logic [#152803]

Original source

The 9.3.8 release contains fixes for potential security vulnerabilities. For details, go to security announcements.

Features and enhancements

Authentication

• Upgrade OpenTelemetry in ms-graph-authz [#151968]

ES|QL

• Implement watchdog timeout for GROK [#152851]

Inference

• Update the commons-text library version to match commons.lang3 [#152092]

Infra/Core

• Upgrade Jackson [#152163]

Machine Learning

• Better messaging regarding OOM process termination [#2841]
• Improve adherence to memory limits for the bucket gatherer [#2848]
• Report the actual memory usage of the autodetect process [#2846]
• Restrict file system access for pytorch models [#2851]
• Update the PyTorch library to version 2.7.1 [#2863]

Fixes

Aggregations

• Avoid out-of-memory errors by accounting for memory during the cardinality aggregation reduction phase [#152773] (issue: [#150290])
• Fix non-monotonic T-Digest centroids caused by floating-point errors [#151979]
• Fix null pointer exception when aggregation parameter scripts return null [#152152] (issue: [#136639])

Authorization

• Fix data stream privilege checking [#148179]
• Improve authorization for the simulate bulk action [#152148]

ES|QL

• Avoid dense run-length allocation in HyperLogLog [#152035]

Geo

• Fix infinite loop in GeoLineDecomposer dateline crossing checks [#152096] (issue: [#152066])

Inference

• Translate tools and tool_choice to Anthropic format for Google Model Garden Anthropic chat completion [#152427]

Ingest Node

• Include ingest metadata in self-reference checks [#152932]

Machine Learning

• Automatically repair ML anomaly results aliases pointing at a .reindexed-v7 index [#147688] (issue: [#147686])
• Extend the anomaly detection results template for reindexed indices and generalize healing beyond job_id mapping [#153755] (issue: [#147686])
• Fix ML jobs stuck in starting state on Serverless trial projects [#150362]
• Roll reindexed ML state indices in daily maintenance [#149555]

Search

• Add defensive protections when parsing query_string [#152385]
• Apply search timeout to the DFS phase query rewrite [#153479]
• Apply search timeout to the query rewrite step [#153082] (issue: [#104187])
• Fork ensureShardSearchActive callbacks off the refresh thread [#152509] (issue: [#97280])
• Return a 400 (Bad Request) for invalid tokens in a script query [#152476]
• Trip request breaker on large completion suggest size/shardSize [#152552]

Snapshot/Restore

• Fix snapshot creation on CIFS shares when an access-denied exception occurs [#153219] (issue: [#152053])

TSDB

• Ignore OTLP number data points without a value [#152733]

Transform

• Tighten retry startup logic [#152803]

Original source

Features and enhancements

Infra/Logging:

• Upgrade to log4j 2.26.0 [#132166] (issue: [#132035])

Machine Learning:

• Better messaging regarding OOM process termination [#2841]
• Improve adherence to memory limits for the bucket gatherer [#2848]
• Report the actual memory usage of the autodetect process [#2846]
• Restrict file system access for pytorch models [#2851]
• Update the PyTorch library to version 2.7.1 [#2863]

Packaging:

• Bump bundled JDK to Java 26.0.1 [#147424]

Search:

• Add logging tracking to _xpack/usage [#148087]

Security:

• Update elastic-apm-agent-java8 to 1.55.6 [#148271]

Fixes

Aggregations:

• Aggs: Account aggs uncompressed size on coordinator reduction [#148885] (issue: [#147190])
• Aggs: Fix auto_date_histogram/date_histogram inside a global agg [#147427] (issue: [#92570])
• Fix ArrayIndexOutOfBoundsException in date_histogram with hard_bounds outside data [#148765] (issue: [#148763])

CCR:

• Fix CCR follow to handle indexing_complete race [#145304]

Data streams:

• Update failure store redirect logic to exclude backpressure exceptions [#148154]

Distributed:

• Fix FsBlobContainer.moveBlobAtomic fallback on CIFS filesystems [#148777] (issue: [#148811])
• Fix race in FsBlobContainer.moveBlobAtomic by replacing move op with hard link [#147405] (issue: [#147171])

EQL:

• Fix management of PIT close failures after successful query [#146634] (issues: [#146263], [#146187])

ES|QL:

• Disallow empty lists in named params, only [#147748] (issue: [#147448])
• Do not enable doc-partitioning for count [#143544] (issue: [#134512])
• Fix ArrayIndexOutOfBoundsException in centroid grouping evaluateFinal [#148815] (issue: [#141318])
• Fix FROM *_logs index name validation [#149395] (issue: [#146073])
• Fix InvalidMappedField equals/hashCode and lazy error message [#146117] (issue: [#145907])
• Fix errors in blockloaders when filtering nulls on histograms [#148983] (issue: [#147854])
• Node-reduce driver should not release search contexts on failure [#145960] (issue: [#145509])
• Normalize _timeseries to JSON for stored source [#148716] (issue: [#148414])
• Use search threadpool for coordination in ES|QL [#149034]

Inference:

• Honor task settings for Amazon Bedrock and Google Vertex chat completion [#149268] (issue: [#148792])
• Implement RerankingInferenceService for Elastic service [#148365]
• Make similarity required for SageMaker ElasticTextEmbeddingPayload [#146885]
• [Inference API] Fix inference initialization thread exhaustion [#147063]
• [Inference API] Fixing potential NPE if InferenceContext is null [#148921]
• [Inference API] Use image/jpeg instead of image/jpg for inference validation MIME type [#148793]

Infra/Core:

• Fix potential OOME on SearchResponse#toString() for extemely large responses [#147267]
• Use toTruncatedString in chunked objects toString [#147860] (issue: [#143694])

Ingest Node:

• Fix pipeline resolution cache for bulk requests [#144648]
• Fix waiting for enrich policy execution for users without the monitor privilege [#145751]

Logs:

• Fixed a bug where match only text delegates loading to an incorrect parent keyword field [#141399]

Machine Learning:

• Fix AD job update overrides categorization_examples_limit with default [#140524]
• Fix NPE when preview datafeed checks date_nanos for an unmapped time field [#144909] (issue: [#144888])
• Fix flaky CIoManagerTest/testFileIoGood test [#3017]
• Fix validation of anomaly detection job config [#139946] (issue: [#29843])
• Handle null columns in ES|QL CHANGE_POINT [#144388] (issue: [#142858])
• Omit uncomputed model stats [#146186]
• Preserve zero-allocation trained-model deployments in cluster state during ML-node shutdown rebalances [#146910] (issue: [#146806])
• Reduce priority of clear-cache tasks [#139685]

Mapping:

• Date fields with ignore malformed do not ignore object and array values [#143533] (issue: [#109539])
• Don't serialize endpoint metadata in semantic text mappings [#145021] (issue: [#144969])
• Drop deprecation warnings when updating a mapping in the cluster state applier [#144040] (issues: [#142544], [#142964], [#142410], [#143884], [#142477])
• Fix match only text decoding surrogate pairs [#146567] (issue: [#146538])
• Re-use previously seen dynamically-created Mappers during document parsing [#143377]
• Resolve *.geo.location as geopoint in otel dynamic mappings [#141397] (issue: [#140506])

PromQL:

• PromQL: absent labels must match empty-string label matchers [#146116]

Reindex:

• Restore initial thread context during reindex etc [#146134]

SQL:

• Fix NPE in aggregations [#139829]
• Fix SQL client parsing of array header values [#143408] (issues: [#143018], [#143019])

Search:

• Add cancellation support to IndicesRequestCache [#141708]
• Adjust the formula for "adaptive replica selection" [#144562]
• Async Search: Ensure cleanup is also done against aliases [#146356] (issue: [#146184])
• Bugfix for CCS BWC in SearchShardsAction [#145820]
• Bugfix: Deleted async search won't show on any API [#140385] (issue: [#132423])
• Collapse pathological regex quantifier stacking to prevent NFA construction OOM [#145452]
• Fix terminate_after not honored for aggs when size=0 [#146199] (issue: [#126665])
• Fix bug parsing "request" parameter in clear cache API, it should clear the request cache only [#145726]
• Fix nested aggregation top_hits with query inner_hits [#137351] (issue: [#136893])
• Fix pooled SearchHits leak in MergeResult#writeTo on transport version mismatch [#146431] (issue: [#146420])
• Fix: correctly serialise sparse field pruning options in mixed cluster scenarios [#147823]

Snapshot/Restore:

• Ensure deletion of snapshot clone propagate state changes [#142192]
• Finalize failed clone entries [#143024] (issue: [#142919])
• Fix incorrect totalTime when shard snapshot never started [#142361]
• Fix read/write counts for copy in repo analysis [#140086] (issue: [#126569])
• Limit heap used tracking IndexMetadata deletions [#140394] (issue: [#140018])
• Synthesize ShardId for SnapshotInfo of failed clones [#142920]
• Terminate Azure retries when node is shutting down [#142187]
• Use IllegalArgumentException over RepositoryException for readonly-repository checks [#140200]

Store:

• Fix potential NPE in SyntheticIdField empty token stream [#140556] (issues: [#140624], [#140665], [#140492])

TSDB:

• Replace IllegalStateException with IllegalArgumentException for conflicting time series metadata [#142370]
• Retrieve routing hash from synthetic id for translog operations [#140221]

Task Management:

• Ensure cancellation reason visible when task is canceled [#144629] (issue: [#142079])

Transform:

• Exclude clusters in license check [#143146] (issues: [#114514], [#114509])

Vector Search:

• Check for support for native vector ops before calling them [#142036] (issue: [#141863])
• Fix AVX-512 fpclass mask to classify negative infinity as negative in MIP scoring [#142514] (issue: [#142289])
• Fix BBQ and INT4 bulk scorers to use sparse vector resolution [#145779] (issue: [#145797])
• Fix NPE in GPU resource pool when CuVSResources creation fails [#146632]
• Fix NPE when having double nested field with knn query [#146933] (issue: [#141830])
• Fix PosixCloseableMappedByteBuffer.slice() to preserve concrete type [#147903]
• Fix int overflow in vector similarity bounds checks for segments larger than 2GB [#143241] (issue: [#143232])
• Fix integer overflow in CuVSResourceManager.estimateNNDescentMemory [#147559]
• Fix intercepted query cases that can create orphaned unique async action consumers [#145577] (issue: [#145444])
• Only use MemorySegment scorers when slices can be obtained from the IndexInput [#145343]
• [DiskBBQ] Fix offHeap size for empty indices [#146347]
• [DiskBBQ] Fix bug in NeighborQueue#popRawAndAddRaw [#145324]
• [DiskBBQ] Wait for queue saturation in MaxScoreTopKnnCollector [#145341]

Watcher:

• Clear watch schedules when starting trigger engine [#145325] (issues: [#137562], [#131964])

Original source

Features and enhancements

Inference:

• [Inference] Add requires org.apache.commons.lang3 to module-info [#151794]

Machine Learning:

• Better messaging regarding OOM process termination [#2841]
• Improve adherence to memory limits for the bucket gatherer [#2848]
• Report the actual memory usage of the autodetect process [#2846]
• Restrict file system access for pytorch models [#2851]
• Update the PyTorch library to version 2.7.1 [#2863]

Fixes

ES|QL:

• Resolve surrogates in union type field resolution before plan serialization [#151633] (issue: [#151475])

Infra/Core:

• Pass through includeSourceInError [#142335]

Machine Learning:

• Fix bug causing incorrect error on force deleting already deleted model [#107188] (issue: [#105420])
• Revert "[ML] Add extra validation in trained_model creation" [#152000]

Original source

Features and enhancements

Authorization

• Update the built-in kibana_system role to grant manage, create_index, read, index, write, and delete privileges on the axonius.alert_finding and axonius.incident indices, replacing the previous axonius.alert_and_incident index #149797

Data streams

• [otel-data] Explicitly map http.response.status_code as long #149631

Inference

• Upgrade commons-lang3 version for the inference plugin #150242

Machine Learning

• Better messaging regarding OOM process termination #2841
• Improve adherence to memory limits for the bucket gatherer #2848
• Report the actual memory usage of the autodetect process #2846
• Restrict file system access for pytorch models #2851
• Update the PyTorch library to version 2.7.1 #2863

Network

• Upgrade netty to 4.1.135.Final #151099

Security

• Optimize literal action automaton builds #151093 (issues: #123872, #105723)

Fixes

Analysis

• Fix Nynorsk stemmer UnsupportedOperationException. The light_nynorsk and minimal_nynorsk stemmers no longer fail with an HTTP 500 error during index creation due to passing an immutable map to Lucene's analysis factory. #150345

Authentication

• Move SAML metadata resolution to background thread #144381 (issue: #138031)

ES|QL

• Catch StackOverflowError in deeply nested RLIKE patterns #150238 (issue: #149838)
• Enforce limit for max nested functions #149971
• Fix wrong warning in expressions with unrolled multivalues #145968
• [ESQL] Refactor Greatest and Least functions to use evaluator map #128429 (issue: #114036)

ILM

• Truncate error for ILM's step_info instead of string #150413

Inference

• Validate inference embedding model before checking for existing uses #150150 (issues: #147062, #150084)
• When a streaming inference request is canceled, immediately release apache client networking resources #149987
• [Inference API] Fix Streaming publisher shutdown race condition #150789 (issue: #150742)

Infra/Core

• Don't apply time zones to epoch-based timestamps #148663
• Filter _source field names by code point #151146
• Fix OS stats for cgroup paths containing colons #151095

Machine Learning

• Add extra validation in trained_model creation #150227

Query Languages

• EQL/SQL: Enforce max expression depth checks #150003

Search

• Add IT covering search during rolling restart #149667 (issue: #86927)

Security

• Make Automatons wildcard code-point aware #151143

TSDB

• Fix OTLP histogram handling for single-count histograms without bucket boundaries #151411

Transform

• Honor ClusterHealth timeout when waiting for transform internal index shards #149462 (issue: #149400)

Original source

Features and enhancements

Infra/Logging:

• Upgrade to log4j 2.26.0 [#132166] (issue: [#132035])

Machine Learning:

• Add EuroBERT and Jina v5 ops to graph validation allowlist [#3015]
• Better error handling regarding quantiles state documents [#2894]
• Better handling of invalid JSON state documents [#2895]
• Better messaging regarding OOM process termination [#2841]
• Downgrade log severity for a batch of recoverable errors [#2889]
• Harden pytorch_inference with TorchScript model graph validation [#3008] (issue: [#2890])
• Improve adherence to memory limits for the bucket gatherer [#2848]
• Report the actual memory usage of the autodetect process [#2846]
• Restrict file system access for pytorch models [#2851]
• Update the PyTorch library to version 2.7.1 [#2863]

Search:

• Add logging tracking to _xpack/usage [#148087]

Fixes

Aggregations:

• Aggs: Account aggs uncompressed size on coordinator reduction [#148885] (issue: [#147190])
• Aggs: Fix auto_date_histogram/date_histogram inside a global agg [#147427] (issue: [#92570])
• Fix ArrayIndexOutOfBoundsException in date_histogram with hard_bounds outside data [#148765] (issue: [#148763])

CCR:

• Fix CCR follow to handle indexing_complete race [#145304]

Distributed:

• Fix FsBlobContainer.moveBlobAtomic fallback on CIFS filesystems [#148777] (issue: [#148811])

ES|QL:

• Disallow casting to snapshot-only type [#148813] (issue: [#147867])
• Fix "optimized incorrectly" error from project reorder [#149053] (issue: [#148612])
• Fix ArrayIndexOutOfBoundsException in centroid grouping evaluateFinal [#148815] (issue: [#141318])
• Fix FROM *_logs index name validation [#149395] (issue: [#146073])
• Fix InvalidMappedField equals/hashCode and lazy error message [#146117] (issue: [#145907])
• Fix errors in blockloaders when filtering nulls on histograms [#148983] (issue: [#147854])
• Lucene TermsEnum and PostingsEnum may only be used from acquiring thread [#149297]
• Node-reduce driver should not release search contexts on failure [#145960] (issue: [#145509])
• Normalize _timeseries to JSON for stored source [#148716] (issue: [#148414])
• Use search threadpool for coordination in ES|QL [#149034]

Inference:

• Honor task settings for Amazon Bedrock and Google Vertex chat completion [#149268] (issue: [#148792])
• [Inference API] Fix NPE for Inference usage when referencing a embedding task type [#149636]
• [Inference API] Fixing potential NPE if InferenceContext is null [#148921]
• [Inference API] Use image/jpeg instead of image/jpg for inference validation MIME type [#148793]

Infra/Core:

• Pin server-launcher native -march to x86-64-v2 [#148542] (issue: [#148326])
• Use toTruncatedString in chunked objects toString [#147860] (issue: [#143694])

Infra/Logging:

• Monitor exceptions in processing queries and log them as failures [#148362]

Machine Learning:

• Fix flaky CIoManagerTest/testFileIoGood test [#3017]

PromQL:

• Default PromQL index to metrics-* [#148489]
• Default Prometheus discovery APIs to metrics-* [#148770]

SQL:

• Fix REPLACE with empty pattern [#148616]

Search:

• Default exclude_inference_fields to exclude_vectors at REST [#148527] (issue: [#146425])

Transform:

• Only run next search once [#148268] (issue: [#147716])
• Propagate deprecation warnings [#147934] (issues: [#82935], [#82936])

Vector Search:

• Fix gpu close resource ordering [#149054]
• Fix integer overflow in CuVSResourceManager.estimateNNDescentMemory [#147559]
• GPU codec: fall back to CPU graph build on flush when GPU is busy [#149373]

Watcher:

• Clear watch schedules when starting trigger engine [#145325] (issues: [#137562], [#131964])

Original source

Features and enhancements

Machine Learning

• Add EuroBERT and Jina v5 ops to graph validation allowlist #3015
• Better error handling regarding quantiles state documents #2894
• Better handling of invalid JSON state documents #2895
• Better messaging regarding OOM process termination #2841
• Downgrade log severity for a batch of recoverable errors #2889
• Harden pytorch_inference with TorchScript model graph validation #3008 (issue: #2890)
• Improve adherence to memory limits for the bucket gatherer #2848
• Report the actual memory usage of the autodetect process #2846
• Restrict file system access for pytorch models #2851
• Update the PyTorch library to version 2.7.1 #2863

Security

• Update elastic-apm-agent-java8 to 1.55.6 #148271

Fixes

Data streams

• Update failure store redirect logic to exclude backpressure exceptions #148154

ES|QL

• Bugfix - Block Loader Pushdown + Union Types #147940
• Disallow empty lists in named params, only #147748 (issue: #147448)
• ES|QL query approximation: enforce minimum number of sampled source rows #147886

Inference

• Implement RerankingInferenceService for Elastic service #148365
• [Inference API] Fix inference initialization thread exhaustion #147063

Machine Learning

• Fix flaky CIoManagerTest/testFileIoGood test #3017

Vector Search

• [DiskBBQ] Check that precondition should not be overwritten on update #148111 (issue: #148004)

Watcher

• Fix race in TickerScheduleTriggerEngine by checking watcher to node allocation #147678 (issue: #146874)

Original source

Highlights

ES|QL now supports Views: virtual indices whose fields are produced by an ES|QL query. A view is referenced inside a `FROM` clause exactly like a regular index, alongside other indices, views, and wildcards. Complex processing pipelines can be hidden behind a view, exposing a stable set of columns without requiring callers to know the underlying source structure. A single query can combine multiple pre-processed data sources by listing several views in one `FROM` clause, with each view's pipeline running independently. Common transformations such as renames, type conversions, derived fields, and aggregations can be defined once in a view and reused across many queries, dashboards, and alerts. PromQL is now supported as a source command in ES|QL (Tech Preview). Users can now leverage their existing knowledge of PromQL while benefiting from the powerful features and scalability of Elasticsearch. This enhancement expands the versatility of ES|QL and makes it easier for users to integrate with Prometheus data sources.The syntax is illustrated in the following example: ```esql PROMQL index=k8s-downsampled start="2026-02-17T08:00:00Z" end="2026-02-17T09:00:00Z" step=30m avg_bytes=(avg(rate(network.total_bytes_in[30m]))) | SORT avg_bytes DESC, step; ``` ES|QL adds the `METRICS_INFO` command for queries that start with a time series (`TS`) source. It returns one row per distinct metric, with columns such as `metric_name`, `data_stream`, `unit`, `metric_type`, `field_type`, and `dimension_fields`, derived from time series metadata in the index. It unlocks inspecting which metrics exist and how they are typed before you aggregate with `STATS`.For example, list metrics sorted by name: ```esql TS my_data_stream | METRICS_INFO | SORT metric_name ``` Or filter to counters only: ```esql TS my_data_stream | METRICS_INFO | WHERE metric_type == "counter" | SORT metric_name ``` We're introducing a Prometheus-compatible `POST /_prometheus/api/v1/write` REST entrypoint that allows receiving data via Prometheus remote write protocol (Tech Preview). Elasticsearch can now be used as a Prometheus storage backend, consuming data sent in Prometheus native format. ES|QL adds the `TS_INFO` command for time series (`TS`) queries. It returns one row per metric and time series combination. You get the same metadata columns as `METRICS_INFO`, plus a `dimensions` column with a JSON object of dimension keys and values for that series. That unlocks inferring which labels apply to each series when exploring or validating time series data.For example: ```esql TS my_data_stream | TS_INFO | SORT metric_name, dimensions ``` Until Elasticsearch `9.3`, both downsampling methods (`aggregate` and `last_value`) used to store only the last value of a counter in the downsampled document. This works great for the `last_value` method where we optimise for storage efficiency, but it is not ideal for the `aggregate` method where we optimise for accuracy.In Elasticsearch `9.4`, we change the way the (default) `aggregate` sampling method is working. We store the first encountered value for a counter in the downsampled document and then we add auxiliary documents when we detect counter resets. This enables the rate calculation to take the counter resets into account and produce more accurate results. This change is backwards compatible. Time series aggregations in ES|QL are enhanced to support windows smaller than the time bucket. ```esql TS metrics | STATS AVG(RATE(requests, 5m)) BY TBUCKET(10m), host ``` Previously, only window values that were equal or exact multiples of the time bucket were supported. Time series aggregations in ES|QL are enhanced to support windows that are not an exact multiple of the time bucket. ```esql TS metrics | STATS AVG(RATE(requests, 15m)) BY TBUCKET(10m), host ``` Previously, only window values that were exact multiples of the time bucket were supported. This updates our diskbbq algorithm and format. - It now provides 3x or more better search performance on very restrictive filters (prefilters on centroids) - Provides a way to condition non iid vectors (expert API for now) - Gives more bit options (1, 2, 4, and 7 bits!) - More native code improvements for overall performance The `_id` field has a significant storage footprint in metrics applications, as it requires both storing and indexing unique document identifiers that are rarely used for direct lookups. To alleviate this, we are introducing synthetic IDs for indices in time-series mode. Instead of indexing the `_id` field, a Bloom filter is used for fast, lightweight duplicate detection at ingest time. Lookups and operations that previously relied on `_id` are delegated to other indexed fields on the document, such as timestamps, or dimension fields, preserving the same query and retrieval functionality.This offers up to 40% storage improvement for OTLP metrics and reduces the cpu overhead for segment merging due to the lack of an inverted index for `_id` fields. We're introducing a Prometheus-compatible /_prometheus/api/v1/query_range REST endpoint (Tech Preview) that: - Accepts the standard Prometheus range query parameters (query, start, end, step, optional index) - Translates the PromQL expression into an ES|QL PROMQL command and executes it via EsqlQueryAction - Converts the columnar ES|QL response into the Prometheus matrix JSON format and returns it to the caller We're introducing a Prometheus-compatible `GET /_prometheus/api/v1/series` REST entrypoint that accepts Prometheus series selectors and returns matching label sets (Tech Preview). This is typically used for auto-completion in web UIs. The first bytes of a time series id (tsid) include a hash of the metric name(s) for each doc of a time-series index. Counter rate evaluation leverages these bytes to assign tsids to workers inside the ES|QL compute engine. This (a) improves parallelism by dividing work in a granular and uniform fashion, and (b) leads to dense, sequential access patterns per time series that have been optimized to avoid copies between counter value decoding and rate calculations.Rate execution performance thus improves substantially, with up to 5x faster query responses. We're introducing a Prometheus-compatible `GET /_prometheus/api/v1/labels` REST entrypoint for time series discovery and label enumeration and introspection (Tech Preview). Web UIs can use this for label auto-completion. We're introducing a Prometheus-compatible `GET /_prometheus/api/v1/query` REST endpoint that evaluates a PromQL expression at a single point in time and returns vector results (Tech Preview). The instant query endpoint currently runs a short range query under the hood and returns the last sample. In Elasticsearch `9.4` we expand the supportability of `aggregate_metric_double` to include non-native operations in ES|QL, such as `std_dev`, using the average. The average is calculated using the `sum` and `value_count` sub-fields. The average was selected because in most cases it is a more representative signal compared to a single sub-field. Native operations such as `max`, `min`, `sum`, `avg`, and `count` will be supported natively by the respective sub-fields.For example, the following query is now supported where `network.eth0.tx` is a an `aggregate_metric_double`: ```esql FROM k8s-downsampled | STATS max = max(network.eth0.tx), std_dev = STD_DEV(network.eth0.tx) by pod | sort pod ``` Response: ``` max:double | std_dev:double | pod:keyword 1060.0 | 275.6970067 | one 824.0 | 184.1213952 | three 1419.0 | 356.9865993 | two ```

Features and enhancements

Aggregations:

• Bump heap usage limits for INLINE STATS #144679

Analysis:

• Inject circuit breaker into forked SynonymMapBuilder #144800
• Support custom rulesets in analysis-icu/icu-transform plugin #143060

Authentication:

• Add Clone API Key endpoint #142633 (issue: #59304)

Authorization:

• Update View CRUD Actions to be Index Actions #141570
• [Entity Store] Add permissions for Entity Store datastream #145981

CCS:

• CPS and project routing support for templated searches #139446

CRUD:

• Do not mark bulk indexing requests as retried after primary relocations #142157 (issue: #141586)

Codec:

• Add dynamic bloom filter sizing based on document count #141342
• Add panama simd implementation of contains function for BinaryDocValuesContainsTermQuery #143922
• Allow loading BYTE_LENGTH without decompressing Zstd byte ref blocks #141322
• ES819 Binary doc values: compact doc offsets using bit packing #142772
• Enable large blocks for binary doc values by default. This mainly affects fields of type wildcard, ignored source, values hitting ignore above threshold and ignore malformed numbers and dates. #145216
• Fast codePointCount implementation for BytesRef #140388
• Push contains binary doc values query down to es819 codec #143898
• Rewrite *substring* wildcard queries to contains term queries for binary doc values keywords #143433
• Track bloom filter disk usage in IndexDiskUsageAnalyzer #142106
• Upgrade zstd to version 1.5.7 #140530
• Use DirectAccessInput in ZstdDecompressor to avoid intermediate heap copy #145658
• Use max instead of median for merged bloom filter size #143302
• CodePointCount implementation using Panama vectors API #140693 (issue: #140567)

Data streams:

• Add 'logs.otel' and 'logs.ecs' stream types #141564 (issue: #141040)
• Ensure DLM only runs one general loop at a time #143883
• Support Failure Stores in Cross Cluster Search #139316

Distributed:

• Batch index creation #144074
• Batch snapshot update tasks after external change #142091
• Ensure that synthetic _id is usable after restarts/relocations #138678
• Health reports GREEN when provisionally unassigned replica #144773
• Increase the per-index limit for merges to half the CPUs #141389
• Opt-in persistent task reassignment on node shutdown #143306

Downsampling:

• Collect dimensions only once per tsid when downsampling #145089
• Rate calculation for downsampled counters becomes aware of counter resets when the aggregate sampling method is used. #143381 (issue: #136178)
• Use the tdigest type and compression from TDigest in downsampling #143247

ES|QL:

• Add APM telemetry for SET statement #141719
• Add Arrow-native Block & Vector implementations #142981
• Add CCS Remote Views Detection #143384
• Add Connector SPI and gRPC/Arrow Flight module #142667
• Add Google Cloud Storage data source plugin #142563
• Add JSON_EXTRACT ES|QL scalar function #142375
• Add LZ4, Snappy, and Brotli decompression codecs #144688
• Add METRICS_INFO command #141667 (issue: #139296)
• Add MMR command for result diversification #143867
• Add MV_UNION Function #139664
• Add ORC predicate pushdown via SearchArgument #144686
• Add Parquet filter pushdown with bloom filter, statistics, and dictionary row-group skipping #144832
• Add TS_INFO information retrieval command #142721 (issue: #139296)
• Add Views Security Model #141050
• Add Warning for Sort Under Lookup Join #141482 (issue: #141483)
• Add FormatReadContext to consolidate FormatReader API #143928
• Add IntRangeVector for selected groups in aggregation #141205
• Add LongLongSwissHash - specialization for grouping by two long fields #140838
• Add appliesTo to the TRange and TBucket functions #142160
• Add anonymous Azure access via auth=none #144475
• Add anonymous GCS access via auth=none #144476
• Add anonymous S3 access via auth=none #144471
• Add blocks and vectors for more Arrow numeric types #145111
• Add cloud API rate limiting for external sources #144734
• Add column pruning for external datasources #143903
• Add configurable bracket-based multi-value support for CSV reader #143890
• Add coordinator-only caching for external source metadata #145300
• Add data node execution for external sources #143209
• Add dense_vector equality and inequality support in ES|QL #140005 (issue: #139929)
• Add error handling and propagation for external source execution #143333
• Add error policy and configurable options for CSV format reader #143779
• Add extended distribution tests and fault injection for external sources #143420
• Add info into the profile of METRICS_INFO and TS_INFO #145634
• Add limit pushdown for external data sources #143515
• Add local parallelism and partition detection for external sources #143154
• Add logic to fold project tags metadata on data nodes #141935
• Add mapper-size plugin's _size metadata attribute #141427 (issue: #136956)
• Add memory tracking for TS_INFO and METRICS_INFO #143491 (issue: #139296)
• Add parallel execution for Arrow Flight multi-endpoint sources #143345
• Add parameter support in PromQL query durations #139873 (issue: #139508)
• Add pluggable partition detection and virtual columns #143120
• Add positional readBytes API to StorageObject SPI #143703
• Add schema reconciliation for multi-file external sources #145220
• Add split SPI, partition detection, and filter hint extraction #143005
• Add split discovery and distribution for external sources #143114
• Add support for ORC file format #142900
• Add support for dense_vector in COALESCE #142974 (issue: #139928)
• Add support for binary operators with AMD #143996 (issue: #142094)
• Add support for project METADATA #140592
• Add support for top-level arithmetic ops to TS|STATS #140135 (issue: #139570)
• Add syntax support and parsing for SET approximate #139908
• Add telemetry (stack) for query settings #141836
• Add timezone to add and sub operators, and ConfigurationAware planning support #140101
• Add xerial snappy-java to compression-libs #145393
• Added three new simple but useful spatial functions: ST_Dimension, ST_GeometryType, ST_IsEmpty #144703
• Added timezone support to date_format #138517
• Adding ES|QL USER_AGENT command #144384 (issue: #134886)
• Adding ES|QL command REGISTERED_DOMAIN #142680 (issue: #133942)
• Adding ES|QL command URI_PART #140004 (issue: #134885)
• Adding MV_INTERSECTS function #140662
• Adding sparkline aggregate function #141388
• Adds LIMIT BY ESQL command in Tech Preview #145225 (issue: #112918)
• Adds ST_SIMPLIFY geospatial function #136309 (issue: #44747)
• Allow TBUCKET to skip the from/to parameters when Kibana adds a timestamp range filter. Exmaple: TBUCKET(100) #144057
• Allow evaluatable grouping functions (Like BUCKET) in LIMIT BY #146642
• Attribute ES|QL shard search load in Lucene operators #142841
• Avoid caching multiple times in doc-partitioning #142913
• Bridge Connector SPI to ExternalSplit #143331
• Buffer reuse in ParquetStorageObjectAdapter and StorageObject #143700
• Byte-based buffer backpressure for external sources #144218
• CSV schema inference and parsing enhancements #144050
• Case Support for Compound Types #140677
• Converted PackedValuesBlockHash.bytes to BreakingBytesRefBuilder for better memory tracking #140171
• Count aggregation for histograms #141138
• DS: Parquet file handling improvements #145123
• Data sources: Azure plugin #143236
• Data sources: ZSTD, BZIP2 #143228
• Datasources: GZIP #143035
• Document and test Parquet page-index filtering #145571
• ESQL - Add dense_vector field type to SUM function #142129
• ESQL - Improve search performance by adding min competitive aware collection when using multiple shards / threads #142406 (issue: #136267)
• ESQL 137269 some csv tests for lookup join behavior with multivalues #144520
• ESQL mv_difference function #141895
• ESQL: Improve field reference tracking in FORK command #137678 (issue: #137283)
• ESQL: Prune unused regex extract nodes in optimizer #140982 (issue: #132437)
• ESQL: Support intra-row field references in ROW command #140217 (issue: #140119)
• ESQL: enable unmapped_fields="load" in tech preview #145052 (issue: #142369)
• ES|QL - Add parsing, preanalysis and analysis timing information to profile #139540
• ES|QL - Top N queries are parallelized #143133
• ES|QL - dense_vector support for COUNT, PRESENT, ABSENT aggregator functions #139914 (issue: #135688)
• ES|QL CHUNK function multi-valued field support #141240
• ES|QL Improve LOOKUP JOIN on single keyword #144704
• ES|QL Top Snippets multi-valued field support #142117
• ES|QL Views support #134995
• ES|QL TEXT_EMBEDDING function is GA #140555
• ES|QL dense vector functions are GA #140545
• ES|QL approximate analytical queries #131828
• ES|QL command RERANK is GA #141508
• Enable PromQL command in ES|QL #140808
• Enable distributed pipeline breakers for external sources via FragmentExec #143696
• Enable doc-partitioning for more queries #143095
• Extract centroid from doc values for ST_CENTROID_AGG over geo_shape and cartesian_shape #142528 (issue: #142640)
• Fix ORC type support gaps #145074
• Fix Parquet and ORC datasource allocation overhead #143791
• Fix Parquet type support gaps #144059
• Fix review feedback and add test coverage for PR #143703 #143900
• Fix window validation in time-series aggregations when TBUCKET uses a numeric target count #144291
• Format "_query" response dates using the given timezone #139529
• GCS native async I/O via ReadChannel #144733
• Harden distributed external source execution #144277
• Implement EXPLAIN for local data node plans #142748
• Implementing rerank on multi values #140672
• Improve Lookup Join performance with CachedDirectoryReader #139314 (issue: #137268)
• Improve memory usage and tracking by moving union types into ValuesSourceReaderOperator #140384
• Improve ndjson schema inference for date-time #145553
• Introduce "Swiss Table"-based hashing to ES|QL, a SIMD-accelerated hash table resulting in significantly higher throughput on uniform, high-cardinality workloads #145010
• Introduce Geospatial functions ST_Buffer and ST_SimplifyPreserveTopology #145154
• Introduce SwissTable-based hashing for ES|QL STATS #139343
• Introduce adaptive block hash for long/int #141237
• JSON_EXTRACT: zero-copy byte slicing for object, array, and number extraction #143702
• LIMIT BY fixed telemetry and tests #146992
• MMR Command: Grammar and Logical Plan #140684
• Make MV_EXPAND GA #144543
• Make datasources plugins lazy #142815
• Minimize Hadoop dependencies for ORC plugin #146944
• Optimize TopNOperator to avoid resorting when input is already sorted #141094 (issue: #131221)
• Partition rate query using tsid prefixes #144818
• Per-file filter pushdown awareness #145755
• Periodically emit partial aggregation results #141392
• Push STARTS_WITH/LIKE prefix to Parquet and ORC #145640
• Push stats to external source via metadata #143940
• Reapply "Introduce pluggable external datasource framework" #142707
• Reapply "NDJSON datasource" #142855
• Refactor inference operator architecture for multi-value field support #139694
• Register TSV as a separate format with tab delimiter #143906
• Remove Hadoop JARs from Parquet plugin #146780 (issue: #146716)
• Remove hadoop-client-runtime from datasource plugins #146206 (issue: #146203)
• Remove implicit limit appended for each subquery branch #139058
• Remove implicit limit for FORK #145429
• Remove snapshot protection from node reduce late materialization #142834
• Review fixes for datasource framework #142565
• Route external source I/O through esql_worker thread pool #144596
• Schema-aware filter pushdown for DATETIME and DECIMAL #145641
• Shrink description #140089
• Skip files with no projected column overlap in UNION_BY_NAME #145701
• Skip time series field type merge for non-TS agg queries #143262
• Speed up remote Parquet reads #144454
• Stats pushdown past EVAL/RENAME for external sources #144806
• Stream results from topn #140088
• Support arithmetic operations for dense_vectors: scalar version #141060 (issue: #140538)
• Support arithmetic operations for dense_vectors: vector version #140539 (issue: #140537)
• Support of a window that is not an exact multiple of the bucket #143704
• Support shapes in ST_CENTROID_AGG #141657
• Support target bucket count in TBUCKET with explicit from/to date range #142747
• Support window smaller than time bucket #143661
• TRange timezone support #139911
• Type conflict resolution in unmapped-fields load #143693 (issues: #142004, #141912)
• Use avg metric for AMD default metric #141331
• Use less memory in ValuesFromMany #140062
• Validate TOP_SNIPPETS query argument is foldable at verification #142763 (issue: #142462)
• Various fixes to spatial functions (ST_ENVELOPE and ST_NPOINTS) #139618
• [ES|QL|DS] Add circuit breaker to the Parquet datasource #144491
• [ES|QL|DS] Parquet row-group level split parallelism #144018
• [ES|QL|DS] Wire parallel parsing into production for text formats #143997
• ToString/ToDatetime/ToDateNanos converters timezone support #138985
• support DATE_RANGE field type #133309
• Add CHICKEN function to ES|QL #140645

Engine:

• Ensure acquired snapshot commit is always flushed #144067 (issue: #143993)

Indices APIs:

• More actionable PUT /{index}/_settings error #138611

Inference:

• Add FireworksAI chat completion support #142664
• Add FireworksAI inference service for embeddings #137130
• Add embedding task support to ElasticInferenceService #141547
• Add provider validation call to Update Inference Endpoint operation #140003 (issue: #122356)
• Added Reasoning support for Chat Completion in the Inference Plugin #143242
• Added service settings update logic for AI21 provider in the Inference Plugin #142597 (issue: #122356)
• Added service settings update logic for Alibaba Cloud Search provider in the Inference Plugin #142738 (issue: #122356)
• Enable multimodal inputs for all chat completion integrations #144509
• Removed the max_tokens request parameter for Chat Completion with Reasoning in the Inference Plugin #143242
• [Inference API] Add Chat Completion to Amazon Bedrock for the Inference API #139411
• [Inference API] Add custom headers for Azure OpenAI Service #142969
• [Inference API] Add support for embedding task to JinaAI service #140323
• [Inference API] Adding OAuth2 support for Azure OpenAI #143896
• [Inference API] Expose Endpoint Heuristics through Inference API #141393
• [Inference API] Handle preconfigured endpoints with embedding task type #141788
• [Inference API] Parse endpoint metadata from persisted endpoints #143081
• [Inference API] Support multimodal inputs for chat completion #142736
• [Inference API] Update authorized endpoints when their fingerprint or version changed #143567

Infra/Core:

• Add DateFormatter.tryParse() #144474
• Expose byte offsets on XContentParser via getCurrentLocation() #143501 (issue: #142873)

Infra/Plugins:

• [Fleet] Add OpAMP field mappings to fleet-agents #142550
• [Fleet] Add metadata mappings for OpAMP #145824

Infra/Scripting:

• Painless hoist constant collection .contains calls #143311 (issue: #137849)

Ingest Node:

• Update Grok to use the new Matcher#setTimeout #139405
• [INGEST] GrokProcessor: add validate_only option to skip field extraction #145126

Logs:

• Default index.mapping.use_doc_values_skipper to true for logsdb #142851
• Store fallback match only text fields in binary doc values #140189

Machine Learning:

• Add EuroBERT and Jina v5 ops to graph validation allowlist #3015
• Add a suggestion for fixing the ML node allocation error #139520
• Add exponential-backoff retry for AD job opening during system-initiated reassignments #144478
• Add support for nested NDJSON records in TextStructure endpoints #141045 (issue: #127777)
• Better error handling regarding quantiles state documents #2894
• Better handling of invalid JSON state documents #2895
• Better messaging regarding OOM process termination #2841
• Downgrade log severity for a batch of recoverable errors #2889
• Harden pytorch_inference with TorchScript model graph validation #3008 (issue: #2890)
• Improve adherence to memory limits for the bucket gatherer #2848
• Report the actual memory usage of the autodetect process #2846
• Restrict file system access for pytorch models #2851
• Update the PyTorch library to version 2.7.1 #2863

Mapping:

• Add option to enable accurate leaf arrays for flattened fields #145376
• Add passthrough support to flattened field type for mapped sub-fields #145131
• Add properties support to flattened field type #144451
• Aggregate metric double use average #142135
• Improve the supportability of aggregate_metric_double by non-native ES|QL aggregation functions, such as std_dev. #145742
• Remove redundant root doc values from flattened fields if index=false #143907
• Set default semantic_text index type to disk_bbq by using dense_vector defaults #145374
• Store flattened field data in binary doc values #140246
• Update semantic text to use BFLOAT16 by default #144236

Monitoring:

• Add mode and codec fields to Stack Monitoring index template #143673

Packaging:

• Flip cloud-ess-fips default from FIPS 140-2 to FIPS 140-3 #140788

Performance:

• Allow intermediate builds in PR-based benchmarks #142472
• Correctly reference non-main branches in benchmark script #142303
• Relax PR-based benchmarks target branch #142297

PromQL:

• Add Prometheus instant query REST endpoint #145321
• Add Prometheus labels REST endpoint #144952
• Add Prometheus query_range endpoint #144416
• Add Prometheus series REST endpoint #144494
• Implement Prometheus remote write indexing support #141957

Ranking:

• Use VectorScorer to consume AcceptDocs iterator for lazy bulk scoring in VectorScoringUtils #145835 (issue: #145834)

Reindex:

• Add reindex-from-remote blocklist setting #145357
• Disable OCC in update/delete-by-query for seq_no-less indices #143465

Relevance:

• GA chunk_rescorer in text_similarity_reranker #139830

SQL:

• Add project_routing to CLI #138965
• Add support for API key to JDBC and CLI #142021

Search:

• Account for ES|QL Lucene query rewrite in recent search load #141819
• Add semantic_text field type to MMR Result Diversification Retriever #141666
• Add search task watchdog to log hot threads on slow search #142746
• Added return_intermediate_results query param to toggle when partial results are returned for a get async results operation #141073 (issue: #139828)
• CPS handles datastreams #140637
• Expose keep_alive in async task status #144010
• Fail MatchQueryParser if it generates a query with more clauses than allowed by max_clause_count #143233 (issue: #143032)
• Ids Query: Use max result window as upper limit #140515 (issue: #138758)
• Makes scroll CPS compatible #140977
• Making use of sort optimization written from search in search shards #144247 (issue: #143945)
• Only consider the primary sort when determining concurrency #143608
• Optimize script sorts that do not require query scores #139748
• Optimize search shard iterator sort #140747 (issue: #135472)
• PIT context relocation work on main repo #137675
• Prevent creating too many nested boolean clauses while creating the lucene query to avoid query explosion #143220
• Ref-counting SearchHits from InternalTopHits to SearchResponse #142732
• Search/query logging support for _search, ES|QL, EQL, SQL #139920
• Semantic text default inference id setting #143486
• Switch default model for semantic_text to jina-v5 #142980
• Take control of max clause count verification in Lucene searcher #139752
• Update text_similarity_rank_retriever to default to chunking settings optimal for inference ID #137397
• Upgrade Elasticsearch to Apache Lucene 10.4 #141882
• Use IndexOrDocValuesQuery in IpFieldType#termQuery #140735
• Use IndexOrDocValuesQuery in NumberFieldType#termQuery implementations #140734
• CanMatch returns numSkipped per cluster instead of all skipped shards #142170

Searchable Snapshots:

• Add SparseFileTracker.getAbsentBytesWithin #141179
• Split blob-cache freelist using decays #142545
• Trigger cache decay at 5% left on freq 0 #142685

Security:

• Allow deleting multiple views in one request #145816
• Don't allow querying views with DLS or FLS #144903
• Make ServiceAccountToken APIs Available in Serverless #140631
• Upgrade bouncycastle to 1.84 #147197
• Use opaque random session IDs for ESQL compute sessions #142249

Snapshot/Restore:

• Batching of snapshot-delete start updates #141998
• Identify Elasticsearch as user-agent in S3 calls #141881
• Reduce memory usage of TransportGetSnapshotsAction #142468
• Report shard snapshot pauses in shutdown status #144717
• Strengthen MPU-based CAS in S3 repo

Original source

Features and enhancements

ES|QL:

• Add filter capture for ESQL query [#149535]
• Upgrade Apache Iceberg to 1.10.2 [#151664]

Inference:

• [Inference] Add requires org.apache.commons.lang3 to module-info [#151794]

Infra/Logging:

• Add query filter logging to SQL and EQL queries [#150127]

Machine Learning:

• Add EuroBERT and Jina v5 ops to graph validation allowlist [#3015]
• Better error handling regarding quantiles state documents [#2894]
• Better handling of invalid JSON state documents [#2895]
• Better messaging regarding OOM process termination [#2841]
• Downgrade log severity for a batch of recoverable errors [#2889]
• Harden pytorch_inference with TorchScript model graph validation [#3008] (issue: [#2890])
• Improve adherence to memory limits for the bucket gatherer [#2848]
• Report the actual memory usage of the autodetect process [#2846]
• Restrict file system access for pytorch models [#2851]
• Update the PyTorch library to version 2.7.1 [#2863]

Fixes

Aggregations:

• Fix AVLTreeDigest CDF monotonicity for penultimate centroid [#151181] (issue: [#151012])
• Fix non-monotonic T-Digest centroids caused by floating-point errors [#151979]

ES|QL:

• Avoid long overflows on AVG by casting to double [#148519] (issue: [#99575])
• Carry over synthetic attributes from conversions to the Project above UnionAll [#149775] (issue: [#149509])
• Fix ValuesReader assertion error [#149683]
• Fix subquery/fork with inline stats pruning [#149783] (issue: [#149589])
• Fix wildcard view resolution losing duplicate copies [#149418] (issue: [#149416])
• Make subquery with counter type not present in all branches behave more consistently [#150704] (issues: [#150640], [#150641], [#150644], [#150645])
• Resolve surrogates in union type field resolution before plan serialization [#151633] (issue: [#151475])

Infra/Core:

• Pass through includeSourceInError [#142335]

Machine Learning:

• Fix bug causing incorrect error on force deleting already deleted model [#107188] (issue: [#105420])
• Fix flaky CIoManagerTest/testFileIoGood test [#3017]
• Revert "[ML] Add extra validation in trained_model creation" [#152000]

Mapping:

• Don't allow runtime fields to shadow fields used in index sort [#150636] (issue: [#149500])

Reindex:

• Fix division of max_docs between slices in reindex [#150808]

SQL:

• Fix CLI tests failures due to unconsumed output [#145967] (issues: [#143646], [#143645])

Search:

• Guard against null zeroTermsQuery in cross_fields multi_match [#149935] (issue: [#149934])
• Ids Query: Throw IllegalArgumentException [#151234] (issue: [#150305])

Security:

• Run remote child task cancellation as system user [#150560]

TSDB:

• Fix prefix escape for synthetic id [#150433] (issue: [#150389])

Vector Search:

• Fix GPU SQ format name to match CPU equivalent [#149512] (issue: [#148975])

Original source

Features and enhancements

Security

• Skip automaton construction for literal resource strings in application privilege checks #144685

Fixes

CAT APIs

• Fix ArrayIndexOutOfBoundsException in cat nodes for partial load average arrays #144859

CCR

• CCR follower index needs to copy transport version from CCR leader index #145035

ES|QL

• Cancel async query on expiry #143016 (issue: #142662)
• Don't over-alloc when unpacking dimension values #144525
• ESQL: Fix incorrectly optimized fork with nullify unmapped_fields #143030 (issue: #142762)
• ESQL: Fix null comparison type checking #140660 (issue: #140460)
• Fix field caps incorrectly synthesizing object parents under subobjects:false passthrough mappers #144183 (issue: #144179)
• Reduce LuceneOperator.Status memory consumption with large QueryDSL queries #143175 (issue: #143164)

FIPS

• Fix createLDAPCertStore failing in FIPS mode #144453 (issues: #144376, #144377)

Geo

• Fix geo_centroid over geo_shape merging multiple shards #144637 (issue: #144504)

Infra/Core

• Fix system index mapping update for reindexed indices after migration #144782 (issue: #144764)

Machine Learning

• Fix "reset anomaly detection API" crashing when only colliding indices exist #144545 (issue: #144544)

Mapping

• Lazily create matcher in BinaryDvConfirmedQuery #144698

Search

• Fix ArrayIndexOutOfBoundsException in fetch phase with partial results #144385 (issue: #140495)
• Fix circuit breaker leak in percolator query construction #144827
• Fix request cache invalidation to use ES cache helper consistently #144581
• Fix wrong return value in ContextIndexSearcher.totalTermFreq #144333

Snapshot/Restore

• Fix up exception messages in AzureBlobStore #144654

Vector Search

• Fix dense_vector default index options when using BFLOAT16 #145202 (issue: #145204)
• [DiskBBQ] Fix index sorting on flush #144938
• [DiskBBQ] Fix index sorting on flush (2nd attempt) #145076

Original source

Features and enhancements

Authentication

• Improve SAML error handling by adding metadata #137598 (issue: #128179)
• In-response-to in saml successful response #137599 (issue: #128179)

Security

• Skip automaton construction for literal resource strings in application privilege checks #144685

Fixes

CAT APIs

• Fix ArrayIndexOutOfBoundsException in cat nodes for partial load average arrays #144859

CCR

• CCR follower index needs to copy transport version from CCR leader index #145035

ES|QL

• Cancel async query on expiry #143016 (issue: #142662)
• Reduce LuceneOperator.Status memory consumption with large QueryDSL queries #143175 (issue: #143164)

FIPS

• Fix createLDAPCertStore failing in FIPS mode #144453 (issues: #144376, #144377)

Geo

• Fix geo_centroid over geo_shape merging multiple shards #144637 (issue: #144504)

Infra/Core

• Fix system index mapping update for reindexed indices after migration #144782 (issue: #144764)

Machine Learning

• Fix anomaly detection jobs stuck in opening by continously retrying to set the state to opened. #139668
• [Inference API] Support chunking settings for sparse embeddings in custom service #138776

Search

• Fix ArrayIndexOutOfBoundsException in fetch phase with partial results #144385 (issue: #140495)
• Fix circuit breaker leak in percolator query construction #144827
• Fix request cache invalidation to use ES cache helper consistently #144581
• Fix wrong return value in ContextIndexSearcher.totalTermFreq #144333

Snapshot/Restore

• Fix up exception messages in AzureBlobStore #144654

Vector Search

• [DiskBBQ] Fix index sorting on flush #144938
• [DiskBBQ] Fix index sorting on flush (2nd attempt) #145076

Watcher

• Validate Watcher Proxy Allowlist #144759

Original source