HAProxy Enterprise Updates & Release Notes

Key changes in the HAProxy Enterprise 3.1r1 release include:

ADFSPIP module

The new ADFSPIP module enables HAProxy Enterprise to give external clients access to web applications running in a Windows corporate network. HAProxy Enterprise becomes a proxy in front of Microsoft Active Directory Federation Services and web applications running inside the corporate network. It can route external clients to an AD FS sign-in page and to the web applications that they would otherwise not be able to reach.

Web Application Firewall module

The Web Application Firewall (WAF) has added support for WAF profiles. Defined in the new waf-profile section, a profile specifies a set of WAF parameters that can be applied to any WAF in the configuration. You can define multiple WAF profiles in a configuration.

Captcha module

New Captcha module options give you better control over the cookie that the module creates in the user’s browser. You can set the cookie’s domain, expiration, max age, path, SameSite option, and Secure option.

Global Profiling Engine

This version updates the Global Profiling Engine, which typically runs on a server separate from HAProxy Enterprise, to have new logging capabilities. You can send log messages to a file, a local syslog UNIX domain socket, or a remote syslog server.

UDP module

When configuring the UDP module via its udp-lb configuration section, you can now set the hash-type and hash-balance-factor directives to control how hash-based load balancing behaves with UDP traffic.

Stick table aggregator

For customers who have not migrated to the Global Profiling Engine, HAProxy Enterprise 3.1 introduces the new Stick Table Aggregator package version 2.1. It has the following enhancements:

• show aggrs now supports multiple buffers, ensuring all data is visible instead of just the first chunk.
• Active connections to down peers are deactivated by default, reducing unnecessary overhead. The legacy behavior can still be enabled with the legacy-active-connect argument.
• A new no-ascend option prevents data from being sent to up peers in multilayer environments.
• Multiple from lines are now supported per aggregation, offering greater flexibility in defining aggregation sources.
• The module now properly handles previously unsupported stick table data types.

Logging

The way that HAProxy Enterprise emits its logs is more flexible now with the introduction of log profiles, which let you assign names to your log formats. By defining log formats with names, you can choose the one best suited for each log server and even emit logs to multiple servers at the same time, each with its own format. Log profiles also allow you to write log messages when different events occur, such as when accepting a connection, receiving a request, connecting to a backend server, and receiving a response.

With the new do-log action, you can emit custom log messages throughout the processing of a request or response, allowing you to add debug statements that help you troubleshoot issues. Add the do-log action at various points of your configuration.

The option tcplog directive now allows an optional argument: clf. When enabled, CLF (Common Log Format) sends the same information as the non-CLF option, but in a standardized format that CLF log servers can parse.

set-retries action

The tcp-request content and http-request directives have a new action named set-retries that dynamically changes the number of times HAProxy Enterprise will try to connect to a backend server if it fails to connect initially.

quic-initial directive

The new quic-initial directive, which you can add to frontend, listen, and named defaults sections, gives you a way to deny QUIC (Quick UDP Internet Connections) packets early in the pipeline to waste no resources on unwanted traffic.

SPOE

In your configuration file, set mode to spop in the backend that contains the SPOE agents. This mode is now mandatory and automatically set for backends referenced by SPOEs. Configuring your backend in this way means that you are no longer required to use a separate configuration file for SPOE. This new spop backend mode adds flexibility to SPOE, optimizes traffic distribution among servers, improves performance, and will ultimately make the entire system more reliable, as future changes to the SPOE engine will only affect pieces specific to SPOE.

You can now pass variables from the main stream that’s processing a request to the child stream of a Stream Processing Offload Agent (SPOA). When reading a variable in an spop backend, prefix the variable scope with the letter p for parent stream. For example, instead of req, use preq. This works for these scopes: psess, ptxn, preq, and pres.

The following SPOE parameters were removed in this version and are silently ignored when present in the SPOE configuration:

• maxconnrate
• maxerrrate
• max-waiting-frames
• timeout hello
• timeout idle

Health checks

When defining health checks, add the new init-state argument to a server directive or server-template directive to control how quickly each server can return to handling traffic after restarting, coming out of maintenance mode, or adding the server through service discovery.

The option httpchk directive supports a new parameter for passing a host HTTP header with each health check to backend servers. This avoids having to specify carriage return and newline characters to do it, unlike previous versions.

Address families

When setting an address on a bind line, you can now set these address family prefixes:

• abnsz@: Zero-terminated abstract namespace. This lets you interconnect with software that determines the length of the namespace’s name by the length of the string, terminated by a null byte.
• mptcp@, mptcp4@, mptcp6@: MultiPath Transmission Control Protocol (MPTCP). MPTCP is a TCP extension that improves resource utilization, increases throughput, and responds quicker to failures.

Fetch methods

This version introduces the following fetch methods:

• ssl_c_san: Returns a string of comma-separated Subject Alt Name fields contained in the client certificate.
• ssl_fc_sigalgs_bin: Returns the content of the signatures_algorithms (13) TLS extension presented during the Client Hello.
• ssl_fc_supported_versions_bin: Returns the content of the supported_versions (43) TLS extension presented during the Client Hello.
• last_entity: Returns the identity of the last entity that was evaluated during stream analysis.
• waiting_entity: Returns the identity of the entity that was waiting to continue its processing when an error or a timeout was encountered.

Converters

This version introduces the following converters:

• date: Converts an HTTP date string to a UNIX timestamp.
• rfc7239_nn: Converts an IPv4 or IPv6 address to a compliant address that you can use in the from field of a Forwarded header. The nn here stands for node name. You can use this converter to build a custom Forwarded header.
• rfc7239_np: Converts an integer into a compliant port that you can use in the from field of a Forwarded header. The np here stands for node port. You can use this converter to build a custom Forwarded header.
• when: Enables you to pass data, such as debugging information, only when a condition is met, such as an error condition.

Runtime API

This version makes the following changes to the Runtime API:

• The debug counters command shows all internal counters placed in the code. Primarily aimed at developers, these debug counters provide insight for analyzing glitch counters and counters placed in the code using the new COUNT_IF() macro. Developers can use this macro during development to place arbitrary event counters anywhere in the code and check the counters’ values at runtime using the Runtime API. For example, glitch counters can provide useful information when they are increasing even though no request is instantiated or no log is produced.
• The dump ssl cert command will display an SSL certificate directly in PEM format; useful for placing delimiters and saving a certificate when it was updated on the CLI and not on the filesystem yet. You can also dump a transaction by prefixing the filename with an asterisk. This command is restricted and can only be issued on sockets configured for level admin.
• The echo command with syntax echo will print what’s contained in to the console output; it’s useful for writing comments in between multiple commands.
• This version improves the show dev Runtime API command by printing more information about arguments provided on the command line as well as the Linux capabilities set at process start and the current capabilities (the ability to preserve capabilities was introduced in Version 2.9 and improved in Version 3.0). This information is crucial for engineers troubleshooting the product.
• The show env command, which dumps environment variables known to the process, can now show information for a specific environment variable.
• The show quic command produces more internal information about the internal state of the congestion control algorithm and other dynamic metrics (such as window size, bytes in flight, and counters).
• The show info command will now report the current and total number of streams. It can help to quickly detect if a slowdown is caused on the client side or the server side and facilitate the export of activity metrics.

Troubleshooting

This release includes a number of troubleshooting and debugging improvements:

• Starting in version 3.1, traces get a dedicated configuration section named traces, providing a better user experience compared to previous versions. Traces report more information than before, too.
• The new when converter adds a condition to the end of a fetch method to only fetch the data when the condition is true, such as an error condition.
• As of version 2.5, you can use the sample fetches fc_err_* for frontends and bc_err_* for backends to help determine the cause of an error on the current connection. In this release, these fetches have been enhanced to include connection-level errors that occur during data transfers.
• The system may produce a core dump on a fatal error or when the watchdog fires, which detects deadlocks. While crucial to diagnosing issues, sometimes these files are truncated or can be missing information vital to analysis. This release includes an internal post_mortem structure to be included in core dumps, which contains pointers to the most important internal structures. This structure, present in all core dumps, allows developers to more easily navigate the process’s memory, reducing analysis time, and prevents the user from needing to change their settings to produce different debug output. Additionally, more hints have been added to the crash output to help in decoding the core dump. To view this debugging information without producing a core dump, use the improved show dev command.
• In previous versions, sometimes stderr outputs of the thread backtraces in core dumps would be missing, or only the last one was present due to the reuse of the same output buffer for each thread. Core dumps now include backtraces for all threads, as each thread’s backtrace is now dumped in its own buffer. Also present in core dumps as of this version are the output messages for each thread, which assists developers in determining the causes of issues even when debug symbols are not present.
• This version includes improvements to HAProxy’s watchdog, which detects deadlocks and kills runaway processes. The watchdog will now watch for stuck threads more often, by default every 100ms, and it will emit warnings regarding a stuck thread’s backtrace before killing it. It will stop the thread if, after the first warning, the thread makes no progress for one second. In this way, you should see ten warnings about a stuck thread before the watchdog kills it. Note that you can adjust the time delay after which HAProxy Enterprise will emit a warning for a stuck thread using the global debugging directive warn-blocked-traffic-after. We do not advise that you change this value, but changing it may be necessary during a debugging session.
• This version enhances the accuracy of the memory profiler by improving the tracking of the association between memory allocations and releases and by intercepting more calls such as strdup() as well as non-portable calls such as strndup() and memalign(). This improvement in accuracy applies to the per-DSO (dynamic shared object) summary as well, and should fix some rare occurrences where it incorrectly appeared that there was more memory free than allocated. New to this version, a summary is provided per external dependency, which can help to determine if a particular library is leaking memory and where.

Performance improvements

HAProxy Enterprise 3.1r1 adds these performance improvements:

• The CPU binding configuration that HAProxy Enterprise determines automatically for your machine provides the best performance, and most users should see no difference in configuration requirements. However, if you are using a large system with many cores and multiple CCX, or a heterogeneous system with both “performance” and “efficiency” cores, some additional configuration tuning can lead to further performance gains. See Performance optimization for large systems.
• The H2 mux is significantly more performant in this version. This was accomplished by optimizing the H2 mux to wake up only when there are requests ready to process, saving CPU cycles, and resulting in using 30% fewer instructions on average when downloading. The POST upload performance has been increased up to 24x with default settings and it now also avoids head-of-line blocking when downloading from H2 servers. Two new global directives, tune.h2.be.rxbuf and tune.h2.fe.rxbuf allow for further tuning of this behavior. Specify a buffer size in bytes using tune.h2.fe.rxbuf for incoming connections and tune.h2.be.rxbuf for outgoing connections. For both uploads and for downloads, one buffer is granted to each stream and 7/8 of the unused buffers is shared between streams that are uploading / downloading, which is the mechanism that significantly improves performance.
• New to this version are two new global directives for tuning QUIC performance:
  • tune.quic.cc.cubic.min-losses takes a number that defines a threshold for how many packets must be missed before the Cubic congestion control algorithm determines that a loss has occurred. This setting allows the algorithm to be slightly more tolerant to false losses, though you should exercise caution when changing the value from the default value of 1. A value of 2 may prove to show some performance improvement, though we do not recommend running this way for extended periods of time, only for analysis, and you should avoid providing a value larger than 2.
  • tune.quic.frontend.default-max-window-size defines the default maximum window size for the congestion controller of a single QUIC connection, by specifying an integer value between 10k and 4g, with a suffix of “k”, “m” or “g”.
• This version sees an efficiency improvement in regards to the QUIC buffer allocator and using this tunable, you are able to vary the size of the memory required per-connection, thus reducing overallocation.
• The transmission path for QUIC has been significantly improved in this version so that it will now adapt to the current send window size and will use Generic Send Offload to let the kernel send multiple packets in a single system call. This offloads processing from HAProxy Enterprise and the kernel and places it onto the hardware. This is especially meaningful when used on virtual machines where system calls have potential to be expensive.
• Small frames for the QUIC buffer handling now use small buffers. This improves both the memory and CPU usage, as the buffers are now more appropriately sized and do not require realignment. QUIC will always send a NEW_TOKEN frame to new clients for reuse in the next connection. This behavior permits clients to reconnect after being validated without going through the address validation process again on the next connection. In other words, the next established connection will improve network performance when a listener is attacked or when dealing with a lossy network.
• To help improve performance in the case of large configurations that consume a lot of CPU on reload, two global configuration directives tune.renice.startup and tune.renice.runtime are new to this version. These global directives take a value between -20 and 19 to apply a scheduling priority to configuration parsing.
• TCP logs saw a 56% performance gain in this version thanks to the implementation of the line-by-line parser into the TCP log forwarder.
• In regards to log servers, the ring sending mechanism sees improvement in this version, as the load is better balanced across available threads, assigning new server connections to threads with the least load. You can now use the max-reuse directive for TCP connections served by rings. When used for this reason, the sink TCP connection processors will not reuse a server connection more times than the indicated maximum. This means that connections to the servers will be forcefully removed and re-created, which helps to better distribute the load across available threads, thus increasing performance. Make sure that when using this directive that the connections are not closed more than a couple of times per second.
• In previous versions, some users may have seen intense CPU usage by the pattern LRU cache when performing lookups with low cardinality. To remedy this, in this version the cache will be skipped for maps or expressions with patterns with low cardinality, that is, less than 5 for regular expressions, less than 20 for others. Depending on your setup, you could see a savings of 5-15% CPU in these cases.
• As of this version, configured servers for backends are now properly indexed, which saves time in detecting duplicate servers. As such, the startup time for a configuration with a large number of servers could see a reduction of up to a factor of 4.
• Variables have been moved from a list to a tree, resulting in a 67% global performance gain for a configuration including 100 variables.
• We saw a performance gain of, on average, 7% regarding arithmetic and string expressions by removing the need for trivial casts for samples and converters of the same types.
• The Lua function core.set_map() has doubled its performance in speed by avoiding duplicate lookups.
• This version includes a performance gain regarding smoother reloads for large systems, that is, systems requiring a large number of file descriptors and a large number of threads. This gain is due to how file descriptors are handled on boot, shortening initialization time from 1.6s to 10ms for a setup with 2M configured file descriptors.
• The master-worker mode was heavily reworked in this version to improve stability and maintainability. Its previous architecture model proved difficult in maintaining forward compatibility for seamless upgrades; the rework aims to remedy this problem. Per the new model, the master process does nothing after starting until it confirms the worker is ready, and it no longer re-executes itself to read the configuration, which greatly reduces the number of potential race conditions. The configuration is now buffered once for both the master and worker and as a result will be identical for both. As such, environment variables shared by both will be more consistent, and the worker will be isolated from variables applicable to the master only. This all improves the separation between the processes. An additional improvement is that this rework will reduce file descriptor leaks across the processes as they are now better separated. All of this to say: you should not notice anything as a result of this change except for improved reliability.

Deprecated features and breaking changes

• The program section is deprecated in HAProxy Enterprise 3.1 and will no longer be supported starting HAProxy Enterprise 3.3.
• The configuration options accept-invalid-http-request and accept-invalid-http-response are deprecated. Instead, use accept-unsafe-violations-in-http-request and accept-unsafe-violations-in-http-response.
• Duplicate names in various families of proxies, for example, frontend, listen, backend, defaults, and log-forward sections, and between servers, are detected and reported with a deprecation warning, specifying that the duplicate names will not be supported in HAProxy Enterprise 3.3.
• The legacy C-based mailers are deprecated and will be removed in HAProxy Enterprise 3.3. Set up mailers using Lua mailers instead.

Original source

Key changes in the HAProxy Enterprise 3.3r1 release include:

HAProxy Enterprise WAF

HAProxy Enterprise 3.3r1 WAF introduces support for OWASP CRS version 4. This enhancement is backported to HAProxy Enterprise versions 3.0r1, 3.1r1, and 3.2r1.

Captcha module

Updates to the Captcha module include:

• Breaking change: Frontend configurations must include the new http-response captcha() action. See Actions.
• Breaking change: The filter directives in frontend configurations must append unless METH_POST { path /.well-known/haproxy/captcha_callback }. For example:

filter htmldom mode strict head-append '%[captcha.js(<captcha_section>)]' unless METH_POST { path /.well-known/haproxy/captcha_callback }

• New captcha section directives: callback-token , callback-token-lf , callback-valid-time , cookie-name , and cust-html-file-lf . See Captcha section directives for details.
• New sample fetch: captcha.callback_token(). See Sample fetches for details.

HTTP/3 over QUIC

Updates to HTTP/3 support are:

• An experimental feature lets you use HTTP/3 over QUIC with backend servers.
• The global directives that were prefixed with tune.quic.frontend are deprecated in favor of the directives prefixed with tune.quic.fe. A few other QUIC directives were also deprecated, as the naming is consolidated.
• The no-quic global directive has been renamed tune.quic.listen, which you can set to on or off to enable and disable the QUIC protocol.
• New QUIC-related global directives are available:
  • tune.quic.be.cc.cubic-min-losses: Defines how many lost packets are needed for the Cubic congestion control algorithm to really consider a loss event.
  • tune.quic.be.cc.hystart: Enables or disables the HyStart++ (RFC 9406) algorithm for QUIC connections used as a replacement for the slow start phase of congestion control algorithms, which may cause high packet loss. It’s disabled by default.
  • tune.quic.be.cc.max-frame-loss: Sets the limit for which a single QUIC frame can be marked as lost. If exceeded, the connection is considered as failing and is closed immediately.
  • tune.quic.be.cc.max-win-size: Sets the default, maximum window size for the congestion controller of a single QUIC connection either on the frontend or backend side.
  • tune.quic.be.cc.reorder-ratio: Sets the ratio applied to the packet reordering threshold calculated.
  • tune.quic.be.max-idle-timeout: Sets the QUIC max_idle_timeout transport parameters on either the frontend or backend side.
  • tune.quic.be.sec.glitches-threshold: Sets the threshold for the number of glitches per connection either on the frontend or backend side, where that connection will automatically be killed.
  • tune.quic.be.stream.data-ratio: Allows you to configure the hard limit of the number of data bytes in flight over each stream.
  • tune.quic.be.stream.max-concurrent: Sets the QUIC initial_max_streams_bidi transport parameter either on frontend or backend side.
  • tune.quic.be.stream.rxbuf: Sets the hard limit for the number of data bytes in flight over a QUIC frontend connection.
  • tune.quic.be.tx.pacing: Enables or disables pacing support for QUIC emission. By default, it’s enabled.
  • tune.quic.be.tx.udp-gso: Enables or disables UDP GSO support for QUIC emission. By default, it’s enabled.
  • tune.quic.listen: Disables the QUIC transport protocol on the frontend side.
  • tune.quic.mem.tx-max: Sets the maximum amount of memory usable by the QUIC stack at the transport layer for emission.

Load balancing

The default load balancing algorithm in a backend is now random instead of roundrobin. When benchmarked, the random algorithm performed better and balanced traffic more fairly across servers. This algorithm, unless otherwise set, behaves as a power-of-two algorithm: it randomly selects two servers from the list and chooses the least loaded one.

Statistics

An experimental feature lets you persist statistics after a reload of the load balancer process. It works by using Linux shared memory.

Traces

Version 3.3r1 adds new sources for traces, acme, and ssl, giving insight into TLS-related events.

SSL/TLS

Changes to SSL/TLS in this version include:

• When you configure HAProxy Enterprise to connect to backend servers over TLS, you can send an SNI (Server Name Indication) value. In this version, HAProxy Enterprise sets this value for you automatically to match the value of the Host header it received from the client. This is a common pattern, so making it the default simplifies the configuration.
• A new global directive, ssl-passphrase-cmd, lets you unlock password-protected TLS private keys.
• An experimental feature in which you add ktls on to a bind line enables support for Kernel TLS, which offloads to the Linux kernel the symmetric cryptography part of the TLS processing. For even greater efficiency, you can combine this with splicing by adding option splice-auto to the frontend. With splicing, you transfer data directly from the client socket to the server socket, bypassing the userland process.
• A new, experimental bind argument, ech, enables HAProxy Enterprise to use TLS ECH (Encrypted Client Hello). This new feature encrypts the ClientHello message sent to the load balancer, protecting sensitive fields including the SNI field, so they remain private and only decryptable by the target server. Using this argument requires the global directive expose-experimental-directives. Additionally, ECH requires clients to retrieve the public key from DNS, so first add your public key to your DNS configuration.
• New preprocessor conditions return true or false depending on the TLS library that HAProxy Enterprise uses. With preprocessor conditions, you can enable or disable portions of your configuration depending on the result. The new conditions are:
  • awslc_api_atleast(): True if the current AWS-LC API number is at least as recent as ver, otherwise false.
  • awslc_api_before(): True if the current AWS-LC API number is strictly older than ver, otherwise false.
  • ssllib_name_startswith(): True if the SSL library name HAProxy was linked with starts with name.

OAuth

When using OAuth authentication, the new jwt_verify_cert fetch method will automatically extract the key from the certificate used to sign the JWT, which had previously been a manual step.

Fetch methods

New fetch methods in this version are:

• req.bytes_in: An alias for bytes_in, this returns the number of bytes received from the client.
• req.bytes_out: Returns the number of bytes sent to the server.
• res.bytes_in: An alias for bytes_out, this returns the number of bytes received from the server.
• res.bytes_out: Returns the number of bytes sent to the client.

Converters

New converters in this release are:

• base2: Converts a binary input sample to a binary string containing eight binary digits per input byte.
• le2dec: Converts a little-endian binary input sample to a string containing an unsigned integer number per a given chunk size of input bytes.

Prometheus

Prometheus metrics introduced in this version are:

• haproxy_process_patterns_added_total: Total number of patterns added (ACL and map entries)
• haproxy_process_patterns_freed_total: Total number of patterns freed (ACL and map entries)
• haproxy_frontend_req_bytes_in_total: Total number of request bytes received since process started. Label: proxy=frontend name.
• haproxy_frontend_req_bytes_out_total: Total number of request bytes sent since process started. Label: proxy=frontend name.
• haproxy_frontend_res_bytes_in_total: Total number of response bytes received since process started. Label: proxy=frontend name.
• haproxy_frontend_res_bytes_out_total: Total number of response bytes sent since process started. Label: proxy=frontend name.
• haproxy_backend_req_bytes_in_total: Total number of request bytes received since process started. Label: proxy=backend name.
• haproxy_backend_req_bytes_out_total: Total number of request bytes sent since process started. Label: proxy=backend name.
• haproxy_backend_res_bytes_in_total: Total number of response bytes received since process started. Label: proxy=backend name.
• haproxy_backend_res_bytes_out_total: Total number of response bytes sent since process started. Label: proxy=backend name.
• haproxy_server_private_idle_connections_current: Current number of private idle connections. Labels: proxy=backend name, server=server name.
• haproxy_server_req_bytes_in_total: Total number of request bytes received since process started. Labels: proxy=backend name, server=server name.
• haproxy_server_req_bytes_out_total: Total number of request bytes sent since process started. Labels: proxy=backend name, server=server name.
• haproxy_server_res_bytes_in_total: Total number of response bytes received since process started. Labels: proxy=backend name, server=server name.
• haproxy_server_res_bytes_out_total: Total number of response bytes sent since process started. Labels: proxy=backend name, server=server name.

Original source

Key changes in the HAProxy Enterprise 3.2r1 release include:

Bot Management module

The Bot Management module now has better integration with third-party proxies. Proxies in front of HAProxy Enterprise often remove some client information before relaying the request, which can impact the accuracy of the module’s scoring. For instance, they sometimes remove the client’s source IP address or information about the client’s TLS stack. To solve that, you can configure the proxy to pass that information as HTTP request headers and then pass it to the Bot Management module. Specifically in this version, you can send the client’s TLS stack profile via a JA4 fingerprint.

The Bot Management module can now tell which type of threat was posed by a suspicious bot. When you enable threat detection, the module will analyze the bot’s behavior and attempt to classify it. Classifications include DDoS, login brute forcing, vulnerability scanning, and web scraping.

You can now use the botmgmt-evaluate directive in a defaults section.

Web Application Firewall module

The Web Application Firewall (WAF) has added support for the http-request waf-evaluate action, which allows you to apply WAF Intelligent WAF Engine (IWE) rules on-demand in a frontend, backend, or listen proxy.

The WAF has added support for WAF profiles. Defined in the new waf-profile section, a profile specifies a set of WAF parameters that can be applied to any WAF in the configuration, whether via a filter waf or an http-request waf-evaluate directive. You can define multiple WAF profiles in a configuration. This enhancement will be backported to HAProxy Enterprise 3.1r1.

Captcha module

When using the Captcha module, you can set the on-error directive to allow users to access your application when the Captcha provider’s verification server is unreachable. See the Captcha module reference.

The cookie-path directive, which indicates whether the login cookie applies to the entire site or only a portion of the site, now defaults to /, the entire site.

51Degrees module

The 51Degrees modules makes it easier to set variables from 51Degrees properties with the one-liner, global directive 51d-set-property-vars.

MaxMind module

When using the MaxMind module, you can now specify any unique string as the database type, not only the strings ANONYMOUS, CITY, CONNTYPE, COUNTRY, DOMAIN, ISP, and ANY. This freedom to use any string as the database type will make it easier to write a valid configuration.

OpenID Connect module

The new OpenID Connect module enables single sign-on by integrating with identity providers such as Microsoft EntraID and Okta. Use it to secure access to your applications, while offloading the integration work to HAProxy Enterprise.

SAML module

The SAML module has a new option, idp_ca_file, for validating the identity provider’s digital signature. Also, the list of supported signing algorithms has been updated. Additionally, you can now use the saml-sso action with the http-after-response directive.

Route Health Injection module

The Route Health Injection module has been redesigned to be simpler to configure and to work without the third-party component BIRD. Previous versions had you configure settings across multiple files. The new version places configuration in a single file.

GSLB module

The GSLB module now supports an HTTPS proxy mode for sending health checks over HTTPS. When run alongside HAProxy Enterprise, it sends health checks to HAProxy Enterprise, which will forward them over a secure connection.

The GSLB module can now aggregate many, smaller GSLB templates into one large file that is then converted all at once into the corresponding GDNSD configuration. These options are available in its service environment file (located at /etc/default/hapee-extras-gslb by default).

Global Profiling Engine

The Global Profiling Engine embeds an API that you can use to inspect stick table data and troubleshoot issues. Now, its show table command supports filtering. You can return stick table records that have a matching key, data value, or memory pointer.

AWS-LC TLS library

In HAProxy Enterprise 3.2, we’ve strategically replaced the default OpenSSL library with AWS-LC. AWS-LC is a general-purpose cryptographic library maintained by the AWS Cryptography team, based on code from Google BoringSSL and the OpenSSL project. While this enhancement improves security and performance, it may constitute a breaking change in environments relying on older cipher suites not supported by AWS-LC. For details, see Cipher suite support.

ACME protocol

This version adds experimental support for the ACME protocol, enabling you to obtain and renew TLS certificates from issuers like Let’s Encrypt.

Runtime API

The add server command’s new help argument shows which server arguments it supports for your version of HAProxy Enterprise.

The clear table command has a new syntax that lets you access data that was configured as an array, such as gpc().

The debug counters command has filters for enabling and disabling counters for COUNT_IF statements.

The prompt command now has an interactive mode.

The set table command has a new syntax that lets you access data that was configured as an array, such as gpc().

The show events command accepts the -0 flag to display the lines of the output with terminating null bytes instead of newlines.

The show pools command has a detailed argument that provides more information about the pools.

The show quic command has a new argument, stream, for showing only active streams and excluding connections that are closing or draining.

The show sess command now supports filtering the output to a given backend and server.

The show table command has a new syntax that lets you access data that was configured as an array, such as gpc().

Prometheus

This version adds new Prometheus metrics:

• haproxy_frontend_current_session_rate
• haproxy_frontend_ssl_ocsp_staple
• haproxy_frontend_ssl_failed_ocsp_staple
• haproxy_backend_current_session_rate
• haproxy_backend_ssl_ocsp_staple
• haproxy_backend_ssl_failed_ocsp_staple
• haproxy_server_current_session_rate
• haproxy_server_http_requests_total
• haproxy_server_ssl_ocsp_staple
• haproxy_server_ssl_failed_ocsp_staple

Traffic policing

You can now delay the processing of a request with the pause response policy, giving you another option for slowing down clients that exceed your rate limit.

Health checks

To send health checks to backend servers over existing, idle connections, use the check-reuse-pool argument on a server directive.

DNS resolver

Use the global directive dns-accept-family to indicate whether to request and accept IPv4 and/or IPv6 records from nameservers.

Compression

You can now set a minimum file size to control which files to compress.

HTTP/3 over QUIC

When you set the QUIC congestion control algorithm with the quic-cc-algo directive, it now automatically enables pacing on top of the chosen algorithm. It had been an opt-in, experimental feature before. Pacing smooths the emission of data to reduce network losses and has shown performance increases of approximately 10-20 fold over lossy networks or when communicating with slow clients at the expense of a higher CPU usage in HAProxy. On a related note, you can set the Bottleneck Bandwidth and Round-trip Propagation Time algorithm, which relies on pacing, without enabling experimental features. Set the quic-cc-algo directive’s bbr argument. Or if you don’t want pacing, disable it completely with tune.quic.disable-tx-pacing.

This version improves QUIC upload performance. Previous versions only supported the equivalent of a single buffer in flight, which would limit the upload bandwidth to about 1.4 Mbps per stream, which was slow for users attempting to upload large images or videos. Starting with 3.2r1, uploading streams can use up to 90% (by default) of the memory allocated to the connection, allowing them to use the full bandwidth even with a single stream. You can adjust this ratio by using the global directive tune.quic.frontend.stream-data-ratio, allowing you to prioritize fairness (small values) or throughput (higher values). The default setting should suit common, web scenarios by striking a balance.

A new, global setting, tune.quic.frontend.max-tx-mem, caps the total memory that the QUIC tx buffers can consume, helping to moderate the congestion window so that the sum of the connections don’t allocate more than that. By default, there’s no limitation.

HTTP protocol

The accept-unsafe-violations-in-http-request and accept-unsafe-violations-in-http-response backend directives introduced in in the previous version allow HTTP messages to violate some rules of the protocol. In version 3.2r1, they allow HAProxy Enterprise to accept WebSocket requests that are missing the Sec-WebSocket-Key HTTP header and responses missing the Sec-WebSocket-Accept HTTP header.

You can now set the HTTP response header Content-Length to 0, which some non-compliant applications need with HTTP 101 and 204 responses.

HAProxy Enterprise has become stricter about not permitting some characters in the Authority and Host HTTP headers.

Two new directives let you drop trailers from HTTP requests or responses, which is useful for removing sensitive information that shouldn’t be exposed to clients:

• option http-drop-request-trailers
• option http-drop-response-trailers

To more efficiently close idle HTTP/2 connections, set the idle-ping argument on a bind or server directive.

Syslog load balancing

New options in the log-forward section let you control how the load balancer parses and validates Syslog messages.

SSL/TLS

Use the new ssl-f-use directive to specify the certificates to load in a frontend. This directive simplifies setting TLS-related properties such as ALPN fields, ciphers, signature algorithms, and CAs for client certificate authentication.

Consistent hashing

When using the balance hash algorithm for consistent-hash load balancing, you can now set the directive hash-preserve-affinity to indicate what to do when servers become maxed out or have full queues. Consistent hashing configures the load balancer to maintain server affinity, but when a server is overwhelmed, blindly preserving that affinity can lead to issues. With hash-preserve-affinity, you can now reroute traffic to available servers while still maintaining affinity.

Overload protection

When setting maxconn on a server line, you can set strict-maxconn to apply the limit to TCP connections instead of HTTP requests.

Layer 7 retries

The retry-on directive, which was introduced in version 2.0r1, allows you to set which errors should trigger the retry of a failed request. It now supports the HTTP response status 421 Misdirected Request as a reason.

Fetch methods

This version introduces the following fetch methods:

• bc_reused: Returns true if the transfer was performed via a reused backend connection.
• req.ssl_cipherlist: Returns the binary form of the list of symmetric cipher options supported by the client as reported in the TLS ClientHello.
• req.ssl_keyshare_groups: Returns the binary format of the list of cryptographic parameters for key exchange supported by the client as reported in the TLS ClientHello.
• req.ssl_sigalgs: Returns the binary form of the list of signature algorithms supported by the client as reported in the TLS ClientHello.
• req.ssl_supported_groups: Returns the binary form of the list of groups supported by the client as reported in the TLS ClientHello and used for key exchange, which can include both elliptic and non-elliptic key exchange.
• sc_key(): Returns the key used to match the currently tracked counter.
• table_clr_gpc([,]): Clears the General Purpose Counter at index of the array and returns its previous value.
• table_inc_gpc([,]): Increments the General Purpose Counter at index of the array and returns its new value.
• term_events: Returns a series of comma-separated values that indicate the states of a request as its flowed through the load balancer. Clone the HAProxy GitHub repository, compile the term_events program, then run it to decode the values.

The accept_date and request_date fetch methods now fall back to using the session’s date if not otherwise set, which can happen when logging SSL handshake errors that occur prior to creating a stream.

Converters

It was discovered that a risk of buffer overflow can occur when using the regsub converter to replace patterns multiple times at once (multi-reference) with longer patterns. Although the risk is low, it has been fixed. CVE-2025-32464 was filed. The vulnerability affects all versions, so the fix will be backported.

Lua

The new patref class gives you a way to modify ACL and Map files from your Lua code and is an improvement over the older core.add_acl function. It makes it easier to dynamically change Map and ACL files from your Lua code, such as to build modules that cache responses only for URLs that have a certain URL parameter attached to them. The patref class offers other features too:

• Manipulate both ACL and Map files.
• For Map files, replace the values of matching keys.
• Add new patterns via bulk entry with the patref.add_bulk function.
• Use prepare() and commit() functions to replace the entire ACL file at once with a new set of data.
• Subscribe to events related to manipulating pattern files with callback functions.

A new global directive, tune.lua.bool-sample-conversion, allows you to opt in to proper handling of booleans returned by fetch methods. The default behavior has been that when the Lua code calls a fetch method that returns a boolean, that return value is converted to an integer 0 or 1. Setting the new global directive to normal enables the correct behavior of treating booleans as booleans.

The AppletTCP class’s receive function now accepts a timeout parameter to limit how long it will wait for data from the client. This makes it easier to design services that take in varying lengths of data, such as interactive utilities that read user input, as opposed to expecting fixed-length data.

This version introduces new Lua functions:

• AppletTCP.try_receive: Reads available data from the TCP stream and returns immediately.
• core.wait: Waits for an event to wake the task. It takes an optional delay after which it will awake even if no event fired.
• HTTPMessage.set_body_len: Changes the expected payload length of the HTTP message.

Other performance updates

By fixing the fairness of the lock that the scheduler uses for shared tasks, heavily loaded machines (64 cores NUMA) will see less latency, typically 8x lower, and 300x fewer occurrences of latencies 32ms or above.

HAProxy Enterprise will now interrupt the processing of TCP and HTTP rules in the configuration at every 50 rules, a number that’s configurable, to perform other concurrent tasks. This will help keep latencies low for configurations that have hundreds of rules.

HAProxy Enterprise servers with many CPU cores will see significantly better performance of queues in regards to CPU usage. Queues were refined to be thread group aware, favoring pending requests in the same group when a stream finishes, which reduces data sharing between CPU cores.

Backends with mode http now set option abortonclose by default. This setting tries to stop processing a request before it’s been sent to a server if the client aborted on their end, such as by closing the tab or refreshing. Also, you can now set option abortonclose in a frontend, which wasn’t allowed before, and HAProxy will avoid computing the TLS handshake on connections that are already closed.

HAProxy Enterprise saves resources by no longer allocating memory for a default-server directive unless you declare one in your configuration. Also, after parsing the directive, the load balancer releases the memory associated with it.

When using a use-server directive or track argument in a backend, startup will be faster now that HAProxy Enterprise uses a more efficient algorithm for finding the server.

You can now choose a different TCP congestion algorithm by setting it with the new cc argument on a bind or server line.

This version relaxes the amount of locking between stick tables and peers by batching the updates and delaying work, leading to a smoother traffic flow and better overall performance.

Some multi-threaded tasks that caused a lot of contention on servers with many CPUs, such as stick table expirations and resolvers connections, were changed to be single threaded.

HAProxy Enterprise’s internal HTTP client, which it uses for tasks such as sending requests to ACME servers, had yielded control of the thread in between sending the HTTP headers and sending the HTTP body. That isn’t necessary if HAProxy Enterprise has the headers and body ready to send. In this release, HAProxy Enterprise will send both, if possible.

HAProxy Enterprise will now correctly revert to using the main route for DNS nameservers after an outage caused by a main route to the nameservers being down. Instead of using the connect() and send() functions, HAProxy Enterprise now performs a bind on the wildcard address for the datagram AF_INET* client socket, then uses sendto() instead of send().

Multithreaded applications may experience performance hits when multiple threads modify data, even if unrelated, in the same cache line, which may invalidate the cache line for other threads and cause a cache eviction. Updates to this version include some initial work towards optimizing HAProxy Enterprise’s memory allocation functions such that they align some objects and memory pools along cache line boundaries (64-byte chunks). Allocating memory aligned to the cache line boundaries helps keep data grouped by locality which prevents multiple threads thrashing the cache when the data is unaligned. This results in less contention, less locking, and fewer cache evictions.

In this version, the mechanisms reusing and purging server-side idle connections saw improvements. Connections created when http-reuse is set to never or when using Basic Authentication can now be purged, preventing resource leaks. Some related features, such as deleting a server via the HAProxy Runtime API, will work more reliably, as the servers’ idle connections are now better managed.

All applets, including the DNS, http-client, Lua, logs, peers, and Prometheus applets which were updated with this release, now maintain their own buffers rather than share buffers with the stream, which is used for socket reads and writes. Each applet having its own buffers requires less locking and improves synchronization, which reduces contention across the applets, improving the applets’ scalability.

Security

Updates to security features include:

• You’ll now get a warning at startup if you’re running HAProxy Enterprise as root and your configuration is missing the global directive user, which sets a Linux user account that HAProxy Enterprise should run as.
• The bind and server directives support a new argument named tcp-md5sig that adds support for Protection of BGP Sessions via the TCP MD5 Signature Option (RCF 2385), which many routers require when placing a TCP proxy like HAProxy Enterprise between them.

Usability

Usability improvements in this version include:

• You can get the HAProxy Enterprise version in different formats. Pass the command-line arguments -vq for version, -vqs for the short version, or -vqb for the branch.
• If you’ve set the expose-experimental-directives global directive, but all of the experimental features you were using are no longer experimental, you’ll get a reminder to remove the directive. That should help users in avoiding having experimental features enabled unintentionally.
• The global directive dns-accept-family that was introduced in the previous version now defaults to the value auto. This directive lets you disable IPv4 or IPv6 DNS resolution. A value of auto will enable IPv4 DNS resolution and check for IPv6 connectivity at startup, then again every 30 seconds to determine whether to enable IPv6 resolution.
• Setting the global directive nbthreads to a total number of threads on which HAProxy Enterprise should run, while also declaring a thread-groups directive with a range of threads that exceeds that number, will now emit a warning and the missing threads will be removed. If a thread group is left with no threads at all, it causes a startup error.
• The tune.disable-fast-forward directive is no longer experimental, so you don’t need to set expose-experimental-directives to use it. This directive was introduced in version 2.8r1 and disables data fast-forwarding.
• For debugging, you may want to prevent all workers from being killed when a segfault occurs. You can use the global directive master-worker no-exit-on-failure.
• The default number of reloads defined by mworker-max-reloads is now 50.

Deprecated features

These features are now deprecated, meaning they’ll be removed in a future version:

• The backend directives dispatch and option transparent are deprecated and will emit a warning to replace them if used.
• Global directives prefixed with tune.quic.frontend are deprecated. Use the same directives prefixed with tune.quic.fe instead.
• The master-worker global directive has been deprecated. Use the command-line arguments -W or -Ws instead.

Breaking changes

This version has the following breaking changes:

• The WAF Offloader has been removed from this and future versions.
• The minimum, default Linux kernel version, the one corresponding to the build target linux-glibc, has been updated to 4.17, which is a version older than all of the currently maintained LTS distros. This version was needed to support the new Kernel TLS feature.
• The program configuration section, which allows you to start and run an external program as a child process, was deprecated in version 3.1r1 and is now removed.
• Using the same name for more than one frontend, backend, listen, defaults, or log-forward section is no longer allowed. Duplicated names, which have emitted a warning since version 3.1r1, will now emit an error at startup.
• Using the same name for more than one server in a backend isn’t allowed.
• When configuring email alerts, you must enable the Lua implementation. If you add a mailers configuration section, but forget to load the Lua file, you’ll get a warning.
• The backend directive http-send-name-header, which lets you send the name of the server HAProxy Enterprise is connecting to as an HTTP request header, had always let you decide which HTTP header to use for that purpose. But now, it won’t allow you to choose the headers connection, content-length, host, or transfer-encoding. Overwriting those headers would only cause an invalid request.
• When declaring an ACL, you can set the match type via the -m flag to explicitly compare the input value as a boolean, string, integer, and others. Specifying more than one match type after this flag is no longer allowed. Previously, HAProxy Enterprise had silently used the last match type. Also, HAProxy Enterprise will emit a warning when the match type is ambiguous, such as in path_beg -m reg, which is ambiguous as to whether it matches the beginning of the path or matches paths with a regular expression.
• This version renames the no-quic global directive to tune.quic.listen, which lets you enable or disable the QUIC transport protocol on all frontend listeners.

Original source