LaunchDarkly Release Notes

This topic describes how to use metric measurement windows to prevent bias in warehouse native experiment results.

A metric measurement window defines a fixed, required period of time during which LaunchDarkly collects metrics for a context. A window begins after a context first receives a flag variation in an experiment, and ends after a configured duration.

Attached experiments only consider those metric events that occur within the configured measurement window, and the window must fully complete before the experiment includes any measurements for a context. When you stop an experiment, any metrics for contexts that have not reached the end of their measurement window do not contribute to experiment results.

Restricted to Snowflake data sources

You can only configure metric measurement windows with metrics created from Snowflake data sources, for use with warehouse native Experimentation. You cannot configure windows on metrics created from LaunchDarkly hosted events or other warehouse data sources. To learn more, read Metric event sources.

Avoiding measurement bias in experiments

Metric measurement windows ensure that all contexts included in an experiment have the same amount of time available to generate metric measurements. You define the period of time required to produce a conversion event or to produce the volume of metric values you want to measure. LaunchDarkly ensures that only measurements from completed windows are considered in experiment results. In this way, metric windows help you prevent late or incomplete user activities from biasing experiment results.

When no measurement window is configured (the default behavior), all contexts that participate in an experiment contribute equally to the experiment result. Contexts added near the end of the experiment have less time to generate metrics, so they can negatively impact the experiment results. For example, consider a conversion event that typically requires three days to complete. Most contexts that join the experiment in the last two days of the experiment will negatively impact the conversion metric, even if some of those contexts eventually generate a conversion. By configuring a metric window of three days, you ensure that the experiment only considers units that have the full three days in which to produce the conversion event.

Prerequisites and limitations

You can configure metric measurement windows for any warehouse native metric created from a Snowflake data source. You cannot configure windows on metrics created from LaunchDarkly hosted events or other warehouse data sources. To learn more, read Snowflake native Experimentation.

The data source you use for creating the metric must map the Timestamp value to a column that uses timestamp or datetime format and encodes timezone information. To learn more, read Metric data sources.

By default, new metrics do not include a metric measurement window. When no measurement window is configured, all metric measurements collected during an experiment contribute equally to the experiment result. This matches the LaunchDarkly experiment and guarded release behavior prior to the introduction of metric windows.

Configuring a metric window

When you choose the duration of a metric measurement window, keep in mind:

• The duration you choose should be long enough to capture all relevant measurements. LaunchDarkly stops measuring events for a context after the window completes.
• If your conversion events require a lengthy period of user activity to generate, ensure that experiments run long enough to capture the needed volume of completed metric windows.
• If you are measuring an activity that requires a lengthy setup process, you can optionally configure an offset value to delay measurements until some point after a context first receives a flag evaluation. Offset values enable you to exclude early metric events that might be considered as “noise” in the overall measurement. For example, if you want to measure input validation errors that occur when customers complete a purchase, you could use an offset value to exclude input validation errors that occur earlier in the purchase process.

You configure optional metric window properties after you specify the analysis method for a warehouse native metric.

To configure a measurement window:

1. Open the Data section and navigate to the Metrics list.
2. Click Create metric. The “Create metric” dialog appears.
3. Select Warehouse native from the “Event source” drop-down menu.
4. Select an available Snowflake data source from the “Metric data source” drop-down menu. To learn more, read Metric data sources.
5. Search for or enter an Event key to use for the metric.
6. Choose options in the “Metric definition” section to configure the metric aggregation type and analysis method. To learn more, read Components of a metric.
7. Select the Enable custom measurement window checkbox.
8. Use the “Window measured in” menu to choose whether to define the window in days, hours, or minutes.
9. (Optional) Change the value in the “Start” field to delay metric measurements for a period of time after a context receives a flag variation. A value of zero specifies no offset, meaning the measurement window begins immediately after a context joins the experiment.
10. Enter a value in the “End” field to configure the length of the measurement window.

The end value must be greater than the start value

Both the “End” and “Start” values are calculated relative to when a context first receives a flag variation in an experiment. The “End” value must be greater than the “Start” value to configure a measurement window.

1. Enter a Metric name and and optional Description.
2. Click Create.

Interpreting metrics excluded from experiments

Metrics with measurement windows introduce additional conditions for excluding their measurements from experiment results. For any connected experiment, a context’s metric measurements are excluded if:

• Measurements occur outside of a configured metric window. Any metric events that occur before the window offset, and any events that occur after the window completes, are excluded from the experiment.
• A configured measurement window does not complete. If the experiment ends before a context’s measurement window completes, all measurements for that context are excluded.
• A configured measurement window completes, but the context generated no events. For numeric metrics, if you select the Exclude units that generate no events option, then contexts that complete their measurement window are excluded if they generate no event. To learn more, read Units without events.

LaunchDarkly experiments show the full accounting of units that are excluded from experiment results, both during the experiment and at experiment completion. To view a breakdown of why units are excluded from an experiment, click a value in the “Sample size” column of the experiment results.

To learn more, read Experiment results data.

Original source

This topic describes how to use configure metric winsorization to eliminate extreme values in experiment results.

Restricted to warehouse native metrics

You can configure winsorization only for warehouse native metrics, for use with LaunchDarkly experiments. You cannot configure winsorization for metrics created from LaunchDarkly hosted events. To learn more, read Metric event sources.

Limiting the impact of extreme values on experiments

Winsorization is a statistical technique that replaces outlying values with values at a configured percentile. Using metric winsorization helps you limit the impact of extreme values in LaunchDarkly experiment results without introducing bias by selectively removing values.

For example, consider a metric that generates the following (sorted) latency values for a single randomization unit during the course of an experiment:
5, 120, 135, 140, 145, 150, 155, 160, 170, 950

The value 950 is an extreme outlier in this sample of data, and would significantly affect the mean latency value. The P90 value for the data is 170, so configuring winsorization for the metric at upper bound P90 percentile replaces all values higher than 170 with the value 170. This yields the modified data set:
5, 120, 135, 140, 145, 150, 155, 160, 170, 170

LaunchDarkly supports configuring one-sided or two-sided winsorization as needed to limit the impact of extremely high values, extremely low values, or both. For example, winsorizing the example metric at both the lower bound P10 percentile and the upper bound P90 percentile mitigates the lower outlying 5 value, yielding the modified data set:
120, 120, 135, 140, 145, 150, 155, 160, 170, 170

Common use cases

Winsorization is most commonly used with metrics that produce a long tail distribution of values. This generally corresponds to metrics that measure revenue, latency, or session duration. Using winsorization at the upper bound for P90 or higher percentiles limits the impact of extremely high values that would negatively skew experiment results.

How LaunchDarkly computes winsorization percentiles

LaunchDarkly computes the percentile values for winsorization using all metrics collected for a randomization unit across all arms of a warehouse native experiment. For metrics that use a metrics measurement window configuration, LaunchDarkly uses only those values collected within the configured window to determine the percentile values. If no window is configured, the measurement duration corresponds to the length of the experiment itself.

If you choose Include units and set the value to 0 for a numeric metric, LaunchDarkly does not include assigned zero values when it computes winsorization percentiles. To learn more, read Units without events.

Prerequisites and limitations

You can configure metric measurement winsorization for any warehouse native metric. You cannot configure winsorization for metrics created from LaunchDarkly hosted events.

Configuring winsorization

You configure optional winsorization properties after you specify the analysis method for a warehouse native metric.

To configure winsorization:

1. Open the Data section and navigate to the Metrics list.
2. Click Create metric. The “Create metric” dialog appears.
3. Select Warehouse native from the “Event source” menu.
4. Select an available data source from the “Metric data source” menu, or create a new data source. To learn more, read Metric data sources.
5. Search for or enter an Event key to use for the metric.
6. Choose the metric aggregation from the “Metric definition” section. The window populates a full metric definition using default values.
7. Change options in the “Metric definition” drop-down menus as needed to change the analysis units or other metric analysis options. To learn more, read Components of a metric.
8. (Optional) Choose Enable custom measurement window if you want to configure a metric measurement window. To learn more, read Metric measurement window.
9. Choose Enable winsorization.
10. Enter percentile values in the Lower bound and Upper bound fields as needed to specify the percentile value(s) used to winsorize extreme values. If you are configuring two-sided winsorization, the Upper bound value must be greater than the Lower bound value.

Set Lower bound to zero, or Upper bound to 100, to disable winsorization for that bound.

1. Enter a Metric name and and optional Description.
2. Click Create.

Original source

This topic explains how to create LaunchDarkly ratio metrics, which measure the ratio of two separate aggregations from Snowflake warehouse data sources.

About ratio metrics

A ratio metric is a complex metric type that computes the ratio of two separate metric aggregations: one for the numerator component and one for the denominator component. Each aggregation component is a simple metric created from a warehouse data source. LaunchDarkly divides the total of the numerator aggregation by the total of the denominator aggregation to compute the final metric value.

Configuring numerator and denominator aggregations for a ratio metric.

A ratio metric helps you determine how flag variations affect the relationship between two measurements. You can use ratio metrics to address the following use cases, which simple metrics do not support:

• Clustered analysis for experiments: Ratio metrics let you measure events that are grouped by an analysis unit that is finer-grained than the randomization unit chosen for the experiment. For example, you could measure the number of session errors generated relative to the total number of prompts that users created. In this example, you randomize the experiment by user contexts and configure a user aggregation for the denominator component, then configure a session aggregation for the numerator.
• Conversion metrics normalized by a separate event: Simple metrics can measure conversion events normalized by users who participate in the experiment. Ratio metrics can measure conversion events normalized by users that generate some other event, configured in the denominator aggregation. For example, you could measure the number of users who clicked on a search result, relative to the number of users who initiated a search.

You can add ratio metrics only to LaunchDarkly experiments that use the frequentist statistical methodology.

LaunchDarkly uses the computed value of the ratio to compare the performance of flag evaluations during an experiment. Because a ratio metric can track two separate analysis units, LaunchDarkly uses the statistical delta method to account for covariance of the two units. This helps to ensure that experiment recommendations remain accurate, even when analysis units in the denominator perform differently for a given experiment arm. To learn more, read Statistical methodology for frequentist experiments.

Aggregation types for ratio metrics

You configure each aggregation in a ratio metric separately. The numerator and denominator can use the same or different aggregation types, and can even use data from different warehouse data sources. However, the selected data sources must include context key mappings for one or more shared context kinds.

For example, if the data source for the numerator contains context key mappings for the “account,” “user,” and “session” context kinds but the denominator includes mappings for “session” and “device” contexts, then the ratio metric can only use “session” as the analysis unit. To learn more about mapping context keys in a warehouse data source, read Create data sources.

Ratio metrics support the same aggregation types available for simple metrics, plus one additional aggregation type, count_distinct, that counts distinct values from a warehouse column you provide. The available aggregation types are:

• Count uses sum aggregation to measure the total number of times a context generates the metric event (conversion metric).
• Sum uses sum aggregation to measure the total of the numerical values provided with a context’s metric events (numeric metric).
• Average uses average aggregation to measure the average numerical value provided with a context’s metric events (numeric metric).
• Count distinct uses count_distinct aggregation to measure the total number of times a unique value appears in a warehouse column associated with a context’s metric events (numeric metric). This option ignores duplicate or null values in the column.

The count_distinct aggregation type is available only for ratio metrics. It provides the flexibility to measure a distinct signal independently of the metric event count or event value. When you choose Count distinct, you provide the name of a warehouse column associated with the metric event to use for measuring distinct values.

For all ratio metric aggregations that correspond to a numeric metric, LaunchDarkly uses imputation to assign a zero value to any units that do not produce a metric event. To learn more, read Units without events.

Prerequisites and limitations

You can create ratio metrics only from Snowflake warehouse native metrics, for use with warehouse native Experimentation. Configure your Snowflake warehouse integration and create a data source before you create new warehouse native metrics. To learn more, read Metric data sources.

You cannot create ratio metrics from LaunchDarkly-hosted metric events, OpenTelemetry traces, or from warehouse data sources other than Snowflake.

You can add ratio metrics only to experiments that use the frequentist statistical methodology. To learn more, read Analyzing experiments.

Create a ratio metric

To create a ratio metric:

1. Open the Data section and navigate to the Metrics list.
2. Click Create metric. The “Create metric” dialog appears.
3. Click Ratio in the “Select metric structure” section. The “Select metric structure” section appears only if you have configured warehouse native Experimentation and a Snowflake warehouse data source.
4. Configure metric options for the “Numerator” component:
   i. Choose an existing data source from the Select metric data source menu. The data sources you choose for the numerator and denominator components must have at least one context key mapping to the same context kind.
   ii. Enter the Event key to use for the numerator component of the metric.
   iii. Choose an aggregation type from the Aggregate as menu:
   - Count uses sum aggregation to measure the total number of times a context generates the metric event (conversion metric).
   - Sum uses sum aggregation to measure the total of the numerical values provided with a context’s metric events (numeric metric).
   - Average uses average aggregation to measure the average numerical value provided with a context’s metric events (numeric metric).
   - Count distinct uses count_distinct aggregation to measure the total number of times a unique value appears in a column associated with a context’s metric events (numeric metric). This option ignores duplicate or null values in the column.
   iv. If you chose Count distinct, enter a column name to use for the measurement in the Count distinct on field.
   v. (Optional) Click + Add winsorization or + Add custom measurement window as needed to configure the aggregation. To learn more, read Metric winsorization or Metric measurement window.
5. Repeat the previous step to configure an aggregation for the “Denominator” component of the metric. The aggregation type or warehouse column does not need to match the configuration of the numerator. However, the data sources you choose for the numerator and denominator must have at least one shared context key mapping.

Both metric windows must complete

If you configure a metric window for the numerator and denominator, windows for both components must complete before LaunchDarkly includes the ratio metric result in an experiment. To learn more, read Interpreting metrics excluded from experiments.

After you configure the numerator and denominator, the window displays a complete metric definition using the values you selected.

1. Use the Analysis unit menu to select the analysis units to use for the metric. The same analysis units apply to both the numerator and denominator components of the ratio metric.
2. Choose higher is better or lower is better to define the success criteria for the computed ratio.
3. Enter a metric Name.
4. (Optional) Add a Description.
5. (Optional) Add any Tags.
6. (Optional) Update the Maintainer.
7. Click Create.

You can also use the REST API: Create metric.

Original source

This topic explains how to create LaunchDarkly custom metrics from OpenTelemetry traces.

This feature is for Early Access Program customers only

Creating metrics from trace spans is only available to members of LaunchDarkly’s Early Access Program (EAP). To request access to this feature, contact your LaunchDarkly account manager.

LaunchDarkly lets you create custom metrics from OpenTelemetry traces instrumented in your code, for use with guarded rollouts.

Creating metrics from traces lets you use existing OpenTelemetry instrumentation to measure how application behavior changes during a guarded rollout, without re-instrumenting those behaviors as metric events using the track() method. You define a query that selects the trace spans you want to use to measure your application, and create a metric based on the query result. Each matching span observed during a guarded rollout indicates a metric event.

To create numeric metrics from traces, such as for measuring regressions, you specify the span attribute name that provides the value you want to monitor.

Trace metrics use the same aggregation and analysis methods as LaunchDarkly metrics created from metric events. To learn more, read Components of a metric.

Prerequisites and limitations

You can create custom metrics from trace spans that include a LaunchDarkly feature flag span event, indicated by the feature_flag.set.id attribute. The LaunchDarkly observability SDK plugins automatically include feature flag span events when you create trace spans.

If your application uses a different OpenTelemetry library or package, you can configure the OpenTelemetry collector or use tracing hooks to add the required feature flag span events. To learn more, read Server-side SDKs.

Trace metrics are compatible with guarded rollouts for managing feature releases. You cannot use trace metrics with LaunchDarkly experiments.

Creating trace metrics

You can create trace metrics using either the LaunchDarkly “Create metric” dialog or the observability “Traces” page. Both methods let you create a span query to select the spans you want to monitor in the metric.

Use the “Create metric” dialog if you are familiar with creating metrics and filtering metric events and you want to use a similar interface to create metrics from spans. Use the “Traces” page if you are familiar with the observability search syntax or you want a real-time preview of your filter query results.

Span query syntax differences

The “Create metric” dialog uses a graphical query builder to help you filter trace spans, while the “Traces” page uses the observability query search specification.

Both interfaces can create equivalent span filters, but they use different syntax for some conditions. The “Create metric” dialog uses the present token to indicate whether an attribute is present in a span. It does not support the EXISTS operator or its negation, NOT EXISTS.

For example, if you use the “Traces” page to create a search that reads service_version EXISTS, the equivalent filter in the “Create metric” dialog is service_version is present.

Using the Create metric interface

To use the “Create metric” dialog to create a new trace metric:

1. Navigate to the Metrics list.

2. Click Create metric. The “Create metric” dialog appears.

3. Select Traces from the “Event source” drop-down.

4. Use the “Filter to spans” field to create a query that selects the trace spans to use for the new metric.

   The “Filter to spans” field uses these basic controls:

   • Click an empty spot in the field to display the span attributes, operators, or relational functions you can add to the query at that spot.
   • Type characters to filter the available selections you can add. Enter or click a name in the drop-down list to add the value.
   • Use checkboxes in the drop-down list to select multiple values, or use commas to separate multiple, typed values.
   • Click an existing attribute name, value, or operator to change or edit the displayed value.
   • Click the x icon next to a name, value, or operator to delete it.

   Span functions (Any span, Any span 2, Any span 3) can only be added to the top level of the filter query, and cannot be nested within parentheses. To learn more, read Span functions for trace search.

5. (Optional.) Click View traces to view recent spans that match your filter query. This opens a new tab to the Traces page with your filter query applied.

6. In the “Metric definition” section, choose the aggregation type for your metric. The window populates a full metric definition using default values.

   • Use Count or Count distinct units (Percent) to measure matching trace spans during a guarded rollout.
   • Use Sum or Average to create a numeric metric that aggregates a value from the filtered trace spans.

7. If you are configuring a numeric metric:
   i. Click the “Choose a numeric span attribute” field and type or select the span attribute that provides the numeric value you want to measure.
   ii. Select Include units and set the value to 0 or Exclude units that generate no events to choose how to handle units that do not generate a matching span during a rollout.
   iii. Enter a Unit of measure to describe the value you are measuring.

8. Use the “Metric definition” section to configure the metric’s analysis method and success criteria. To learn more, read Analysis method.

9. Enter a Metric Name.

10. (Optional) Add a Description.

11. (Optional) Add any Tags.

12. (Optional) Update the Maintainer.

13. Click Create.

Using the Traces interface

To use the “Traces” page to create a new trace metric:

1. Navigate to the Traces page.
2. Use the Search… field to create a query that selects the trace spans to use for the new metric. Use feature_flag.set.id EXISTS to limit results to trace spans that have the required feature flag span event. To learn more, read Search specification.
3. Review the filter query results in the “Spans” list.
4. Select Create metric from the three-dot menu next to your query. This opens the Create metric dialog and populates it with the query you created.
5. In the “Metric definition” section, choose the aggregation type for your metric. The window populates a full metric definition using default values.
   • Use Count or Count distinct units (Percent) to measure matching trace spans during a guarded rollout.
   • Use Sum or Average to create a numeric metric that aggregates a value from the filtered trace spans.
6. If you are configuring a numeric metric:
   i. Click the “Choose a numeric span attribute” field and type or select the span attribute that provides the numeric value you want to measure.
   ii. Select Include units and set the value to 0 or Exclude units that generate no events to choose how to handle units that do not generate a matching span during a rollout.
   iii. Enter a Unit of measure to describe the value you are measuring.
7. Use the “Metric definition” section to change the metric’s analysis method, analysis units, and success criteria as needed. To learn more, read Analysis method.
8. Enter a Metric Name.
9. (Optional) Add a Description.
10. (Optional) Add any Tags.
11. (Optional) Update the Maintainer.
12. Click Create.

Adding trace metrics to guarded rollouts

You select trace metrics for use with guarded rollouts in the same way you select other LaunchDarkly metrics. To learn more, read Creating guarded rollouts.

Original source

OpenTelemetry in server-side SDKs

This topic explains how to enable OpenTelemetry (OTel) in server-side SDKs, and how to display and use OpenTelemetry data in LaunchDarkly.

Looking for information on OpenTelemetry in client-side SDKs?

For information about using OpenTelemetry in LaunchDarkly client-side SDKs, read OpenTelemetry in client-side SDKs.

About OpenTelemetry

OpenTelemetry is an open source observability framework and toolkit designed to create and manage telemetry data such as traces, metrics, and logs. Because OpenTelemetry is vendor- and tool-agnostic, you can reuse OpenTelemetry instrumentation that already exists in your code to send OpenTelemetry data to LaunchDarkly.

We recommend enabling OpenTelemetry in LaunchDarkly server-side SDKs if you use the Observability, Experimentation, or Guarded rollouts features, or if you use third-party observability tools that support the OpenTelemetry framework.

If you want to implement OpenTelemetry features in new code, we recommend using the LaunchDarkly observability plugins to automate the process of adding flag evaluation metadata and forwarding telemetry data directly to LaunchDarkly endpoints. LaunchDarkly provides observability plugins for many client and server SDKs. To learn more, read Observability SDKs.

Using OpenTelemetry data in LaunchDarkly

You can use OpenTelemetry data in LaunchDarkly in two ways:

1. OpenTelemetry data ingestion: LaunchDarkly provides OpenTelemetry protocol (OTLP) HTTP and gRPC endpoints to support ingesting traces, metrics, and logs from your applications and from third-party application integrations. LaunchDarkly observability features help you search and visualize ingested OTel data, or configure alerts to respond to OTel signals. To learn about configuring OTel ingestion, read Observability SDKs and Observability Integrations. To learn about viewing ingested OTel data, read Observability.
2. LaunchDarkly metrics: LaunchDarkly automatically generates metrics from certain OpenTelemetry traces, and these metrics are available in the Experimentation and guarded rollouts features. When ingested trace data contains feature flag evaluation span events along with certain HTTP span attributes or error span events, LaunchDarkly correlates the data and creates metrics to use with guarded rollouts. To learn more, read OpenTelemetry autogenerated metrics.

You can also use any OTel traces that include feature flag span events to create custom LaunchDarkly metrics for use with experiments or guarded rollouts. To learn more, read Creating metrics from traces.

Sending OpenTelemetry data to LaunchDarkly

To configure and send OpenTelemetry data to LaunchDarkly when you are using LaunchDarkly server-side SDKs:

1. (Optional) Add flag evaluation information to OpenTelemetry spans in your application. To learn how, read the section for your SDK under Server-side SDKs, below.
   • This step is optional but highly recommended, as it means that flag-specific details are available when you review your OTel data in the LaunchDarkly UI.
   • You might choose to skip this step, at least temporarily, if you have already instrumented OpenTelemetry in your application and only want to send your existing OTel data to LaunchDarkly.
   • If you create spans using a LaunchDarkly server-side observability plugin, the SDK adds the required flag evaluation information automatically.
2. Configure your application or OpenTelemetry collector to use the LaunchDarkly OTel endpoints:
   • For HTTP, use https://otel.observability.app.launchdarkly.com:4318 or https://otel.observability.app.launchdarkly.com:443
   • For gRPC, use https://otel.observability.app.launchdarkly.com:4317.
   • If you use a LaunchDarkly server-side observability plugin, the SDK automatically sends telemetry to the default endpoints.
   • You may need to update your infrastructure to make sure your application can reach this domain. To learn more, read Accessing LaunchDarkly by domain.
3. (Optional) Include an active LaunchDarkly SDK key as a resource attribute in your telemetry data.
   • This step is required if you are using OpenTelemetry SDKs. To learn how, read Setting resource attributes, below.
   • This step is not required if you are using LaunchDarkly server-side SDKs. If you use LaunchDarkly SDKs, this is set automatically.
4. Send the OpenTelemetry data to LaunchDarkly. LaunchDarkly supports traces, metrics, and logs. Only traces data generates LaunchDarkly metrics. You can view metrics and logs in the LaunchDarkly UI when you use the Observability features.
   • You can use the OpenTelemetry Collector to pre-filter and aggregate this data. To learn how, read Configuring the collector, below.
   • Alternatively, you can send data directly from your application using OpenTelemetry SDKs.
   • LaunchDarkly server-side observability plugins automatically send telemetry to the default LaunchDarkly endpoints.

Setting resource attributes

[Expandable section]

Configuring the collector

[Expandable section]

Automatic attribute normalization

OpenTelemetry semantic conventions define standard attribute names for common concepts like HTTP requests, database operations, and messaging systems. These conventions evolve over time, and attribute names are occasionally renamed or restructured. For example, http.method was renamed to http.request.method and http.status_code was renamed to http.response.status_code. For more details on how attribute names change across OpenTelemetry versions, see the semantic conventions changelog.

LaunchDarkly automatically normalizes deprecated OpenTelemetry semantic convention attributes to their current equivalents when it ingests your telemetry data. This means you can send data using older attribute names, and LaunchDarkly will create the corresponding current attribute names for you. This normalization applies to all OpenTelemetry data LaunchDarkly ingests, whether from LaunchDarkly SDKs with tracing hooks enabled or from external OTLP endpoints.

This automatic normalization provides several benefits. LaunchDarkly ensures consistent attribute names across your telemetry data, even when services use different OpenTelemetry versions. You can search and filter using the current attribute names regardless of which version your services emit. You can also upgrade your instrumentation gradually without losing the ability to correlate data across services.

When viewing trace attributes in LaunchDarkly, the UI displays indicators showing which attributes have been remapped from deprecated conventions.

Server-side SDKs

LaunchDarkly processes metrics, logs, and traces from your OpenTelemetry data:

• LaunchDarkly processes OpenTelemetry metrics and logs and displays them in the LaunchDarkly UI under Observability
• LaunchDarkly processes two types of data from OpenTelemetry traces:
  • HTTP span attributes, including latency, 5xx occurrences, and other errors, where the span has or overlaps with another span that has at least one feature flag span event. This includes nested spans.
  • Exception span events that occur after a feature flag span event on the same trace. If the exception occurs before the feature flag event, LaunchDarkly does not capture it.

A feature flag span event is defined as any span event that contains a feature_flag.context.key attribute. LaunchDarkly ignores traces that do not include span events with this attribute.

In most server-side SDKs, the OTel traces are specific to the LaunchDarkly project and environment that you specify in the collector configuration, described above. In the .NET (server-side) and Node.js (server-side) SDKs, you can provide the LaunchDarkly client-side ID in the tracing hook. This enables you to send traces for multiple projects and environments using one collector.

The following sections describe, for each supported SDK, how to ensure your spans have compatible feature flag events.

Sending feature flag event data requires feature flag evaluation

The OpenTelemetry tracing hook automatically attaches feature flag event data to your OTel traces for you. Feature flag event data, including the feature_flag.context.key attribute, is only generated when your application uses the LaunchDarkly SDK to evaluate a feature flag.

This feature is available in the following SDKs:

• .NET (server-side)
• Go
• Java
• Node.js (server-side)
• PHP
• Python
• Ruby

.NET (server-side)

Use the .NET (server-side) SDK observability plugin for new applications

We recommend you use the .NET (server-side) SDK observability plugin for new application development or OpenTelemetry instrumentation. The observability plugin provides additional configuration options and automatically adds LaunchDarkly environment and flag attributes to OpenTelemetry spans.

The instructions that follow are provided for reference, for OpenTelemetry applications that were developed before the observability plugin was available.

[Expandable .NET (server-side) code sample]

Go

Use the Go SDK observability plugin for new applications

We recommend you use the Go SDK observability plugin for new application development or OpenTelemetry instrumentation. The observability plugin provides additional configuration options and automatically adds LaunchDarkly environment and flag attributes to OpenTelemetry spans.

The instructions that follow are provided for reference, for OpenTelemetry applications that were developed before the observability plugin was available.

[Expandable Go code sample]

Java

[Expandable Java code sample]

Node.js (server-side)

Use the Node.js (server-side) observability plugin for new applications

We recommend you use the Node.js (server-side) SDK observability plugin for new application development or OpenTelemetry instrumentation. The observability plugin provides additional configuration options and automatically adds LaunchDarkly environment and flag attributes to OpenTelemetry spans.

The instructions that follow are provided for reference, for OpenTelemetry applications that were developed before the observability plugin was available.

[Expandable Node.js (server-side) code sample]

PHP

[Expandable PHP code sample]

Python

[Expandable Python code sample]

Ruby

[Expandable Ruby code sample]

Original source

Analyze your experiments on the same trusted data your business already runs on, so results never come with an asterisk.

Your warehouse is the source of truth for your decisions. A year ago, we made it the source of truth for your experiments, too, bringing warehouse-native experimentation to Snowflake so teams could analyze experiments directly on the data they already trust, with no copies and no second version of the truth.

Since then, adoption has grown steadily, and we've learned a lot from teams running real experiments against their own warehouse data. Today, we're putting those lessons to work, with updates on two fronts:

• Wherever your data lives. Warehouse-native experimentation now runs on BigQuery, Databricks, and Redshift, alongside Snowflake, so you can run it on the warehouse you already use.
• Whatever your analysis demands. It now includes advanced statistical capabilities that were previously available only in hosted experimentation.

Wherever your data lives

Whichever warehouse your organization relies on, you can now experiment directly on your own data. With BigQuery, Databricks, and Redshift joining Snowflake, warehouse native experimentation gives you the same trusted experience, while keeping your sensitive data in your warehouse. LaunchDarkly only receives aggregated, de-identified experiment results to power reporting in our product.

And the workflow stays the same, no matter which warehouse you rely on:

1. LaunchDarkly syncs experiment exposure data into the warehouse via Data Export.
2. Metrics are defined from metric sources, which draw on the tables in your warehouse, and are computed directly against them.
3. Results surface back in LaunchDarkly for analysis and decision-making.

You define your metrics in LaunchDarkly, and they compute against the same governed data your team already trusts. No reconciling, no second version of the truth, and no asterisk on your results.

Whatever your analysis demands

Trusted data is only half of it. You also need the statistical rigor to act on results with confidence. Over the last few months, we've brought the depth of hosted experimentation (where LaunchDarkly stores your data and computes results on our own infrastructure) directly to your warehouse. These are a few of the capabilities we've shipped:

• Sequential Testing: Call experiments the moment they're conclusive, minimizing the false positives that come from peeking early.
• Multiple Comparisons Correction: Test many metrics and variations at once while keeping your false-positive risk under control, even as the comparisons add up.
• Adding metrics post-experiment start: Add metrics on the fly and see results immediately, without committing to a fixed set of metrics upfront.
• Result Segmentation: See how different user segments respond to your hypothesis, not just the aggregate.

Snowflake, our longest-running integration, goes a step further with metric winsorization and windowing for even finer control over how outliers and measurement windows shape your results. We'll also be rolling these features out to other warehouse integrations soon, and going forward, we're aiming to bring new capabilities to every supported warehouse at the same time.

For product teams, this means faster experimentation cycles. For data teams, metrics stay governed in the systems they already manage.

Get started

Warehouse-native experimentation is available today on BigQuery, Databricks, Redshift, and Snowflake. Set it up on the warehouse you already use:

• BigQuery
• Databricks
• Redshift
• Snowflake

New to warehouse-native experimentation? Request a demo and we'll walk you through running your first experiment on your own data.

Original source

LaunchDarkly AI SDK (server-side) for .NET

This AI SDK is in pre-release and not subject to backwards compatibility guarantees. The API may change based on feedback.

Pin to a specific minor version and review the changelog before upgrading.

The LaunchDarkly AI SDK (server-side) for .NET is designed primarily for use in multi-user systems such as web servers and applications. It follows the server-side LaunchDarkly model for multi-user contexts. It is not intended for use in desktop and embedded systems applications.

Currently there is no client-side AI SDK for .NET. If you're interested, please let us know by filing an issue!

LaunchDarkly overview

LaunchDarkly is a feature management platform that serves trillions of feature flags daily to help teams build better software, faster. Get started using LaunchDarkly today!

Supported .NET versions

This version of the AI SDK is built for the following targets:

• .NET 8.0: runs on .NET 8.0 and above (including higher major versions).
• .NET Framework 4.6.2: runs on .NET Framework 4.6.2 and above.
• .NET Standard 2.0: runs in any project that is targeted to .NET Standard 2.x rather than to a specific runtime platform.

The .NET build tools should automatically load the most appropriate build of the SDK for whatever platform your application or library is targeted to.

Getting started

Refer to the SDK documentation for instructions on getting started with using the SDK.

Signing

The published version of this assembly is digitally signed with Authenticode and strong-named. Building the code locally in the default Debug configuration does not use strong-naming and does not require a key file. The public key file is in this repository at LaunchDarkly.pk as well as here:

Public Key:
0024000004800000940000000602000000240000525341310004000001000100f121bbf427e4d7edc64131a9efeefd20978dc58c285aa6f548a4282fc6d871fbebeacc13160e88566f427497b62556bf7ff01017b0f7c9de36869cc681b236bc0df0c85927ac8a439ecb7a6a07ae4111034e03042c4b1569ebc6d3ed945878cca97e1592f864ba7cc81a56b8668a6d7bbe6e44c1279db088b0fdcc3552f746b4

Public Key Token: f86add69004e6885

Learn more

Read our documentation for in-depth instructions on configuring and using LaunchDarkly. You can also head straight to the complete reference guide for this SDK.

The authoritative description of all types, properties, and methods is in the generated API documentation.

Contributing

We encourage pull requests and other contributions from the community. Check out our contributing guidelines for instructions on how to contribute to this SDK.

Verifying build provenance with the SLSA framework

LaunchDarkly uses the SLSA framework (Supply-chain Levels for Software Artifacts) to help developers make their supply chain more secure by ensuring the authenticity and build integrity of our published packages. To learn more, see the provenance guide.

About LaunchDarkly

• LaunchDarkly is a continuous delivery platform that provides feature flags as a service and allows developers to iterate quickly and safely. We allow you to easily flag your features and manage them from the LaunchDarkly dashboard. With LaunchDarkly, you can:
  • Roll out a new feature to a subset of your users (like a group of users who opt-in to a beta tester group), gathering feedback and bug reports from real-world use cases.
  • Gradually roll out a feature to an increasing percentage of users, and track the effect that the feature has on key metrics (for instance, how likely is a user to complete a purchase if they have feature A versus feature B?).
  • Turn off a feature that you realize is causing performance problems in production, without needing to re-deploy, or even restart the application with a changed configuration file.
  • Grant access to certain features based on user attributes, like payment plan (eg: users on the ‘gold’ plan get access to more features than users in the ‘silver’ plan). Disable parts of your application to facilitate maintenance, without taking everything offline.
• LaunchDarkly provides feature flag SDKs for a wide variety of languages and technologies. Read our documentation for a complete list.
• Explore LaunchDarkly
  • launchdarkly.com for more information
  • docs.launchdarkly.com for our documentation and SDK reference guides
  • apidocs.launchdarkly.com for our API documentation
  • blog.launchdarkly.com for the latest product updates

Original source

This topic explains how to use prompt snippets to create, manage, and reuse prompts across AgentControl config variations.

Prompt snippets are reusable, versioned pieces of prompt text that you manage as standalone resources in LaunchDarkly. Snippets let you define prompt content one time and reuse it across configs and variations.

Create and manage snippets in the Library, and reference them inside variation messages to reuse prompt content. The centralized snippets library helps you maintain consistency across config variations and quickly find a snippet you wish to use again. Snippet versioning lets you update prompt content without losing earlier versions of each snippet.

Use prompt snippets to:

• Reduce duplicated prompts across config variations
• Define and reuse prompt content such as tone, formatting, or governance language
• Identify which configs and variations use a specific snippet
• Track updates to prompt content by version
• Maintain prompt consistency across teams

How prompt snippets work

Each snippet has a name, key, and prompt content, along with versioning information and metadata such as tags, description, and maintainer. You reference snippets using their key and version. When LaunchDarkly evaluates a variation, it resolves these references and returns the fully assembled prompt.

These fields determine how you reference, update, and organize snippets:

• The key identifies the snippet and lets you reference it in variation messages
• The version lets you update prompt content without affecting existing variations
• Metadata helps you organize snippets across your project

Snippets are included as part of a config variation’s message body. When LaunchDarkly evaluates a variation, it replaces the reference with the snippet content and returns the fully assembled prompt to your application. A single message can include multiple snippet references along with other prompt content.

Core functions of prompt snippets

Prompt snippets’ core functions include:

• Editing a snippet creates a new version instead of modifying the existing one
• Config variations reference a specific snippet version
• Updating a snippet does not automatically update variations
• Snippets cannot reference other snippets

These behaviors let you update prompt content safely, review changes before applying them, and keep existing variations stable until you choose to update them.

You manage prompt snippets from the Library under AI. The snippets library provides a centralized place to create, view, and update reusable prompts across your project.

View snippets

Use the snippets list in the library to browse, search, and manage snippets across your project.

The snippets list provides a centralized view of all prompt snippets, so you can review existing content, identify reusable snippets, and understand how snippets are organized.

From the snippets list, you can:

• View all snippets in your project in a centralized list
• Search for snippets by name, key, or content
• View snippet details, including versions and usage

Create a snippet

Create a snippet to define reusable prompt content that you can reference across config variations. Use snippets for shared content such as tone, formatting, or common instructions.

To create a snippet from the Snippets tab:

1. Navigate to your project.
2. In the left sidebar, click Agents. The AgentControl menu appears.
3. Click Library.
4. Select the Snippets tab. The snippets list appears.
5. Click New snippet. The “Save snippet” dialog appears.
6. Enter a name for the snippet. The snippet’s key auto-populates based on the name.
7. (Optional) Add a description to help others understand when to use this snippet.
8. In the “Body” field, enter the prompt text you want to save as a snippet.
9. (Optional) Click the tag icon to add tags to organize snippets across your project.
10. Click Create.

New snippets appear in the library’s “Snippets” tab.

You can also create a snippet inline while editing a variation message. To create a snippet this way, read Use snippets in config variations.

Snippet versioning

New snippets start at version 1. When you edit a snippet, LaunchDarkly creates a new version of that snippet instead of modifying the existing one. Previous versions remain available, so you can refer back to them or use them again if needed.

To edit a snippet, click the snippet in the snippets list. This opens the editing page, where you can update the snippet’s name, prompt content, and metadata.

You must manually update config variations to use new snippet versions.

Config variations continue to use the version of a snippet you assigned until you update them to use a different version. For example, if a config variation uses version 1 of a snippet and you update the snippet, creating version 2, the config will continue to use version 1 until you specify otherwise.

Manage snippet usage and versions to understand where snippets are used and control how updates are applied across config variations.

Each snippet includes version history and usage information. Use this information to review how a snippet is used before updating variations to a new version. From the snippet list, you can view all config and variations that reference the snippet and see which version each variation is using. This helps you understand where a snippet is in use before making changes.

From a config variation, you can compare the current snippet version to the latest version and update the variation to use a newer version. Use this workflow to review how changes affect variations and update them individually as needed. This approach lets you apply changes intentionally and keep existing variations stable until you update them.

Compare snippet versions

You can view and compare versions of a snippet from the snippets list. Here’s how:

1. From the snippets list, find the snippet with versions you wish to view.
2. Click the version number in the “Versions” column. The “Version history” window appears.
3. Click Compare and select two versions to see a code diff of the changes between them.

Delete a snippet

Delete a snippet when you no longer need it and it is not used by any config variations. Here’s how:

1. From the snippets list, find the snippet you wish to delete.
2. Click snippet in the snippets list. The snippet detail page appears.
3. Click Delete. The snippet is deleted.

Before you delete a snippet, remove any references to it from config variations. LaunchDarkly prevents you from deleting snippets that are still in use.

Use snippets in config variations

You can use snippets in config variations in two ways:

• Attach existing snippets to a variation
• Create new snippets directly in variation messages

Attach snippets to a variation

Attach snippets from the variation editor to include reusable prompt content in a variation message.

To attach snippets to a variation:

1. Navigate to your config.
2. Select the Variations tab and open a variation.
3. There are three ways to use snippets in messages:
   • Click the “Load prompt snippet” button in the variation message body toolbar to open search and find a snippet.
   • In the variation message body, use the {{snippet.}} selector to open search and find a snippet.
   • Reference a snippet directly using {{snippet.example-snippet-key#version-number}}.
4. Click Review and save.

Create inline snippets in messages

You can also create a snippet directly while editing a variation message.

To create snippets in a message:

1. Open a variation and begin editing the message content.
2. Click “Save as” in the variation message body toolbar.
3. Choose to either save the message body text as a New snippet or New version of existing snippet.

Behavior for in-use variations

You can use prompt snippets in variations that are part of active experiments or guarded rollouts.

If a variation that uses a snippet is part of an active experiment or guarded rollout, updating the snippet version may be blocked. LaunchDarkly blocks these updates to prevent changes that could affect experiment results or rollout behavior. If an update is blocked, LaunchDarkly provides details about the active usage, including which experiment or rollout is affected. Use this information to determine when to update the variation after it is no longer part of an active experiment or rollout.

Audit logs and access control

Prompt snippets use the same access control model as AgentControl configs, so you can manage access using existing roles and permissions. Access to create, update, and delete snippets follows the same permissions as other AgentControl resources.

LaunchDarkly records changes to snippets and their attachments in audit logs. These records show when snippets are created, updated, or attached to variations.

Permissions align with existing AgentControl actions. Audit log entries are created for snippet creation, updates, and changes to snippet attachments on variations.

Use audit logs to review changes to snippets and their attachments. This helps you track how snippets change over time and maintain control over changes across AgentControl config variations.

Original source

AI insights

This topic explains how to use AI insights to understand the impact of AgentControl configs by analyzing metrics across your project. AI insights provides a project-level view across configurations, models, and targeting rules. Use it to identify changes, compare configurations, and determine which configurations, models, or variations are driving results and their impact on performance and outcomes.

Use AI insights to:

• Detect changes in cost, usage, and quality metrics
• Identify regressions after updates
• Compare performance across configs, models, and providers
• Analyze trends over time and understand how changes affect performance
• Investigate specific configs and recent changes
• Establish consistent, organization-wide practices for evaluating AI model performance

To analyze performance for a single config, use the Monitoring tab.

AI insights page

The Insights page provides a unified, aggregated view of metrics across your configs. It includes time series charts, summary metrics, and a configuration-level table so you can review performance, identify changes, and understand their impact. All components reflect the selected time range and filters.

Use the controls at the top of the page to select a metric view, group results, filter configurations, and adjust the time range.

To open the Insights page:

1. Navigate to your project.
2. In the left sidebar, expand AI, then select Insights.

Use this page to monitor performance and investigate changes across your configs. A typical workflow includes:

1. Use the trends view to analyze changes over time and understand how updates affect performance.
2. Review quick stats to identify changes in key metrics.
3. Use the configs and variations table to compare configurations and identify which require further investigation. This helps you understand what changed and decide whether to act.

Alerts highlight changes in key metrics so you can identify where to investigate without reviewing each configuration individually.

Trends

The trends view displays metrics as time series charts so you can compare performance and understand how metrics change over time across configurations. You can group results by config, model, provider, or agent graph, including multi-agent workflows. You can analyze metrics such as token usage, latency, satisfaction, error rate, and evaluation scores over time.

Use the trends view to track changes over time and understand their impact on performance. Apply filters to focus your analysis on specific configurations or models.

You can also review changes to prompts, models, and targeting rules alongside performance metrics to understand how updates affect cost, latency, or satisfaction.

Quick stats

Quick stats summarize key metrics, including active configs and experiments, average satisfaction, and cost.

Use quick stats to identify changes in these metrics and determine where to investigate further. These changes may reflect shifts in performance and impact.

Configs and variations

The configs and variations table shows metrics for each config, including generations, token usage, satisfaction, latency, error rate, model and provider, and experiment status.

Use this table to compare configurations and identify differences in metrics that require further investigation.

Alerts

AI insights includes system-generated alerts for changes in key metrics. Use alerts to determine which configurations are driving changes and require further investigation.

Alerts provide a proactive way to monitor performance by highlighting configurations with recent changes so you can focus your investigation. Each alert includes the affected config and details about the change.

Instrumentation requirements

AI insights depends on metrics recorded from your application.

To populate insights, use a LaunchDarkly AI SDK to evaluate configs and record generation metrics such as latency, token usage, success, and error. You can also record evaluation metrics using judges.

If your application does not record metrics, the Insights page may not display data.

Choose a view

Use the following guidance to select the appropriate view:

• Use AI insights to monitor metrics and identify changes across configurations.
• Use the Monitoring tab to analyze performance for a single config and its variations.

These views support different levels of analysis, from investigating a single config to understanding patterns across configurations.

Original source

This topic documents how to get started with the Python AI SDK, and links to reference information on all of the supported features.

The Python AI SDK is designed for use with AgentControl. It is currently in a pre-1.0 release and under active development. You can follow development or contribute on GitHub.

SDK quick links

LaunchDarkly’s SDKs are open source. In addition to this reference guide, we provide source, API reference documentation, and sample applications:

• SDK API documentation: SDK API docs
• GitHub repository: python-server-sdk-ai
• Sample application: Using Bedrock, Using OpenAI, Using Gemini, Using LangChain
• Published module: PyPI

Get started

LaunchDarkly AI SDKs interact with AgentControl configs. Configs are the LaunchDarkly resources that manage model configurations and messages for your generative AI applications.

Try the Quickstart

This reference guide describes working specifically with the Python AI SDK. For a complete introduction to LaunchDarkly AI SDKs and how they interact with configs, read Quickstart for AgentControl.

You can use the Python AI SDK to customize your config based on the context that you provide. This means both the messages and the model evaluation in your generative AI application are specific to each end user, at runtime. You can also use the AI SDKs to record metrics from your AI model generation, including duration and tokens.

Follow these instructions to start using the Python AI SDK in your Python application.

Understand version compatibility

The LaunchDarkly Python AI SDK is compatible with Python 3.8.0 and higher.

Install the SDK

First, install the AI SDK as a dependency in your application using your application’s dependency manager. If you want to depend on a specific version, refer to the SDK releases page to identify the latest version.

Here’s how:

$ pip install launchdarkly-server-sdk
$ pip install launchdarkly-server-sdk-ai

Next, import the LaunchDarkly LDAIClient into your application code:

import ldclient
from ldclient import Context
from ldclient.config import Config
from ldai import LDAIClient, AICompletionConfigDefault, ModelConfig, LDMessage, ProviderConfig

Initialize the client

After you install and import the AI SDK, create a single, shared instance of LDAIClient. Specify your SDK key here to authorize your application to connect to a particular environment within LaunchDarkly.

The Python AI SDK uses an SDK key

The Python AI SDK uses an SDK key. Keys are specific to each project and environment. They are available on the SDK keys page under Settings. To learn more about key types, read Keys.

Here’s how:

ldclient.set_config(Config("YOUR_SDK_KEY"))
aiclient = LDAIClient(ldclient.get())

Configure the context

Configure the context that will encounter generated AI content in your application. Context attributes determine which variation of the config LaunchDarkly serves, based on the targeting rules in your config. If you use template variables in config variation messages, context attributes also provide values for those variables.

Here’s how:

context = Context.builder("example-context-key") \
  .set("firstName", "Sandy") \
  .set("lastName", "Smith") \
  .set("email", "[email protected]") \
  .set("groups", ["Acme", "Global Health Services"]) \
  .build()

Customize a config

Customize your config to set values for variables used in messages or instructions based on the context attributes and variables you provide.

The details of customizing a config depend on whether you are using configs in completion mode or agent mode. You set the mode for a particular config when you create it in the LaunchDarkly UI.

Customize configs in completion mode

In completion mode, each variation in your config includes a single set of roles and messages used to prompt your generative AI model.

In completion mode, use completion_config() to customize the config.

The completion_config() function takes an AgentControl config key, a context, a fallback value, and optional variables. It performs the evaluation, then returns an AICompletionConfig object that includes the customized messages and model configuration. Call create_tracker() on the returned object to get a tracker instance you can use to record metrics.

If it cannot perform the evaluation or LaunchDarkly is unreachable, the SDK returns the fallback configuration you provided. For example, you might use an empty, disabled AICompletionConfigDefault as a fallback value, or a fully configured default. Either way, check for this case and handle it appropriately in your application.

Here’s how:

Customize a config in completion mode

fallback_value = AICompletionConfigDefault(enabled=False)

config = aiclient.completion_config(
  'example-config-key',
  context,
  fallback_value,
  { 'example_custom_variable': 'example_custom_value' }
)

tracker = config.create_tracker()

Evaluate input and output pairs with a judge

Use create_judge() when you need to score a specific input and output pair in application code. This is useful for external pipelines, custom orchestration, or any case where you want to evaluate output directly instead of attaching a judge to a completion-mode config variation.

create_judge() retrieves the judge configuration for the provided context. Then, call evaluate() on the returned judge to score the input and output pair.

Here’s how:

Evaluate input and output directly with a judge

from ldai import AIJudgeConfigDefault

fallback = AIJudgeConfigDefault(enabled=False)

judge = aiclient.create_judge(
  "your-judge-key",
  context,
  fallback,
)

if judge:
  result = await judge.evaluate(input_text, output_text)

  if result.sampled and result.success:
    print(result.score, result.reasoning)

In this example:

• create_judge() retrieves the judge configuration for the provided context
• evaluate() scores the input and output pair
• the returned result includes structured evaluation data

If you want to record returned evaluation scores as metrics associated with a config, track those scores in your application code using the config’s tracker.

Programmatic judge evaluation does not automatically emit Monitoring tab metrics. It also does not attach judges to variations in the LaunchDarkly UI.

To learn more, read Online evaluations in AgentControl.

Customize configs in agent mode

In agent mode, each variation in your config includes a set of instructions that enable multi-step workflows.

In agent mode, use agent_config() or agent_configs() to customize the config. The agent_config() function customizes a single agent config, while the agent_configs() function customizes a list of them.

The instructions returned by the SDK come directly from the instructions you define for the variation in the LaunchDarkly UI. The goal or task shown in the UI is delivered unchanged as the instructions field in the SDK.

Both functions take an AgentControl config key, a context, a fallback value, and optional variables. They perform the evaluation, then return an AIAgentConfig object that includes the customized instructions. Call create_tracker() on the returned object to get a tracker instance for recording metrics.

If the function cannot perform the evaluation or LaunchDarkly is unreachable, it returns the fallback value.

For example, you might use an empty, disabled AIAgentConfigDefault as a fallback value, or a fully configured default. Either way, you should make sure to check for this case and handle it appropriately in your application.

Here’s how:

Customize a config in agent mode

from ldai import AIAgentConfigDefault

agent = aiclient.agent_config(
  'example-config-key',
  context,
  AIAgentConfigDefault(
    enabled=False
  ),
  { 'example_custom_variable': 'example_custom_value'}
)

To learn more, read Customizing configs.

Call the provider and record metrics from AI model generation

Finally, make a request to your generative AI provider and record metrics from your AI model generation.

In completion mode, use the tracker’s track_metrics_of() method to call your generative AI provider and record metrics from model generation. Pass a metrics extractor function (such as get_ai_metrics_from_response from the provider-specific package) and the call you want to make. Check whether the returned config is enabled and handle the disabled case appropriately in your application.

If your AgentControl config uses agent mode, you can access the instructions returned from the customized AgentControl config to send to your AI model. Call create_tracker() on the returned AIAgentConfig object to get a tracker instance you can use to record metrics.

Tracking AI metrics does not automatically associate traces with the config.

Run model calls inside an active span to associate traces

LaunchDarkly can associate traces with an evaluated config only when the model request runs inside an active parent span.

If you use the LaunchDarkly Observability SDK, import observe from ldobserve and wrap each model request in observe.start_span(). This creates the parent span LaunchDarkly uses to link the LLM span to the evaluated config. LaunchDarkly cannot associate the trace with the config if the model call does not run inside an active span.

To learn more, read LLM observability and Monitor configs.

The following example shows how to wrap an OpenAI completion request in an active span so LaunchDarkly can associate traces with the evaluated config.

Using OpenAI model with trace association, completion mode

from ldobserve import observe
from ldai_openai import get_ai_metrics_from_response

if config.enabled:
  messages = [] if config.messages is None else config.messages

  # Wrap the provider call in a span so LaunchDarkly can associate traces
  # with the evaluated config.
  with observe.start_span("llm.chat"):
    completion = tracker.track_metrics_of(
      get_ai_metrics_from_response,
      lambda: openai_client.chat.completions.create(
        model=config.model.name,
        messages=[message.to_dict() for message in messages],
      )
    )
else:
  # Application path to take when the config is disabled

If you are not using observability, or if you have already ensured the model call runs inside an active span, you can call your provider and record AI metrics as shown below.

Using OpenAI model, completion mode

from ldai_openai import get_ai_metrics_from_response

if config.enabled:
  # Pass in a metrics extractor and a callable that invokes your OpenAI client.
  # When calling the OpenAI operation, use details from config.
  # For instance, you can pass config.model.name
  # and config.messages[0].content to your specific OpenAI operation.
  #
  # CAUTION: If the call inside of track_metrics_of throws an exception,
  # the SDK will re-throw that exception

  messages = [] if config.messages is None else config.messages
  completion = tracker.track_metrics_of(
    get_ai_metrics_from_response,
    lambda: openai_client.chat.completions.create(
      model=config.model.name,
      messages=[message.to_dict() for message in messages],
    ),
  )
else:
  # Application path to take when the config is disabled

Alternatively, you can use the SDK’s other track_* functions to record these metrics manually. You may need to do this if your application requires streaming, since track_metrics_of() expects the wrapped call to return a complete response.

In completion mode, make sure to call completion_config() each time you generate content from your AI model:

from ldai_openai import get_ai_metrics_from_response

config = aiclient.completion_config(
  'example-config-key',
  context,
  fallback_value,
  { 'example_custom_variable': 'example_custom_value' }
)

tracker = config.create_tracker()

completion = tracker.track_metrics_of(get_ai_metrics_from_response, lambda: ...)

To learn more, read Tracking AI metrics.

Supported features

This SDK supports the following features:

• Anonymous contexts
• Context configuration
• Customizing configs
• Private attributes
• Tracking AI metrics

Original source

AgentControl lets you manage model configuration and instructions for your AI agents outside of your application code. With AgentControl, you can update prompts and models without redeploying.

By the end of this quickstart, you will have:

• Created your first agent-based config
• Deployed your config and called it from your application
• Made a change to your prompt or model without redeploying

Scope of this quickstart

This quickstart focuses on AgentControl configs in completion mode, which lets you configure prompts with messages and roles for single-step model responses. Completion mode supports multi-message prompts, including chat-style prompts. Completion mode does not map to a specific provider API, so you can use it with any provider LaunchDarkly supports. AgentControl configs can also be created in agent mode, which uses instructions to define multi-step workflows. Agent mode is documented separately in Agents.

Follow the steps below to incorporate AgentControl configs into your app, or use the in-app onboarding to set up your first config directly in the LaunchDarkly UI.

Prerequisites

To complete this quickstart, you need the following:

• A LaunchDarkly account and a server-side SDK key for your environment. To find your SDK key, read SDK credentials. If you haven’t installed a LaunchDarkly SDK yet, continue reading. This topic explains how to install a supported SDK.
• A LaunchDarkly role that allows AgentControl config actions. The LaunchDarkly Project Admin, Maintainer, and Developer project roles, as well as the Admin and Owner base roles, include this permission.
• An API key for your model provider, such as OpenAI or Anthropic, made available to your application as an environment variable.
• Python 3.10 or higher. To get started with NodeJS or other languages, read the AI SDK documentation.

This quickstart uses LangChain as the agent framework. You can adapt the same configuration code to the OpenAI Agents SDK, Strands, or the Claude Agent SDK. For an end-to-end code sample showing different frameworks and model providers, read the example code.

Complete the in-app quickstart

This procedure documents the process of installation through the built-in onboarding experience.

Using an AI coding assistant?

You can set up AgentControl by giving a prompt to your coding assistant. In the quickstart workflow, select Install with AI, choose your LLM provider, and give it the prompt available in the UI.

You can connect your app to LaunchDarkly with an AI agent or with a manual installation process. The procedure below explains how to connect to LaunchDarkly manually.

Step 1: Install the SDK

First, install the LaunchDarkly server-side SDK and AI SDK, as well as the agent framework you want to use.

Installing manually

1. In the in-app quickstart, click Install manually to reveal the SDK configuration instructions.
2. Choose between using your existing app or a sample app.
3. Click to select your SDK language. The instructions for installation update based on which language you choose. In this example, we use Python and LangChain.

Python installation commands:

$ pip install launchdarkly-server-sdk
$ pip install launchdarkly-server-sdk-ai
$ pip install langchain langchain-anthropic

To get started with other frameworks or model providers, read the AgentControl guides.

Step 2: Initialize the client

Initialize the client to connect LaunchDarkly to your app. To do this, you must have your LaunchDarkly SDK key. To find your SDK key, read SDK credentials.

Example Python code to initialize client:

import ldclient
from ldclient.config import Config
from ldai.client import LDAIClient
from ldai import AIAgentConfig, AIAgentConfigDefault
from langchain.agents import create_agent
from langchain.messages import HumanMessage

ldclient.set_config(Config("YOUR-SDK-KEY"))
aiclient = LDAIClient(ldclient.get())
context = ldclient.Context.create("user-123")
config = aiclient.agent_config('YOUR-CONFIG-NAME', context, AIAgentConfigDefault())
tracker = config.tracker

def handle_agent_call_langchain(config: AIAgentConfig, user_input: str,) -> str:
    model = config.model.get_parameter("name") if config.model else "anthropic:claude-sonnet-4-6"
    agent = create_agent(
        model=model,
        system_prompt=config.instructions,
    )
    response = agent.invoke({"messages": [HumanMessage(user_input)]})
    return response["messages"][-1].content

Step 3: Create an agent-based config in LaunchDarkly

Create an agent-based config to store your model settings and instructions. You can do this through the LaunchDarkly UI, or agentically using the LaunchDarkly MCP server from your AI coding assistant. For example, Claude Code and Cursor can both perform this task.

To create the config in the UI:

1. In the left sidebar, click Agents. The AgentControl menu appears.
2. Click Configs.
3. Click Create config. The “Create config” dialog opens, with Completion selected by default.
4. Enter a name for your config, and optionally assign a maintainer.

Save the config key

When you name your config, a key generates automatically. Copy and save this key now. You’ll use it in Step 5.

1. (Optional) If you use flags and segments, you can click Add views to create a view of this config.
2. Click Create.

After you create the config, the Variations tab appears.

Then, create a variation. Every completion-based config has one or more variations. Each variation includes a model configuration and, optionally, one or more messages.

Here’s how:

1. In the Variations tab, replace “Untitled variation” with a variation Name. You’ll use this to refer to the variations when you set up targeting rules below.
2. Click Select a model and choose the model to use.
   • LaunchDarkly provides a list of common models, and updates it regularly.
   • You can also choose + Add a model and create your own. To learn more, read Create and manage AI model configurations.
3. (Optional) Select a message role and enter the message for the variation. If you’d like to customize the message at runtime, use {{ example_variable }} or {{ ldctx.example_context_attribute }} within the message. The LaunchDarkly AI SDK will substitute the correct values when you customize the config from within your app.
   • To learn more about how variables and context attributes are inserted into messages at runtime, read Customizing configs.
4. Click Review and save.

Here’s an example of a completed variation:

(The Variations tab of a config with one variation.)

1. Click Review and save.

To learn more about configuring agent variations, read Agents.

Step 4: Set up targeting

When you’re ready to add more config variations, come back to this step and set up additional targeting rules. If you are familiar with LaunchDarkly’s flag targeting, the process is very similar: with AgentControl, you can target individuals or segments, or target contexts with custom rules. To learn how, read Config targeting.

1. Select the Targeting tab for your agent-based config.
2. In the Default rule section, click Edit.
3. Set the default rule to serve your new variation.
4. Click Review and save.

Your agent-based config is now active. When your application calls the SDK, LaunchDarkly evaluates the targeting rules and returns the variation you configured.

To learn more, read Config targeting.

Step 5: Use the agent-based config with your agent framework

First, retrieve the agent-based config in your application. Replace your-agent-config-key with config Key you saved in Step 3:

Example Python code:

from ldai import AIAgentConfigDefault

config = aiclient.agent_config(
    "your-agent-config-key",
    context,
    AIAgentConfigDefault(),
)

Next, pass the model name and instructions from config into your agent framework. The example below uses LangChain:

from ldai import AIAgentConfig
from langchain.agents import create_agent
from langchain.messages import HumanMessage

def handle_agent_call_langchain(config: AIAgentConfig, user_input: str,) -> str:
    model = config.model.get_parameter("name") if config.model else "anthropic:claude-sonnet-4-5"
    agent = create_agent(
        model=model,
        system_prompt=config.instructions,
    )
    response = agent.invoke({"messages": [HumanMessage(user_input)]})
    return response["messages"][-1].content

Finally, run the agent:

response = handle_agent_call_langchain(
    config=config,
    user_input="Hello, what can you help me with?",
)
print(response)

For an equivalent setup with the OpenAI Agents SDK, Strands, or the Claude Agent SDK, read Example code.

Step 6: Make a change without redeploying

One of the key benefits of AgentControl is that you can update your model or prompt at any time without redeploying your application.

To update without redeploying:

1. Select the Variations tab for your agent-based config.
2. Open your variation and change the model, model provider, or instructions.
3. Click Review and save.

LaunchDarkly immediately serves the updated configuration to your application. The next time your application calls the SDK, it receives the new model and instructions without needing to redeploy.

Step 7: Monitor the agent-based config

In the config page, click the Monitoring tab. When end users use your application, LaunchDarkly monitors config performance. Metrics update approximately every minute.

To learn more, read Monitor config performance.

Summary

Congratulations! You now have an agent-based config that can:

• Load model and instructions from LaunchDarkly at runtime, so your app doesn’t hard-code either
• Change in production without redeploying, by editing the variation in LaunchDarkly
• Keep a full version history of every change to prompts and model settings
• Report cost, token usage, and error rates on the Monitoring tab as real traffic flows through

From here, add a judge, run evals, log traces, and explore the managed AI SDKs.

Example code

This section shows an end-to-end code sample with different frameworks and model providers.

Evaluation results appear on the Monitoring tab as metrics such as accuracy, relevance, and toxicity.

To learn more about evaluation patterns and programmatic usage, read Online evaluations.

Next steps

Now that you have a working config and can read its evaluation results, you can continue to build complexity on that config or create another to do something else.

Here are some options:

• Add a judge to your agent
• Run your first eval
• Create and manage AI model configurations
• View your monitoring data
• Log traces
• Explore our AI SDKs

Learn more about AgentControl

The following sections provide answers to common questions about working with AgentControl.

Integration with AI providers

In AgentControl, LaunchDarkly does not handle the integration to the AI provider. The LaunchDarkly AI SDKs provide your application with model configuration details for providers and frameworks such as OpenAI, Amazon Bedrock, and LangChain, including customized messages and model parameters such as temperature and tokens. It is your application’s responsibility to pass this information to the AI provider or framework.

The LaunchDarkly AI SDKs provide methods to help you track how your AI model generation is performing, and in some cases, these methods take the completion from common AI providers as a parameter. However, it is still your application’s responsibility to call the AI provider. To learn more, read Tracking AI metrics.

To request a new model, click the Give feedback option and let us know what models you’d like to have included.

Privacy and personally identifiable information (PII)

LaunchDarkly does not send any of the information you provide to any models, and does not use any of the information to fine tune any models.

You should follow your own organization’s policies regarding if or when it may be acceptable to send end-user data either to LaunchDarkly or to an AI provider. To learn more, read AgentControl and information privacy.

Original source

The topics in this category explain how to use LaunchDarkly AgentControl to manage your configs.

You can use AgentControl to customize, test, and roll out new large language models (LLMs) in your generative AI applications.

An AgentControl config is a single resource that you create in LaunchDarkly to control how your application uses large language models. It lets teams manage prompts, instructions, and model settings outside of application code so they can iterate, experiment, and release changes more safely without redeploying. To learn how to create one, read Create configs.

Choose a configuration mode

When you create a config, you select a configuration mode that defines how the model behaves in your application.

AgentControl supports two modes:

• Completion mode: Configure prompts using messages and roles for single-step model responses. You can attach judges to completion-mode config variations in the LaunchDarkly UI. To learn more, read Create and manage config variations.
• Agent mode: Configure multi-step workflows using structured instructions. For agent-based variations, invoke a judge programmatically using the AI SDK. Agent mode does not create a separate resource. To learn more, read Agents.

Both modes use the same config resource and support variations, targeting rules, monitoring, experimentation, and lifecycle management.

Both completion mode and agent mode can integrate with external tools or APIs. Tool usage depends on how your application and SDK are implemented, not on the selected configuration mode. Agent mode enables structured, multi-step workflows. You can integrate external tools in either mode.

With AgentControl, you can:

• Manage model configuration outside of your application code so you can update prompts and settings at runtime without deploying changes.
• Upgrade to new model versions and roll out changes gradually and safely.
• Add new model providers and progressively shift production traffic between them.
• Compare variations to determine which performs better based on cost, latency, satisfaction, or other metrics.
• Run experiments to measure the impact of generative AI features on end-user behavior.

AgentControl supports advanced use cases such as retrieval-augmented generation, integration with external tools or APIs, and evaluation in production. You can:

• Track which knowledge base or vector index is active for a given model or audience.
• Experiment with different chunking strategies, retrieval sources, or prompt and instruction structures.
• Evaluate outputs using side-by-side comparisons or online evaluations with judges in completion mode, or invoke a judge programmatically using the AI SDK for other variations.
• Build guardrails into runtime configuration using targeting rules to block risky generations or switch to fallback behavior.
• Apply different safety filters by user type, geography, or application context.
• Use live metrics, including satisfaction and quality signals you define, to guide rollouts.

These capabilities let you evaluate model behavior in production, run targeted experiments, and adopt new models safely without being locked into a single provider or manual workflow.

If you use an AI agent to create and manage configs, you can use LaunchDarkly agent skills to help AI coding agents execute common tasks safely and consistently.

Availability

AgentControl is an add-on feature. Access depends on your organization’s LaunchDarkly plan. If AgentControl does not appear in your project, your organization may not have access to it.

To enable AgentControl for your organization, contact your LaunchDarkly account team. They can confirm eligibility and assist with activation.

For information about pricing, visit the LaunchDarkly pricing page or contact your LaunchDarkly account team.

How AgentControl works

Every config contains one or more variations. Each variation defines model settings with messages for completion mode or instructions for agent mode. You define targeting rules to control which variation LaunchDarkly serves to a given context.

In your application, you use one of LaunchDarkly’s AI SDKs to evaluate a config for a given context. The LaunchDarkly SDK evaluates targeting rules and selects a variation. The AI SDK plug-in then uses that variation to return the resolved configuration, including model settings and messages or instructions.

As part of this evaluation, the AI SDK resolves any variables in your prompts using context attributes and additional variables you provide. This enables you to tailor prompts and model settings for each context at runtime. When you update prompts, instructions, or model configuration in LaunchDarkly, those changes take effect immediately without requiring you to redeploy your application.

LaunchDarkly does not invoke model providers on your behalf. Your application is responsible for calling the model provider directly using its own credentials and the configuration returned by the AI SDK. LaunchDarkly does not proxy or independently invoke model providers.

After your application calls the model provider, use the AI SDK to track AI metrics such as generation count, token usage, latency, errors, and evaluation scores. LaunchDarkly aggregates these metrics and displays them on the Monitoring tab.

The topics in this category explain how to create configs and variations, update targeting rules, monitor related metrics, and incorporate AgentControl into your application.

Additional resources

In this section:

• Set up AgentControl configs
• Quickstart for AgentControl
• Create configs
• Create and manage config variations
• Create and manage AI model configurations
• Tools
• Prompt snippets

Config evaluations

• Playgrounds
• Offline evaluations
• Datasets
• Online evaluations
• Judges
• Run experiments with AgentControl

Agents

• Agents
• Agent graphs

Deliver and monitor configs

• Config targeting
• Monitor config performance
• Understand AI impact with AI Insights
• Manually instrument LLM spans

Manage AgentControl configs

• Manage AgentControl configs
• Compare config variation versions
• AgentControl and information privacy

In our guides:

• Managing AI model configuration outside of code
• Using targeting to manage AI model usage by tier

In our SDK documentation:

• .NET AI SDK reference
• Go AI SDK reference
• Node.js (server-side) AI SDK reference
• Python AI SDK reference
• Ruby AI SDK reference

Original source

Control your agents in production.

Agent behavior shifts in production without warning. AgentControl helps keep agents on track, blocking bad behavior and steering responses in real time.

One place to control agent behavior.

• 01 Agents that self-heal.
• 02 Benchmark changes with offline evals.
• 03 Iterate in milliseconds, not sprints.
• 04 Monitor quality with online evals.
• 05 Visualize your entire agent system.

Set your thresholds with Adaptive Triggers. Your agents handle the rest.

Other tools tell you when an agent fails. AgentControl fixes it automatically. When a response drops below your quality threshold, it escalates to a more capable config—within the same conversation turn, before the customer sees anything.

Stop hardcoding. Start iterating faster.

Replace hardcoded config with a single SDK call wherever you initialize an agent. Manage prompts, models, and tools in AgentControl, instead of scattered across every service in your stack.

BEFORE

# config buried in code — any change = redeploy
# repeat this for every agent — every team, every update cycle
MODEL = "gpt-4o"
TEMPERATURE = 0.3
MAX_TOKENS = 512
SYSTEM_PROMPT = """You are a triage agent for a medical insurance company. Classify the query and route to: provider_agent, policy_agent, or billing_agent."""
def run_triage(query: str) -> str:
    return openai_client.chat.completions.create(
        model=MODEL,
        temperature=TEMPERATURE,
        max_tokens=MAX_TOKENS,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": query},
        ],
    ).choices[0].message.content

AGENTCONTROL SDK

# one-time SDK setup
ctx = Context.builder("user-123").kind("user").build()

def handle_model_call(config, tracker):
    response = tracker.track_openai_metrics(
        lambda: openai_client.chat.completions.create(
            model=config.model.name,
            messages=[m.to_dict() for m in config.messages] + [{"role": "user", "content": query}],
        )
    )
    return response.choices[0].message.content

# each agent is two lines — config lives in AgentControl
config, tracker = aiclient.completion_config("triage-agent", ctx, fallback_config)
handle_model_call(config, tracker)

Works with leading providers and frameworks.

Built for every stage of agent development.

Configure, benchmark, release, observe, and iterate—everything you need to build and run agents in production, without stitching together a different tool for each.

01 Configure

Define agent behavior from one place.

Model settings, prompts, tool configs—all in a central store, separate from the code that deploys them. Shared prompt components propagate across every config automatically. Every change is versioned, auditable, and access-controlled.

02 Benchmark

Nothing ships without clearing your quality bar.

Run offline evals of prompt and model variants against your golden datasets before anything ships. LLM judges score each candidate against your defined thresholds—and only what clears the bar gets to production.

03 Release

Guarded rollouts with automatic rollback.

Roll out a prompt or model change progressively to users—no deployment required. Traffic splits and user targeting let you expand at your own pace. Quality metrics watch every stage: Drift triggers automatic rollback before it reaches more users, and critical failures halt the rollout immediately.

04 Observe

Understand what every agent is doing and why.

Full traces across every agent invocation: What was called, in what order, and how long each step took. Online evals run continuously against production traffic, scoring for quality, cost, and any custom metrics you define. When metrics shift, you'll know which config change caused it.

05 Iterate

Run experiments on live traffic and ship what wins.

A/B and multi-armed bandit experiments on live traffic, scored by LLM judges and business metrics. When a winner emerges, it ships automatically. Every experiment leaves you with better data for the next.

Enterprise-ready from Day 1.

AgentControl is built on the same infrastructure LaunchDarkly uses to serve 50 trillion flag evaluations a day across some of the largest engineering teams in the world. That means reliability, security, and compliance are solved problems before a single agent goes live.

• 50T+ Flag evaluations per day
• < 200ms Config propagation, globally
• 99.99% Enterprise uptime SLA
• SOC 2 Type II Certified
• ISO 27001 Certified
• ISO 27701 Certified
• FedRAMP Moderate ATO

Original source

Add a safety check before experiment changes reach users.

Feature flag changes are not the only production changes that need governance. Experiments do, too. Every experiment has the potential to change a user’s experience, shift traffic, affect business metrics, or influence a product decision. For teams operating in regulated environments—or any organization with strict change management practices—that makes review and accountability essential. That’s why we’re introducing Experiment Approvals, now available in beta. Experiment Approvals extends LaunchDarkly approval workflows to experimentation, giving teams a governed way to start, modify, stop, and ship experiments with the right review in place. The result: more control over experimentation without slowing down the teams responsible for learning fast.

More end-to-end control

Experimentation is one of the most powerful tools a product team can wield. It turns gut feelings into evidence and feature launches into learning opportunities. And for teams already using LaunchDarkly approval workflows for flag changes, it's always made sense to extend that same level of oversight to experiments—the moments when traffic allocation, variant configurations, and key metrics are on the line. With Experiment Approvals, organizations now have a safeguard to help ensure best practices are upheld. For teams operating under compliance requirements, internal change management policies, or strict audit obligations, this means LaunchDarkly can cover the full picture. Not just flags, but every experiment that touches production. For regulated industries, this is especially meaningful. Dual-control requirements, self-approval restrictions, and audit trail mandates apply to production changes of all kinds. Experiment Approvals helps ensure LaunchDarkly meets those requirements end-to-end.

What Experiment Approvals does

Experiment Approvals works just like flag approvals, applied to your entire experimentation workflow. If you already have flag approvals enabled, you're nearly set up. For everyone else, an Admin can turn it on at the environment level.

Here's what it covers:

• Starting an experiment requires approval before traffic is allocated to variants. Experiment creators submit the experiment for review, select approvers, add context, and wait for sign-off before anything goes live.
• Stopping an experiment or shipping a variation follows the same pattern. Whether you're ending a test early or rolling out a winning variant, that action can require explicit approval, giving your team visibility into every significant change before it affects users.
• Modifying a running experiment—changes that would initiate a new iteration—also can require approval. Approvers see a clear diff of what's changing, including both the experiment configuration and the associated flag-targeting rules, so decisions are made with key context.
• A centralized approvals interface gives reviewers a single place to see pending requests, review change details, and approve or decline. Notifications land in the LaunchDarkly inbox via email and Slack. Additional integrations are planned.
• Audit log integration captures every approval request, decision, and application, including who requested it and who approved it. Self-approval prevention is configurable for organizations that require it.

Built for the agentic future

LaunchDarkly has been investing in making our platform accessible to AI agents, and experimentation will be a key part of that journey. That future is exciting, but it introduces a new set of governance challenges.

Experiment Approvals is the foundation that makes agentic experimentation safer. When an agent proposes a large traffic ramp, launches an experiment on a sensitive surface, or makes a change that crosses a risk threshold, the approval workflow is designed so that a human stays in the loop. Low-risk, routine actions can move quickly. Higher-stakes changes route through review.

We're building toward a policy-based model where organizations can define which agent actions require human approval and which can proceed automatically. Experiment Approvals is step one.

Learn more about approvals here.

Get access to Experiment Approvals

Experiment Approvals is available in beta to all customers with LaunchDarkly Experimentation. If you’re interested in getting access to try it out, please contact us.

Original source

Adaptive Triggers is now available in closed beta.

The gap between something going wrong with a production AI system and getting the right fix live has never been zero. An alert fires, someone diagnoses the cause, a decision gets made, the fallback goes live—and users are experiencing the problem throughout. For traditional software, that gap is at least bounded: Behavior stays stable between deploys, and the sources of change are largely things you shipped.

Agents don't work that way, as agents and models are inherently unpredictable and indeterminate. Model checkpoints update on the provider's schedule, provider health fluctuates, and environment changes that nobody on the team initiated can shift behavior that was working reliably the day before. The gap between detection and response is the same as it always was, but the surface area for something going wrong is much larger, and most of it is outside your control.

Most teams running agents in production have already defined what to do when something goes wrong: a fallback model with a backup provider, a more conservative configuration for when the primary fails. The alternative is there, already wired up. What's been missing is the mechanism that activates it at the moment the signal arrives, without waiting for someone to make the call. Today, we’re happy to introduce the missing piece.

Adaptive Triggers is now available in closed beta. Teams define a rule directly on a config: When a monitored metric breaches a threshold within a time window, switch to a specified variation. When the threshold is crossed, AgentControl makes the switch automatically, with no human in the loop.

A team running their agent on Claude Sonnet hosted on AWS Bedrock has a backup variation configured to route the same model to GCP Vertex. When Bedrock error rates climb past the configured threshold (say, more than 10 failures in five minutes), the trigger fires. AgentControl switches the default variation to the Vertex configuration and traffic reroutes. The experience is uninterrupted, and nobody gets paged.

The response that doesn't wait

What makes the switch instant isn't only that it's automated. Because AgentControl controls the configuration layer, the fallback variation has already been pulled by the SDK and is instrumented in the running system. When the trigger fires, there's nothing to build and no deployment to kick off. The switch happens in under 200 milliseconds because AgentControl sits inside the application, not between it and the model provider. The team made the decision about what to do in advance, and Adaptive Triggers executes it the moment the signal arrives.

Most tools can surface a problem or change a setting. Adaptive Triggers does both automatically, in real time, inside the same platform. The gap between seeing a production problem and responding to it closes when the response is already defined.

The direction from here extends to every signal in the observe-and-act loop: quality scores that drop below threshold, cost spikes that warrant routing to a lighter configuration, paired triggers that restore the primary variation automatically when metrics recover. The response that doesn't wait becomes a loop that runs on its own.

Adaptive Triggers is available in closed beta. Reach out to your account manager to request access.

Original source