LaunchDarkly Release Notes

This topic documents how to get started with the Go AI SDK, and links to reference information on all of the supported features.

The Go AI SDK is designed for use with AgentControl. It is in a pre-1.0 release and the API may change based on feedback. You can follow development or contribute on GitHub.

This version replaces the previous Config and Tracker API

If your codebase calls aiClient.Config() or tracker.TrackRequest(), those methods have been deprecated. This version introduces separate completion, agent, and judge config modes, along with a new set of tracker methods. Review this reference before you upgrade.

SDK quick links

LaunchDarkly’s SDKs are open source. In addition to this reference guide, we provide source, API reference documentation, and sample applications:

Get started

LaunchDarkly AI SDKs interact with AgentControl configs. Configs are the LaunchDarkly resources that manage model configurations and messages for your generative AI applications.

Try the Quickstart

This reference guide describes working specifically with the Go AI SDK. For a complete introduction to LaunchDarkly AI SDKs and how they interact with configs, read Quickstart for AgentControl.

You can use the Go AI SDK to customize your config based on the context that you provide. This means both the messages and the model evaluation in your generative AI application are specific to each end user, at runtime. You can also use the AI SDK to record metrics from your AI model generation, including duration and tokens, and to evaluate model output with judges.

Follow these instructions to start using the Go AI SDK in your application.

Install the SDK

First, install the AI SDK as a dependency in your application. How you do this depends on what dependency management system you are using:

• If you are using the standard Go modules system, import the SDK packages in your code and go build will automatically download them. The SDK and its dependencies are modules.
• Otherwise, use the go get command and specify the SDK version, such as go get github.com/launchdarkly/go-server-sdk-ai.

The Go AI SDK is built on the Go SDK, so install that as well.

Here is how:

import (
  ld "github.com/launchdarkly/go-server-sdk/v7"
  "github.com/launchdarkly/go-server-sdk-ai/ldai"
)

Initialize the client

After you install and import the SDK, create a single, shared instance of LDClient. Then, use it to initialize the AI client. The AI client is how you interact with configs. Specify the SDK key to authorize your application to connect to a particular environment within LaunchDarkly.

The Go SDK uses an SDK key

The Go SDK uses an SDK key. Keys are specific to each project and environment. They are available on the SDK keys page under Settings. To learn more about key types, read Keys.

Here is how:

client, _ := ld.MakeClient("YOUR_SDK_KEY", 5*time.Second)
aiClient, err := ldai.NewClient(client)
if err != nil {
  // Client couldn't be created
}

This example assumes you have imported the LaunchDarkly SDK package as ld, as shown above.

Best practices for error handling

The second return type in these code samples (_) represents an error in case the LaunchDarkly client does not initialize. Consider naming the return value and using it with proper error handling.

Configure the context

Next, configure the context that will use the config, that is, the context that will encounter generated AI content in your application. The context attributes determine which variation of the config LaunchDarkly serves to the end user, based on the targeting rules in your config. If you are using template variables in the messages in your config’s variations, the context attributes also fill in values for the template variables.

Here is how:

context := ldcontext.NewBuilder("example-context-key").
  Kind("user").
  Name("Sandy Smith").
  SetString("email", "[email protected]").
  SetValue("groups", ldvalue.ArrayOf(ldvalue.String("Acme"), ldvalue.String("Global Health Services"))).
  Build()

Customize a config

Then, use one of the config retrieval methods to customize a config. Customization means that any variables you include in the messages when you define the config variation have their values set to the context attributes and variables you pass in. The AI SDK provides three config modes, completion, agent, and judge. You set the mode for a particular config when you create it in the LaunchDarkly UI.

The customization process within the AI SDK is similar to evaluating flags in one of LaunchDarkly client-side, server-side, or edge SDKs, in that the SDK completes the customization without a separate network call. If it cannot perform the evaluation or LaunchDarkly is unreachable, it returns the fallback value you provide. For example, you might use an empty, disabled default as a fallback value, or a fully configured default. Either way, you should make sure to check for this case and handle it appropriately in your application.

All three config modes share a common set of methods through an embedded base: Key(), Enabled(), Model(), ModelName(), Provider(), ProviderName(), and CreateTracker().

Customize configs in completion mode

In completion mode, each variation in your config includes a single set of roles and messages used to prompt your generative AI model. Use CompletionConfig to customize the config.

The CompletionConfig method takes a config key, a context, a fallback value, and optional variables. It performs the evaluation, then returns an AICompletionConfig object with the customized messages and model configuration.

Here is how:

fallbackValue := ldai.NewAICompletionConfigDefault().
  WithEnabled(true).
  WithModelName("claude-sonnet-4-6")
config := aiClient.CompletionConfig(
  "example-config-key",
  context,
  fallbackValue,
  map[string]interface{}{"exampleCustomVariable": "exampleCustomValue"},
)

After you call CompletionConfig, you can pass the customized messages directly to your AI provider. To learn more, read Customizing AgentControl configs.

Customize configs in agent mode

In agent mode, use AgentConfig or AgentConfigs to customize the config. The AgentConfig method customizes a single agent config. The AgentConfigs method customizes a batch of them. Agent configs add Instructions() and JudgeConfiguration() on top of the shared base methods.

Here is how:

fallbackValue := ldai.NewAIAgentConfigDefault().
  WithEnabled(true).
  WithModelName("claude-sonnet-4-6")
agent := aiClient.AgentConfig("example-agent-key", context, fallbackValue, variables)
instructions := agent.Instructions()

Customize model parameters

Every default value supports builder-style setters that customize the model, provider, and underlying parameters before evaluation:

fallbackValue := ldai.NewAICompletionConfigDefault().
  WithEnabled(true).
  WithModelName("claude-sonnet-4-6").
  WithProviderName("anthropic").
  WithModelParam("temperature", 0.7).
  WithCustomModelParam("top_k", 40).
  WithTool(myTool)

Disable a config by default

Provide a fallback value with Enabled set to false so the client falls back to your default behavior if the flag targeting rules do not enable the config, or if LaunchDarkly is unreachable.

Use template configs

CompletionConfigTemplate, AgentConfigTemplate, and JudgeConfigTemplate skip Mustache interpolation and do not accept a variables parameter. Use a template method if you plan to interpolate message content yourself, or if a config has no placeholders to fill.

Evaluate input and output pairs with a judge

Use JudgeConfig to retrieve a judge config. Judge configs add Messages() and EvaluationMetricKey() to the shared base methods.

Judges are constructed directly, not through the client

Unlike completion mode and agent mode, the AI SDK does not expose a client-level method to create a judge. Construct one directly from the judge subpackage, and implement the Provider interface yourself so the judge can call your model.

Here is how:

judgeConfig := aiClient.JudgeConfig("example-judge-key", context, fallbackValue, variables)
type myProvider struct{}
func (p *myProvider) InvokeStructuredModel(
  messages []datamodel.Message,
  schema map[string]interface{},
) (judge.StructuredResponse, error) {
  // Call your AI provider and return a structured response.
}
j, err := judge.New(judgeConfig, tracker, &myProvider{}, "example-judge-key", loggers)

Call Evaluate to score a single input and output pair, or EvaluateMessages to evaluate a full message list against a response. samplingRate is a value between 0.0 and 1.0. If Evaluate skips the call because of sampling, or because the judge config is empty, it returns nil, nil rather than an error:

result, err := j.Evaluate(input, output, samplingRate)

Recording the judge response is your responsibility, not the judge’s. Call tracker.TrackJudgeResponse yourself after Evaluate or EvaluateMessages returns. The judge does not call it for you.

Call provider, record metrics from AI model generation

TrackDuration, TrackSuccess, TrackTimeToFirstToken, and TrackTokens are at-most-once, which means LaunchDarkly records only the first call for each. TrackSuccess and TrackError are also mutually exclusive, the first one you call wins. TrackTokens replaces the deprecated TrackUsage method, use TrackTokens in new code.

Here is how:

if config.Enabled() {
  tracker, err := config.CreateTracker()
  // Make a request to a provider using details from config.
  // For example, you can pass model parameters (config.ModelParam) or messages (config.Messages).
  tracker.TrackSuccess()
  tracker.TrackDuration(elapsed)
  tracker.TrackTokens(tokenUsage)
} else {
  // Application path to take when the config is disabled.
}

Each tracker call shares a single runId, a UUIDv4 created when you create the tracker. The runId correlates every event you record on that tracker as one AI run.

Alternatively, you can use TrackDurationOf to wrap a function call and measure its wall-clock duration automatically. For applications that require streaming, use the package-level TrackMetricsOf function to wrap an operation that returns a value and an error. Pass the tracker to TrackMetricsOf along with a function that extracts an AIMetrics value from the operation result. TrackMetricsOf records the operation’s success or error, duration, and tokens, and is the recommended replacement for the deprecated TrackRequest method.

TrackFeedback is multi-fire. Call it as many times as you receive feedback, for example once per user reaction. TrackJudgeResponse is also multi-fire. Use it to record a judge evaluation result.

Go names this tracker method differently than Java and .NET

Go names this method TrackJudgeResponse. The Java and .NET AI SDKs name the equivalent method TrackJudgeResult.

Use GetSummary to read the metrics recorded on a tracker so far, ResumptionToken to get a token for reconstructing the tracker later, and GetTrackData to read the raw track data.

To learn more, read Tracking AI metrics.

Build and traverse agent graphs

An agent graph links multiple agent configs together into nodes and edges. Retrieve a graph definition with AgentGraph:

graph := aiClient.AgentGraph("example-graph-key", context, variables)

AgentGraphDefinition exposes Enabled(), RootNode(), GetNode(key), GetChildNodes(key), GetParentNodes(key), TerminalNodes(), and CreateTracker(), which returns a *GraphTracker.

Each AgentGraphNode exposes Key(), Config(), Edges(), and IsTerminal(). Each GraphEdge exposes the target node’s Key() and a Handoff() map of values passed along that edge.

Traverse and ReverseTraverse walk the graph in topological order. Traverse visits a node only after every reachable predecessor of that node has been visited, starting from the root. ReverseTraverse visits a node only after every reachable descendant has been visited, ending at the root. Both orderings are cycle-safe and deterministic, and give each callback its own dependency-scoped context. LaunchDarkly never mutates the initialContext value you pass in:

graph.Traverse(func(node *ldai.AgentGraphNode, context map[string]interface{}) interface{} {
  // Run the node.
  return result
}, initialContext)

GraphTracker records graph-level and edge-level metrics separately. Graph-level methods, such as TrackInvocationSuccess, TrackInvocationFailure, TrackDuration, TrackTotalTokens, and TrackPath, are at-most-once. TrackInvocationSuccess and TrackInvocationFailure are mutually exclusive, so the first call wins, and TrackDuration ignores non-finite values. Edge-level methods, such as TrackHandoffSuccess, TrackHandoffFailure, and TrackRedirect, are multi-fire.

Use GetSummary to read a GraphMetricSummary, and ResumptionToken to get a token for reconstructing the tracker later.

Resume tracking across processes

Both trackers support resumption, so you can start tracking an AI run in one process and continue it in another.

For a completion, agent, or judge tracker, call CreateTracker on the client with a saved token. For a graph tracker, call CreateGraphTracker on the client, or the package-level TrackerGraphFromResumptionToken function directly:

tracker, err := aiClient.CreateTracker(token, context)
graphTracker, err := aiClient.CreateGraphTracker(token, context)

Reconstructing a tracker from a resumption token preserves the original runId, so at-most-once guards still apply across the resumed run.

Supported features

This SDK supports the following features:

• Anonymous contexts
• Context configuration
• Customizing AgentControl configs
• Private attributes
• Tracking AI metrics

Original source

AgentControl

The topics in this category explain how to use LaunchDarkly AgentControl to manage your configs. You can use AgentControl to customize, test, and roll out new large language models (LLMs) in your generative AI applications.

An AgentControl config is a single resource that you create in LaunchDarkly to control how your application uses large language models. It lets teams manage prompts, instructions, and model settings outside of application code so they can iterate, experiment, and release changes more safely without redeploying. To learn how to create one, read Create configs.

Choose a configuration mode

When you create a config, you select a configuration mode that defines how the model behaves in your application.

AgentControl supports two modes:

• Completion mode: Configure prompts using messages and roles for single-step model responses. To learn more, read Create and manage config variations.
• Agent mode: Configure multi-step workflows using structured instructions. Agent mode does not create a separate resource. To learn more, read Agents.

You can attach judges to both completion-mode and agent-mode config variations directly in the LaunchDarkly UI. You can also invoke a judge programmatically using the AI SDK.

Both modes use the same config resource and support variations, targeting rules, monitoring, experimentation, and lifecycle management.

Both completion mode and agent mode can integrate with external tools or APIs. Tool usage depends on how your application and SDK are implemented, not on the selected configuration mode. Agent mode enables structured, multi-step workflows. You can integrate external tools in either mode.

With AgentControl, you can:

• Manage model configuration outside of your application code so you can update prompts and settings at runtime without deploying changes.
• Upgrade to new model versions and roll out changes gradually and safely.
• Add new model providers and progressively shift production traffic between them.
• Compare variations to determine which performs better based on cost, latency, satisfaction, or other metrics.
• Run experiments to measure the impact of generative AI features on end-user behavior.

AgentControl supports advanced use cases such as retrieval-augmented generation, integration with external tools or APIs, and evaluation in production. You can:

• Track which knowledge base or vector index is active for a given model or audience.
• Experiment with different chunking strategies, retrieval sources, or prompt and instruction structures.
• Evaluate outputs using side-by-side comparisons or online evaluations with judges in completion mode or agent mode, or invoke a judge programmatically using the AI SDK.
• Build guardrails into runtime configuration using targeting rules to block risky generations or switch to fallback behavior.
• Apply different safety filters by user type, geography, or application context.
• Use live metrics, including satisfaction and quality signals you define, to guide rollouts.

These capabilities let you evaluate model behavior in production, run targeted experiments, and adopt new models safely without being locked into a single provider or manual workflow.

If you use an AI agent to create and manage configs, you can use LaunchDarkly agent skills to help AI coding agents execute common tasks safely and consistently.

Availability

AgentControl is an add-on feature. Access depends on your organization’s LaunchDarkly plan. If AgentControl does not appear in your project, your organization may not have access to it.

To enable AgentControl for your organization, contact your LaunchDarkly account team. They can confirm eligibility and assist with activation.

For information about pricing, visit the LaunchDarkly pricing page or contact your LaunchDarkly account team.

How AgentControl works

Every config contains one or more variations. Each variation defines model settings with messages for completion mode or instructions for agent mode. You define targeting rules to control which variation LaunchDarkly serves to a given context.

In your application, you use one of LaunchDarkly’s AI SDKs to evaluate a config for a given context. The LaunchDarkly SDK evaluates targeting rules and selects a variation. The AI SDK plug-in then uses that variation to return the resolved configuration, including model settings and messages or instructions.

As part of this evaluation, the AI SDK resolves any variables in your prompts using context attributes and additional variables you provide. This enables you to tailor prompts and model settings for each context at runtime. When you update prompts, instructions, or model configuration in LaunchDarkly, those changes take effect immediately without requiring you to redeploy your application.

LaunchDarkly does not invoke model providers on your behalf. Your application is responsible for calling the model provider directly using its own credentials and the configuration returned by the AI SDK. LaunchDarkly does not proxy or independently invoke model providers.

After your application calls the model provider, use the AI SDK to track AI metrics such as generation count, token usage, latency, errors, and evaluation scores. LaunchDarkly aggregates these metrics and displays them on the Monitoring tab.

The topics in this category explain how to create configs and variations, update targeting rules, monitor related metrics, and incorporate AgentControl into your application.

Additional resources

In this section:

Set up AgentControl configs

• Quickstart for AgentControl
• Create configs
• Create and manage config variations
• Create and manage AI model configurations
• Tools
• Prompt snippets

Config evaluations

• Playgrounds
• Offline evaluations
• Datasets
• Online evaluations
• Judges
• Run experiments with AgentControl

Agents

• Agents
• Agent graphs

Deliver and monitor configs

• Config targeting
• Monitor config performance
• Understand AI impact with AI Insights
• Manually instrument LLM spans

Manage AgentControl configs

• Manage AgentControl configs
• Compare config variation versions
• AgentControl and information privacy

In our guides:

• Managing AI model configuration outside of code
• Using targeting to manage AI model usage by tier

In our SDK documentation:

• .NET AI SDK reference
• Go AI SDK reference
• Node.js (server-side) AI SDK reference
• Python AI SDK reference
• Ruby AI SDK reference

Original source

How LLM observability works

This topic explains how LaunchDarkly captures and displays large language model (LLM) spans, and how it groups related LLM spans into conversations. You can use LaunchDarkly LLM observability features to monitor the performance of your models in production and diagnose problems with them.

LLM observability helps your team:

• Optimize LLM latency by monitoring token usage and request duration
• Investigate provider errors
• Compare model outputs by reviewing prompt and response pairs across environments
• Read what an agent did, to determine the turn where it went wrong
• Compare cost and latency across different agent runs
• Analyze downstream impact by connecting LLM spans with session or error data

How LLM observability works

When your application calls an LLM provider:

1. Instrumentation in the LaunchDarkly observability SDK captures telemetry about the model request.
2. The SDK exports LLM telemetry as span attributes in OpenTelemetry traces.
3. LaunchDarkly records the LLM spans for display on the Traces page.

Each span includes the detailed information you need to evaluate model behavior across environments, such as the model name, prompt and response content, token usage, request duration, and provider information.

LaunchDarkly marks LLM spans with a green indicator labeled “LLM” in the traces view.

About LLM conversations

A single agent run rarely fits into one trace. A typical agent run involves a variety of activities that generate multiple traces over a period of time, such as:

• Answering a question
• Calling one or more tools to perform tasks or gather information
• Waiting for a person to reply, or for a tool call to return data
• Repeating these actions minutes or hours later after new information becomes available

Because each of these activities arrives as a separate trace, reading a single, logical conversation with an LLM requires organizing and connecting the different activities span-by-span.

LaunchDarkly automatically stitches together all spans that share a common conversation identifier. It orders the messages and tool calls into a single, readable transcript. LaunchDarkly also rolls up key attributes such as the duration, token usage, models, and errors across the entire agent run.

You can use LaunchDarkly LLM conversations to browse and display full agent runs as they occurred in response to user prompts and tool responses. Conversations also work as a starting point to dive into span information at any turn in the conversation to learn details about problems or errors that occurred when interacting with an agent.

Set up LLM observability

To set up LLM observability, you configure a LaunchDarkly observability SDK in your application to instrument the generative AI attributes LaunchDarkly reads on each span. The steps vary by SDK and by LLM provider.

To learn more about instrumenting your application so LLM spans and conversations render correctly, read Instrumenting LLM applications.

Associate traces with AgentControl

LLM observability captures spans for any instrumented model call. When you use LaunchDarkly AgentControl together with the LaunchDarkly Observability plugin and spans are successfully exported, LaunchDarkly associates traces with the AgentControl config that generated them.

The AI SDK annotates the root span with the underlying feature flag key for the evaluated config. LaunchDarkly uses this annotation to link related spans to the correct config when spans are exported through the Observability SDK.

With this integration, you can:

• Filter traces by config key or variation
• Correlate model behavior with variations and targeting
• Investigate latency, errors, and quality signals in context

If you do not use the LaunchDarkly Observability SDK, you can still associate traces with AgentControl by adding span attributes in your tracing pipeline. For example, attach the AgentControl config key and evaluated variation to the root span when your application calls the model provider.

View and analyze LLM spans

LaunchDarkly displays LLM observability data in two places:

• The Monitoring tab on a config, when spans relate to that config
• The global Traces page

Each view serves a different purpose.

View traces for a specific config

To view trace data for a config:

1. Click Agents. The AgentControl menu appears.
2. Click Configs.
3. Open the Monitoring tab.

If LaunchDarkly links spans to that config, it displays them in this panel.

Why "No traces detected" appears

If the config page shows No traces detected, LaunchDarkly has not linked any spans to that config.

LaunchDarkly evaluates configs using a context. To associate spans with a config:

• The config must evaluate with a valid context.
• The LLM call must occur within the same request flow.

If your application calls a model without a LaunchDarkly context, or outside the config evaluation flow, LaunchDarkly records the span on the global Traces page but does not associate it with the config. To learn more about how LaunchDarkly makes this association, read Associate traces with AgentControl.

Show all LLM spans

To explore all captured LLM spans, open the Telemetry section and navigate to the Traces list.

The Traces page lists all captured spans, including LLM spans that may not relate to a specific config. LaunchDarkly marks LLM spans with a green indicator. Select a span to view detailed model telemetry.

Use the search bar, filters, and time range selector to analyze spans. The trace detail panel shows a timeline of generation steps and related spans, provider and model metadata such as latency and token usage, prompt and response content, and any provider errors or exceptions.

Filter on LLM span attributes

LaunchDarkly builds LLM observability and conversation views from the OpenTelemetry generative AI semantic conventions. It normalizes several common non-standard formats when it receives them, including the OpenLLMetry llm.* attributes and Claude Code telemetry. Search and display use the normalized names, so filter on the names below rather than the names your instrumentation emits.

LLM spans include the following attributes:

• gen_ai.request.model: The model that handled the request.
• gen_ai.provider.name: The provider that handled the request.
• gen_ai.usage.input_tokens: The number of input tokens processed.
• gen_ai.usage.output_tokens: The number of tokens in the output.
• gen_ai.prompt.0.content: The input prompt text. Subsequent messages use increasing indexes.
• gen_ai.completion.0.content: The generated response. Subsequent responses use increasing indexes.
• duration: The total latency of the span.
• service.name: The name of the emitting service.

Use these attributes to filter, search, and investigate model behavior. For example, to find slow requests to a specific model:

Example search query:

gen_ai.request.model=gpt-4 AND duration>2s

Instrumentation that follows the current semantic conventions reports message content as gen_ai.input.messages and gen_ai.output.messages instead of the indexed attributes above. For the full set of attributes LaunchDarkly reads and how to set them, read Instrumenting LLM applications.

Read LLM conversations

The identifier LaunchDarkly groups on is the gen_ai.conversation.id span attribute. Every span that carries the same value joins the same conversation.

Conversation terminology

Understanding these terms will help you read the conversation view:

• Conversation: One complete interaction, identified by a conversation ID set on every related span. A conversation can span many traces.
• Trace: One connected unit of work within a conversation, such as a single agent invocation. LaunchDarkly displays trace boundaries in the conversation transcript, but they are a background detail rather than the main structure.
• Turn: One message or one tool call. Turns are the unit you read, navigate, and select in the conversation view.
• Mission: One question from a person, combined with everything the agent did to answer the question. Missions start at each user turn, so each one covers a single request and its follow-through.
• Evaluation: A quality score attached to a turn or to a whole conversation, such as the relevance or safety rating produced by a model-graded evaluator.

Find a conversation

Navigate to “Monitor,” click Traces, and select the Conversations tab.

The list displays one row per conversation, with the following columns:

• Conversation: The conversation ID your application set.
• Last activity: When the most recent span in the conversation started.
• Duration: Elapsed time from the first span to the end of the last span.
• Traces: How many distinct traces the conversation spans.
• Spans: Total spans carrying the conversation ID.
• Model: Every distinct model used in the conversation.
• Provider: Every distinct provider used in the conversation.
• Input tokens, Output tokens: Token counts summed across the conversation.
• Errors: How many spans reported an error status.

Sort by any column, and add or remove columns with the column picker. The list covers the last seven days.

You can also open a conversation from any span inside it. Select a span on the traces page, then use the scope control to switch from the single trace to the whole conversation.

Read the transcript

The conversation view has two tabs:

• Messages displays a continuous transcript of user messages, assistant replies, and tool calls in the order they happened, grouped into missions. Use this tab to follow what happened.
• Waterfall displays the conversation’s spans on a timeline. Use this tab to investigate timing and nesting.

Click the span icon on a turn to display raw data for the turn in a span details drawer.

Each turn displays the metadata available for it, which may include the model, token counts, latency, cost, and any evaluation scores. Tool calls display the tool name with a summary of the arguments and result, and expand to the full payload.

Turns whose timing LaunchDarkly could not determine do not display a time offset. This happens when a message is recovered from a later snapshot of the conversation history rather than observed directly. To learn more, read Emit message content.

Read the summary

The conversation header summarizes the whole thread:

Metric - How LaunchDarkly calculates it

• Duration: Elapsed time from the earliest span to the end of the latest span.
• Traces: Distinct traces. Hover to read the per-trace breakdown.
• Steps: Number of turns in the transcript, after LaunchDarkly removes duplicate observations of the same message.
• LLM: Spans identified as model calls or agent invocations.
• Tools: Tool execution spans. Hover to read the per-tool breakdown.
• Total tokens: Input, output, cache read, and cache write tokens combined. Hover to read the breakdown of tokens.
• Models: Distinct models. Hover to read the per-model call count.
• Estimated cost: Token counts priced using your configured model costs.

Why some totals differ between views

The Conversations list and the conversation header count different things.

The header adds prompt-caching tokens to its total, because cache reads often make up most of a cached workload’s context. The list reports only input and output tokens. For a conversation that uses prompt caching, expect the header total to be substantially larger than the list total.

The header’s Steps count removes duplicate observations of the same message, while LLM and Tools count spans. An agent runtime that emits several spans for one logical tool call raises the Tools count without changing Steps.

Read evaluations

If your application emits evaluation results, LaunchDarkly displays them in two places:

• On a turn, as a badge with the evaluation name and score. Scores between 0 and 1 display as fractions, and other numeric scores display on a 0 to 10 scale. Names that suggest a risk measure, such as toxicity or hallucination, use low scores to indicate positive results.
• On the conversation, as an overall score in the header for when an evaluator scored the conversation as a whole.

Attach evaluation results to the span where you want them to appear. If you attach a result to a separate evaluation span, it appears on that span rather than on the turn it describes.

Troubleshooting

When a conversation does not render as expected, the cause is usually a missing or misplaced attribute in your instrumentation. The following table maps common symptoms to their likely causes:

• The conversation does not appear in the list: gen_ai.conversation.id is missing, or set under a different key. It must be on every span in the conversation.
• The conversation appears, but the transcript is empty: No span carries gen_ai.input.messages, gen_ai.output.messages, or gen_ai.system_instructions.
• A tool renders before the message that requested it: The model call’s output has no matching tool_call part.
• Every turn appears at the start of the conversation: tool_call parts are on the envelope span rather than the per-call spans.
• The transcript opens with the agent replying to nothing: The opening question was never recorded.
• The person’s question appears as agent output: Message text is accumulated in a shared buffer rather than keyed per message.
• Tokens appear in the conversation but the list displays zero: Token usage uses the older prompt_tokens and completion_tokens names.
• Estimated cost is blank: At least one model call is missing a model, a provider, or a token count, or uses a model with no configured cost.
• Turns display no timing: Those messages were recovered from a later history snapshot rather than observed directly.
• Evaluation badges do not appear: Evaluation events are attached to a separate evaluation span rather than to the span being scored.
• The LLM Agent default dashboard reports less activity than expected: Inference spans carry HTTP or database attributes and are classified by those instead.
• A failed run appears as an empty conversation: The run failed before its first model call, so the error is recorded on a span that contributes no turns.

Privacy and data handling

LLM spans and conversations display prompt and response text, tool arguments, and tool results. This content can include personally identifiable information (PII) depending on your application.

Review your organization’s data-handling policies before you enable LLM observability or emit message content. Redact sensitive values in your instrumentation rather than after LaunchDarkly receives them.

Original source

Adaptive triggers are available to customers on select plans

Adaptive triggers are only available to customers on select plans. To learn more, read about our pricing. To upgrade your plan, contact Sales.

This topic explains how to use adaptive triggers to automatically serve a different flag variation and send an alert when a metric indicates a problem.

An adaptive trigger serves a different flag variation in response to a metric’s degraded performance over a specified time period. When the metric’s performance crosses a threshold you specify, LaunchDarkly sends an alert to team members or Slack channels you specify, so they can take further action if needed.

Each trigger applies only to the environment where you create it. To monitor the same flag in more than one environment, create a separate trigger in each environment.

Triggers do not run on flags with active experiments

If you create a trigger on a flag that has a running experiment, the trigger will not work. Similarly, if you add a trigger to a flag and then add an experiment to that flag later, the trigger will not work after you start the experiment.

Prerequisites

To use adaptive triggers with flags, you must have a LaunchDarkly role of Owner or Admin, or a custom role that allows the following actions:

• createTriggers, deleteTriggers, updateFallthrough, and updateTriggers actions for Feature flags.
• createAlert, deleteAlert, and updateAlertConfiguration actions for Alerts.

Adaptive triggers do not require approval

Adaptive triggers bypass the approval process. Members can add or edit a trigger without an approval, and when a trigger fires, it switches the variation without an approval, even if your environment requires approvals for flag changes.

Adaptive triggers do not work with page viewed metrics or clicked or tapped metrics

Adaptive triggers work only with custom metrics or observability metrics.

Configure adaptive triggers

When you create a trigger, many configuration options are available to ensure the trigger fires only when you mean it to.

Configure source settings

The Source section of the trigger creation screen lets you choose a source for the event to track. Available sources are:

• LaunchDarkly hosted metrics: Choose from the list of available LaunchDarkly metrics. If the metric you choose measures more than one analysis unit, an Analysis unit menu appears. Use it to choose the context kind the metric is measured per.
• Logs: The default function is Count. You can choose a Function to aggregate data with and then refine the data into categories with Group by.
• Traces: The default function is Count. You can choose a Function to aggregate data with and then refine the data into categories with Group by.
• Sessions: Click + Add filters to refine a query. The adaptive trigger will alert one time for every session that matches the filters.
• Errors: Click + Add filters to refine a query. The adaptive trigger will alert one time for every open error that matches the filters.
• Events: Use filters and functions to identify and aggregate data, then refine the data into categories with Group by.
• Observability Metrics: The default function is Avg. You can choose a Function to aggregate data with and then refine the data into categories with Group by.

If you choose Sessions or Errors and set the threshold Type to Anomaly, the Function and Group by options become available for those sources.

Triggers do not update when a metric changes

LaunchDarkly records the version of the metric a trigger was built from. If you change the metric’s definition later, existing triggers continue to use the definition from when you created them. To pick up the new definition, delete the trigger and create it again.

Configure threshold settings

The Threshold section of the trigger creation screen lets you configure four fields to specify when a trigger should fire. Threshold options are:

• Type: The threshold type determines which options are available in the other three fields. Choose Constant to fire the alert when the value crosses a fixed number, or Anomaly to fire the alert when the value deviates significantly from its baseline. Anomaly is unavailable for LaunchDarkly hosted metrics that count or total events per unit. Those metrics support constant thresholds only.
• Condition: The evaluation rule for the alert. Choose Above to fire the alert when the value rises above the alert threshold, or Below to fire the alert when the value falls below it. If you use the Anomaly threshold type, Outside is also available. It fires the alert when the value deviates from its baseline in either direction.
• Alert threshold: For Constant alerts, enter the value the source must reach before the alert fires. Enter a whole number between 1 and 999,999. If the metric you chose declares a unit, the field displays that unit. For Anomaly alerts, choose a confidence level of 80%, 90%, 95%, or 99%. Higher confidence levels require larger deviations from the baseline before the alert fires.
• Alert window: Set a window of time in which the conditions must be met for the alert to fire.

Optional advanced threshold settings are also available. They are:

• Cooldown: Choose a duration from the dropdown to designate how long after an alert has been triggered before it can be triggered again. Setting this to a shorter duration may result in more frequent alerts, but setting it to a longer time period may mean events occur that match your criteria but do not trigger alerts.
• Evaluation delay: Set a number of minutes to wait after an alert window ends before evaluation begins. This is useful if you know data from your alerts will arrive late.

Triggers monitor an entire environment

An adaptive trigger evaluates its source across the whole environment, not only the contexts that receive the flag. If other flags or code paths in that environment produce the same events, those events also count toward the alert threshold.

Create adaptive triggers

To create an adaptive flag trigger:

1. In the left navigation, click Flags. The flags list appears.
2. Find the flag you want to add a trigger to and click its name. The flag’s Targeting tab opens.
3. In the Rules section, click Add adaptive trigger. The trigger configuration screen appears.
4. In the Source section, choose a metric source from the dropdown and configure it with the customization options that appear. To learn more, read Configure source settings.
5. In the Threshold section, set the Type, Condition, Alert threshold, and Alert window. To learn more, read Configure threshold settings. If you chose Sessions as your source and Constant as your type, only the Type field appears, because session alerts fire one time for each matching session.
6. (Optional) Click to expand Advanced threshold settings.
7. (Optional) Set a Cooldown period and Evaluation delay duration.
8. In the Switch variation to section, choose a config variation to serve if the trigger fires.
9. (Optional) Click + Add notification and designate individual team members, Slack channels, or both to receive notifications if this trigger fires.
10. Click Add.

An summary of the trigger’s behavior appears at the bottom of the config variation’s rule. When the conditions are met, the trigger fires.

Where alerts appear

When the trigger fires, an alert is sent to the people and channels you specify.

An alert also appears on the observability Alerts page in the LaunchDarkly UI. The alert displays the flag name, the variation name, and other information, such as which Slack channels got a notification. Click into the alert to see more information about it, such as the error rate over a period of time.

Edit adaptive triggers

To edit an adaptive trigger:

1. Navigate to the Targeting tab of the flag you want to edit a trigger for.
2. Click the pencil icon next to the trigger.
3. (Optional) Select a variation to switch to if the trigger is activated. If you have multiple triggers on the rule, they must all switch to the same variation.
4. (Optional) Click Add notification and choose to Add members or Add Slack channels to be notified when the trigger fires.
5. Click Save changes.

Delete adaptive triggers

To delete an adaptive trigger, navigate to the Targeting tab of the flag you want to delete a trigger for and click the x next to the trigger. LaunchDarkly removes the trigger from the rule.

Delete trigger alerts

You cannot disable or delete alerts that come from triggers. Instead, delete the trigger to remove the alert, or edit the trigger to change the alert threshold.

Original source

Agent optimization is in beta.

Agent optimization is in beta. Aspects of this feature may change without notice or become deprecated.

Agent optimization lets you iteratively improve your agent’s instructions and parameters using judges and acceptance criteria. An optimization is a form of config that runs tests against an existing AgentControl config to optimize, tweak, or model performance. This lets you identify unintended or undesired behavior before your end users do.

During optimization runs, LaunchDarkly collects the following information:

• The total tokens used
• The input/output of an optimization run
• The evaluator’s scores and their rationale
• Latency for each individual run, as well as for each evaluator

Prerequisites

• Python 3.14+
• Your agent runtime of choice. The SDK is agnostic towards what you use.

Exploratory and expected output modes

Agent optimization runs in two modes:

• Exploratory mode
• Expected output mode

Exploratory mode

Exploratory mode is for new agents, or agents that have a large, unbound surface area that might take in many distinct and varied inputs. Exploratory mode is rooted in chaos engineering. The core idea is to supply a set of possible inputs that will be randomly selected to help build resilience and identify possible unintended outcomes.

For example, when you use the orchestrator pattern for agents, your orchestrator may need to handle a wide variety of differently formatted and unique inputs. Exploratory mode works well for this use case because you can put down multiple user inputs and expected variables and have it test different permutations without the need to specify each one. You can ensure that the agent is resilient to a wide variety of conditions because the optimization scores the output based on the input and output relationship, which is comprised of the data the agent had access to and its ultimate response.

This is also a great way to test things that have access to tools with non-deterministic outcomes. Fetching user data from your system should always return the same results, but fetching things from the internet can have a large disparity in the amount and usefulness of data that returns. Using exploratory mode ensures that even if the data available to the LLM is different on each invocation, the response is still logically sound.

Exploratory mode is not for situations with specific mappings of data, contexts, or variables that are only relevant to specific invocations. For example, if you’re passing different variables depending on the users’ context values, this could lead to failing results because of the conflation of variables and context values. Unless you’re testing for resiliency, in a situation like that you probably want the ground truth mode.

Here’s an example of an exploratory mode setup:

The exploratory mode input configuration in the LaunchDarkly UI, showing separate lists of user inputs and variable choices.

Expected output mode

Expected output mode is for already established agents where you have a corpus of knowledge of expected inputs and outputs already available.

For example, if you have an agent that handles retrieving user preferences and mapping those to specific products within a system, you may already have an idea of which inputs should lead to which outputs. Expected output is the right choice if you want to change agent behavior, such as tone, formatting, or which tools it uses, without losing effective responses around the data returned.

Expected output is not for situations with volatile or non-deterministic tools, such as internet search, because the results you receive back may not match exactly and the system penalizes results that don’t contain the same information as your expected outputs. Expected output can also lead to overfitting if there’s not a wide variety of expected test cases provided.

Here is an example in the UI of a ground truth setup:

The ground truth mode input configuration in the LaunchDarkly UI, showing paired rows of user inputs, expected responses, and variable values.

Optimization results

After an optimization run completes, the results appear in the Optimization results page.

The optimization results UI, showing a passing run banner with options to open the variation or deploy it, and a list of past runs below.

Optimization results show a banner indicating the state of your latest run with options for next steps

• Open Variation opens the variation generated by the optimization process
• Deploy opens the config targeting page for the agent so that you can promote it to your chosen targeting rules

Earlier runs appear on this page as well. Click an entry for a previous run to expand it and display:

• Performance over iterations, which is a chart displaying the scores, latency, tokens and estimated cost for the run
• Config iterations, a section which contains all of the specific details for each of the iterations of the run. If you used ground truth mode, these are further stratified by the individual inputs that were tested within each pass.
• Each of these iterations or inputs shows the input data used for that test pass, the output from the agent as well as the rationale and scores for each of the judges or acceptance statements you’ve attached.

LaunchDarkly collects this data automatically if you use optimize_from_config or if you use optimize_from_options and set the auto_commit property to True.

To view results, set the API key environment variable

Data collection requires you to set the LAUNCHDARKLY_API_KEY environment variable. If data does not appear in the UI, verify this is configured correctly. To learn more, read the Optimization SDK quickstart.

Start using agent optimization

Configure agent optimization in your SDK. To get started, read Optimization SDK quickstart.

Original source

Migrating your configuration to v3 of the LaunchDarkly provider

Overview

This topic explains how to upgrade your Terraform configuration from v2 to v3 of the LaunchDarkly provider. v3 changes every nested block to a nested attribute, so configurations written for v2 do not parse after you upgrade the provider. You must rewrite them before your first plan against v3. The provider ships the migrate-tf-syntax tool to automate most of the rewrite, and it upgrades your state automatically on first apply.

Here is the same resource in both syntaxes:

# v2 block syntax
resource "launchdarkly_feature_flag" "example" {
  variation_type = "boolean"
  variations {
    value = "true"
  }
  variations {
    value = "false"
  }
}

# v3 nested attribute syntax
resource "launchdarkly_feature_flag" "example" {
  variation_type = "boolean"
  variations = [
    {
      value = "true"
    },
    {
      value = "false"
    },
  ]
}

Attributes that hold exactly one object rather than a list use object syntax, a bare { ... } with no brackets: client_side_availability and defaults on launchdarkly_feature_flag, default_client_side_availability on launchdarkly_project, fallthrough on launchdarkly_feature_flag_environment, approval_settings on launchdarkly_environment (and inside each project environment), segment_approval_settings on launchdarkly_environment, instructions on launchdarkly_flag_trigger, and boolean_defaults on launchdarkly_flag_templates. The migrate-tf-syntax tool emits this form for you:

# v2 block syntax            # v3 object syntax (no brackets)
client_side_availability {
  using_environment_id = true
}
client_side_availability = {
  using_environment_id = true
}

When you read one of these from a data source, use object access without a list index: data.launchdarkly_feature_flag.x.client_side_availability.using_environment_id.

launchdarkly_project.environments becomes a map keyed by the environment key rather than an ordered list, so reordering, adding, or removing one environment no longer shifts the others or forces a destructive plan. The environment's key is also kept inside the object and equals the map key, so references like launchdarkly_project.example.environments["production"].key keep working. The migrate-tf-syntax tool performs this rewrite for you:

# v2 block syntax              # v3 map syntax (keyed by env key)
environments {
  key = "production"
  name = "Production"
  color = "EEEEEE"
}
environments = {
  "production" = {
    name = "Production"
    key = "production"
    color = "EEEEEE"
  }
}

The map is authoritative: an environment removed from the map is deleted, and a project must have at least one environment. To manage the project in Terraform but its environments in the LaunchDarkly UI, or with launchdarkly_environment resources, declare your environments and add lifecycle { ignore_changes = [environments] }.

Warning:

Changing an environment's key, which is the map key, deletes that environment, including its SDK keys and all flag targeting, and creates a new one.

Reference an environment by its key instead of by index: a v2 interpolation such as launchdarkly_project.example.environments[0].client_side_id becomes launchdarkly_project.example.environments["production"].client_side_id. The migrate-tf-syntax tool does not rewrite these positional references, because auto-editing arbitrary expressions risks corrupting your config, but it detects them and prints the exact replacement to make, including the resolved key, so the fix is mechanical. See "Finish the migration by hand" below.

Three more collections follow the same key-addressed map pattern, and the tool rewrites all of them:

• custom_properties on launchdarkly_feature_flag becomes a map keyed by the custom property key: custom_properties = { "my.key" = { name = ..., value = [...] } }.
• role_attributes on launchdarkly_team and launchdarkly_team_member becomes a plain map of string lists keyed by the role attribute key: role_attributes = { myAttribute = ["value1", "value2"] }. This is the same shape launchdarkly_team_role_mapping already uses.
• edges on the new launchdarkly_ai_agent_graph resource is a map keyed by edge key, net-new in v3, so no rewrite applies.

Prerequisites

You need the following things to complete this migration:

• Terraform 1.0 or later
• A v2 configuration that applies cleanly, with an empty plan before you start
• A committed copy of your configuration and state, so you can review the upgrade as a diff

Convert your configuration with migrate-tf-syntax

The provider ships migrate-tf-syntax, a deterministic command-line tool that rewrites every affected attribute across a directory of .tf files. It also updates the attributes that v3 removed. For example, it renames policy_statements to inline_roles on launchdarkly_access_token, and it updates references to renamed data source attributes such as client_side_availability on the launchdarkly_project data source. It adds the now-required variations to boolean flags that omitted them.

To convert a configuration directory:

1. Download the migrate-tf-syntax archive for your platform from the provider release assets, or run the tool with Go. Replace v3.0.0 with the version you are upgrading to:

   go run github.com/launchdarkly/terraform-provider-launchdarkly/scripts/[email protected] \
   -dir ./my-config -direction v2-to-v3 -dry-run
   

2. Review the dry-run output. The tool prints each file it intends to change.
3. Run the same command without -dry-run to write the changes. Add -recursive to convert locally vendored modules in the same pass.
4. Run terraform fmt to normalize whitespace.
5. Check any note: lines the tool printed. For example, an environment whose key is a variable or local becomes a parenthesized map key, such as (local.env_key) = { ... }. Confirm the expression is the one you expect.
6. Update the provider version constraint to ~> 3.0 and run terraform plan.

Finish the migration by hand

The tool converts syntax only. Complete these follow-ups yourself:

• Add variations by hand only for a flag whose variation_type is a non-literal expression, such as a variable or local. The tool cannot resolve those statically, so it warns and skips them. Boolean flags with a literal variation_type are handled automatically, and the provider preserves any variation name or description set outside Terraform when your configuration omits them.
• Rewrite dynamic blocks. A dynamic "variations" block needs a for expression, for example variations = [for v in var.values : { value = v }]. The tool warns with the file and resource address, and it leaves the attribute unchanged.
• Upgrade modules sourced from a registry or a git URL. The tool rewrites only files it reaches on disk, so upgrade those modules at their source.
• Rewrite positional references to launchdarkly_project environments. The tool converts the environments block to a map but does not edit index expressions elsewhere in your config; it warns on each one with the exact replacement, for example environments[0] → environments["production"] and environments[*] → values(...). Apply those edits by hand.
• Review projects that pair lifecycle { ignore_changes = [environments] } with standalone launchdarkly_environment resources. This v2 pattern keeps working in v3: ignore_changes preserves the standalone-managed environments in the authoritative map, so nothing is deleted. If you remove the ignore_changes entry, first declare every environment in the project's environments map, or the next apply deletes the undeclared ones.

How v3 upgrades your state

The provider includes a state upgrader for every resource whose state shape changed. On your first apply, the provider migrates your state automatically. You do not edit the state file by hand:

• launchdarkly_access_token: moves policy_statements into inline_roles, and discards expire.
• launchdarkly_custom_role: converts policy into policy_statements.
• launchdarkly_feature_flag: converts include_in_snippet into client_side_availability, and re-keys custom_properties into a map keyed by property key.
• launchdarkly_project: converts include_in_snippet into default_client_side_availability, re-keys the ordered environments list into a map keyed by environment key, and converts each environment's approval_settings to an object.
• launchdarkly_environment: converts approval_settings (and segment_approval_settings) to objects.
• launchdarkly_feature_flag_environment: converts fallthrough to an object.
• launchdarkly_flag_trigger: converts instructions to an object.
• launchdarkly_flag_templates: converts boolean_defaults to an object.
• launchdarkly_team and launchdarkly_team_member: re-key role_attributes into a map of string lists.
• launchdarkly_metric: discards is_active, and renames randomization_units to analysis_units, following the LaunchDarkly API's rename.

Your first plan after upgrading

Expect a non-empty first plan after you upgrade the provider binary. v2 stored empty lists where v3 stores null, so diffs such as policy_statements = [] -> null appear once and apply cleanly. You may also see one-time in-place updates that re-assert values the upgrader normalized away, such as client_side_availability or approval_settings objects that match the LaunchDarkly API defaults; they apply cleanly and do not recur. A few computed attributes show as known after apply on the first plan, and they resolve on apply. No resource is destroyed or recreated. The follow-up plan is empty.

What does not change

• v3 removes no resources and no data sources. Every v2 resource and data source remains available.
• Authentication is unchanged. The access_token, oauth_token, api_host, http_timeout, and max_concurrency provider settings keep the same names and behavior.

If you consume the provider through Crossplane

If you embed this provider through Crossplane's Upjet, the block-to-attribute change alters the generated custom resource definition (CRD) shape, even though the attribute names do not change. We recommend that you test CRD regeneration against v3 before you upgrade.

Original source

This topic describes how to use metric measurement windows to prevent bias in warehouse native experiment results.

A metric measurement window defines a fixed, required period of time during which LaunchDarkly collects metrics for a context. A window begins after a context first receives a flag variation in an experiment, and ends after a configured duration.

Attached experiments only consider those metric events that occur within the configured measurement window, and the window must fully complete before the experiment includes any measurements for a context. When you stop an experiment, any metrics for contexts that have not reached the end of their measurement window do not contribute to experiment results.

Restricted to Snowflake data sources

You can only configure metric measurement windows with metrics created from Snowflake data sources, for use with warehouse native Experimentation. You cannot configure windows on metrics created from LaunchDarkly hosted events or other warehouse data sources. To learn more, read Metric event sources.

Avoiding measurement bias in experiments

Metric measurement windows ensure that all contexts included in an experiment have the same amount of time available to generate metric measurements. You define the period of time required to produce a conversion event or to produce the volume of metric values you want to measure. LaunchDarkly ensures that only measurements from completed windows are considered in experiment results. In this way, metric windows help you prevent late or incomplete user activities from biasing experiment results.

When no measurement window is configured (the default behavior), all contexts that participate in an experiment contribute equally to the experiment result. Contexts added near the end of the experiment have less time to generate metrics, so they can negatively impact the experiment results. For example, consider a conversion event that typically requires three days to complete. Most contexts that join the experiment in the last two days of the experiment will negatively impact the conversion metric, even if some of those contexts eventually generate a conversion. By configuring a metric window of three days, you ensure that the experiment only considers units that have the full three days in which to produce the conversion event.

Prerequisites and limitations

You can configure metric measurement windows for any warehouse native metric created from a Snowflake data source. You cannot configure windows on metrics created from LaunchDarkly hosted events or other warehouse data sources. To learn more, read Snowflake native Experimentation.

The data source you use for creating the metric must map the Timestamp value to a column that uses timestamp or datetime format and encodes timezone information. To learn more, read Metric data sources.

By default, new metrics do not include a metric measurement window. When no measurement window is configured, all metric measurements collected during an experiment contribute equally to the experiment result. This matches the LaunchDarkly experiment and guarded release behavior prior to the introduction of metric windows.

Configuring a metric window

When you choose the duration of a metric measurement window, keep in mind:

• The duration you choose should be long enough to capture all relevant measurements. LaunchDarkly stops measuring events for a context after the window completes.
• If your conversion events require a lengthy period of user activity to generate, ensure that experiments run long enough to capture the needed volume of completed metric windows.
• If you are measuring an activity that requires a lengthy setup process, you can optionally configure an offset value to delay measurements until some point after a context first receives a flag evaluation. Offset values enable you to exclude early metric events that might be considered as “noise” in the overall measurement. For example, if you want to measure input validation errors that occur when customers complete a purchase, you could use an offset value to exclude input validation errors that occur earlier in the purchase process.

You configure optional metric window properties after you specify the analysis method for a warehouse native metric.

To configure a measurement window:

1. Open the Data section and navigate to the Metrics list.
2. Click Create metric. The “Create metric” dialog appears.
3. Select Warehouse native from the “Event source” drop-down menu.
4. Select an available Snowflake data source from the “Metric data source” drop-down menu. To learn more, read Metric data sources.
5. Search for or enter an Event key to use for the metric.
6. Choose options in the “Metric definition” section to configure the metric aggregation type and analysis method. To learn more, read Components of a metric.
7. Select the Enable custom measurement window checkbox.
8. Use the “Window measured in” menu to choose whether to define the window in days, hours, or minutes.
9. (Optional) Change the value in the “Start” field to delay metric measurements for a period of time after a context receives a flag variation. A value of zero specifies no offset, meaning the measurement window begins immediately after a context joins the experiment.
10. Enter a value in the “End” field to configure the length of the measurement window.

The end value must be greater than the start value

Both the “End” and “Start” values are calculated relative to when a context first receives a flag variation in an experiment. The “End” value must be greater than the “Start” value to configure a measurement window.

1. Enter a Metric name and and optional Description.
2. Click Create.

Interpreting metrics excluded from experiments

Metrics with measurement windows introduce additional conditions for excluding their measurements from experiment results. For any connected experiment, a context’s metric measurements are excluded if:

• Measurements occur outside of a configured metric window. Any metric events that occur before the window offset, and any events that occur after the window completes, are excluded from the experiment.
• A configured measurement window does not complete. If the experiment ends before a context’s measurement window completes, all measurements for that context are excluded.
• A configured measurement window completes, but the context generated no events. For numeric metrics, if you select the Exclude units that generate no events option, then contexts that complete their measurement window are excluded if they generate no event. To learn more, read Units without events.

LaunchDarkly experiments show the full accounting of units that are excluded from experiment results, both during the experiment and at experiment completion. To view a breakdown of why units are excluded from an experiment, click a value in the “Sample size” column of the experiment results.

To learn more, read Experiment results data.

Original source

This topic describes how to use configure metric winsorization to eliminate extreme values in experiment results.

Restricted to warehouse native metrics

You can configure winsorization only for warehouse native metrics, for use with LaunchDarkly experiments. You cannot configure winsorization for metrics created from LaunchDarkly hosted events. To learn more, read Metric event sources.

Limiting the impact of extreme values on experiments

Winsorization is a statistical technique that replaces outlying values with values at a configured percentile. Using metric winsorization helps you limit the impact of extreme values in LaunchDarkly experiment results without introducing bias by selectively removing values.

For example, consider a metric that generates the following (sorted) latency values for a single randomization unit during the course of an experiment:
5, 120, 135, 140, 145, 150, 155, 160, 170, 950

The value 950 is an extreme outlier in this sample of data, and would significantly affect the mean latency value. The P90 value for the data is 170, so configuring winsorization for the metric at upper bound P90 percentile replaces all values higher than 170 with the value 170. This yields the modified data set:
5, 120, 135, 140, 145, 150, 155, 160, 170, 170

LaunchDarkly supports configuring one-sided or two-sided winsorization as needed to limit the impact of extremely high values, extremely low values, or both. For example, winsorizing the example metric at both the lower bound P10 percentile and the upper bound P90 percentile mitigates the lower outlying 5 value, yielding the modified data set:
120, 120, 135, 140, 145, 150, 155, 160, 170, 170

Common use cases

Winsorization is most commonly used with metrics that produce a long tail distribution of values. This generally corresponds to metrics that measure revenue, latency, or session duration. Using winsorization at the upper bound for P90 or higher percentiles limits the impact of extremely high values that would negatively skew experiment results.

How LaunchDarkly computes winsorization percentiles

LaunchDarkly computes the percentile values for winsorization using all metrics collected for a randomization unit across all arms of a warehouse native experiment. For metrics that use a metrics measurement window configuration, LaunchDarkly uses only those values collected within the configured window to determine the percentile values. If no window is configured, the measurement duration corresponds to the length of the experiment itself.

If you choose Include units and set the value to 0 for a numeric metric, LaunchDarkly does not include assigned zero values when it computes winsorization percentiles. To learn more, read Units without events.

Prerequisites and limitations

You can configure metric measurement winsorization for any warehouse native metric. You cannot configure winsorization for metrics created from LaunchDarkly hosted events.

Configuring winsorization

You configure optional winsorization properties after you specify the analysis method for a warehouse native metric.

To configure winsorization:

1. Open the Data section and navigate to the Metrics list.
2. Click Create metric. The “Create metric” dialog appears.
3. Select Warehouse native from the “Event source” menu.
4. Select an available data source from the “Metric data source” menu, or create a new data source. To learn more, read Metric data sources.
5. Search for or enter an Event key to use for the metric.
6. Choose the metric aggregation from the “Metric definition” section. The window populates a full metric definition using default values.
7. Change options in the “Metric definition” drop-down menus as needed to change the analysis units or other metric analysis options. To learn more, read Components of a metric.
8. (Optional) Choose Enable custom measurement window if you want to configure a metric measurement window. To learn more, read Metric measurement window.
9. Choose Enable winsorization.
10. Enter percentile values in the Lower bound and Upper bound fields as needed to specify the percentile value(s) used to winsorize extreme values. If you are configuring two-sided winsorization, the Upper bound value must be greater than the Lower bound value.

Set Lower bound to zero, or Upper bound to 100, to disable winsorization for that bound.

1. Enter a Metric name and and optional Description.
2. Click Create.

Original source

This topic explains how to create LaunchDarkly ratio metrics, which measure the ratio of two separate aggregations from Snowflake warehouse data sources.

About ratio metrics

A ratio metric is a complex metric type that computes the ratio of two separate metric aggregations: one for the numerator component and one for the denominator component. Each aggregation component is a simple metric created from a warehouse data source. LaunchDarkly divides the total of the numerator aggregation by the total of the denominator aggregation to compute the final metric value.

Configuring numerator and denominator aggregations for a ratio metric.

A ratio metric helps you determine how flag variations affect the relationship between two measurements. You can use ratio metrics to address the following use cases, which simple metrics do not support:

• Clustered analysis for experiments: Ratio metrics let you measure events that are grouped by an analysis unit that is finer-grained than the randomization unit chosen for the experiment. For example, you could measure the number of session errors generated relative to the total number of prompts that users created. In this example, you randomize the experiment by user contexts and configure a user aggregation for the denominator component, then configure a session aggregation for the numerator.
• Conversion metrics normalized by a separate event: Simple metrics can measure conversion events normalized by users who participate in the experiment. Ratio metrics can measure conversion events normalized by users that generate some other event, configured in the denominator aggregation. For example, you could measure the number of users who clicked on a search result, relative to the number of users who initiated a search.

You can add ratio metrics only to LaunchDarkly experiments that use the frequentist statistical methodology.

LaunchDarkly uses the computed value of the ratio to compare the performance of flag evaluations during an experiment. Because a ratio metric can track two separate analysis units, LaunchDarkly uses the statistical delta method to account for covariance of the two units. This helps to ensure that experiment recommendations remain accurate, even when analysis units in the denominator perform differently for a given experiment arm. To learn more, read Statistical methodology for frequentist experiments.

Aggregation types for ratio metrics

You configure each aggregation in a ratio metric separately. The numerator and denominator can use the same or different aggregation types, and can even use data from different warehouse data sources. However, the selected data sources must include context key mappings for one or more shared context kinds.

For example, if the data source for the numerator contains context key mappings for the “account,” “user,” and “session” context kinds but the denominator includes mappings for “session” and “device” contexts, then the ratio metric can only use “session” as the analysis unit. To learn more about mapping context keys in a warehouse data source, read Create data sources.

Ratio metrics support the same aggregation types available for simple metrics, plus one additional aggregation type, count_distinct, that counts distinct values from a warehouse column you provide. The available aggregation types are:

• Count uses sum aggregation to measure the total number of times a context generates the metric event (conversion metric).
• Sum uses sum aggregation to measure the total of the numerical values provided with a context’s metric events (numeric metric).
• Average uses average aggregation to measure the average numerical value provided with a context’s metric events (numeric metric).
• Count distinct uses count_distinct aggregation to measure the total number of times a unique value appears in a warehouse column associated with a context’s metric events (numeric metric). This option ignores duplicate or null values in the column.

The count_distinct aggregation type is available only for ratio metrics. It provides the flexibility to measure a distinct signal independently of the metric event count or event value. When you choose Count distinct, you provide the name of a warehouse column associated with the metric event to use for measuring distinct values.

For all ratio metric aggregations that correspond to a numeric metric, LaunchDarkly uses imputation to assign a zero value to any units that do not produce a metric event. To learn more, read Units without events.

Prerequisites and limitations

You can create ratio metrics only from Snowflake warehouse native metrics, for use with warehouse native Experimentation. Configure your Snowflake warehouse integration and create a data source before you create new warehouse native metrics. To learn more, read Metric data sources.

You cannot create ratio metrics from LaunchDarkly-hosted metric events, OpenTelemetry traces, or from warehouse data sources other than Snowflake.

You can add ratio metrics only to experiments that use the frequentist statistical methodology. To learn more, read Analyzing experiments.

Create a ratio metric

To create a ratio metric:

1. Open the Data section and navigate to the Metrics list.
2. Click Create metric. The “Create metric” dialog appears.
3. Click Ratio in the “Select metric structure” section. The “Select metric structure” section appears only if you have configured warehouse native Experimentation and a Snowflake warehouse data source.
4. Configure metric options for the “Numerator” component:
   i. Choose an existing data source from the Select metric data source menu. The data sources you choose for the numerator and denominator components must have at least one context key mapping to the same context kind.
   ii. Enter the Event key to use for the numerator component of the metric.
   iii. Choose an aggregation type from the Aggregate as menu:
   - Count uses sum aggregation to measure the total number of times a context generates the metric event (conversion metric).
   - Sum uses sum aggregation to measure the total of the numerical values provided with a context’s metric events (numeric metric).
   - Average uses average aggregation to measure the average numerical value provided with a context’s metric events (numeric metric).
   - Count distinct uses count_distinct aggregation to measure the total number of times a unique value appears in a column associated with a context’s metric events (numeric metric). This option ignores duplicate or null values in the column.
   iv. If you chose Count distinct, enter a column name to use for the measurement in the Count distinct on field.
   v. (Optional) Click + Add winsorization or + Add custom measurement window as needed to configure the aggregation. To learn more, read Metric winsorization or Metric measurement window.
5. Repeat the previous step to configure an aggregation for the “Denominator” component of the metric. The aggregation type or warehouse column does not need to match the configuration of the numerator. However, the data sources you choose for the numerator and denominator must have at least one shared context key mapping.

Both metric windows must complete

If you configure a metric window for the numerator and denominator, windows for both components must complete before LaunchDarkly includes the ratio metric result in an experiment. To learn more, read Interpreting metrics excluded from experiments.

After you configure the numerator and denominator, the window displays a complete metric definition using the values you selected.

1. Use the Analysis unit menu to select the analysis units to use for the metric. The same analysis units apply to both the numerator and denominator components of the ratio metric.
2. Choose higher is better or lower is better to define the success criteria for the computed ratio.
3. Enter a metric Name.
4. (Optional) Add a Description.
5. (Optional) Add any Tags.
6. (Optional) Update the Maintainer.
7. Click Create.

You can also use the REST API: Create metric.

Original source

This topic explains how to create LaunchDarkly custom metrics from OpenTelemetry traces.

This feature is for Early Access Program customers only

Creating metrics from trace spans is only available to members of LaunchDarkly’s Early Access Program (EAP). To request access to this feature, contact your LaunchDarkly account manager.

LaunchDarkly lets you create custom metrics from OpenTelemetry traces instrumented in your code, for use with guarded rollouts.

Creating metrics from traces lets you use existing OpenTelemetry instrumentation to measure how application behavior changes during a guarded rollout, without re-instrumenting those behaviors as metric events using the track() method. You define a query that selects the trace spans you want to use to measure your application, and create a metric based on the query result. Each matching span observed during a guarded rollout indicates a metric event.

To create numeric metrics from traces, such as for measuring regressions, you specify the span attribute name that provides the value you want to monitor.

Trace metrics use the same aggregation and analysis methods as LaunchDarkly metrics created from metric events. To learn more, read Components of a metric.

Prerequisites and limitations

You can create custom metrics from trace spans that include a LaunchDarkly feature flag span event, indicated by the feature_flag.set.id attribute. The LaunchDarkly observability SDK plugins automatically include feature flag span events when you create trace spans.

If your application uses a different OpenTelemetry library or package, you can configure the OpenTelemetry collector or use tracing hooks to add the required feature flag span events. To learn more, read Server-side SDKs.

Trace metrics are compatible with guarded rollouts for managing feature releases. You cannot use trace metrics with LaunchDarkly experiments.

Creating trace metrics

You can create trace metrics using either the LaunchDarkly “Create metric” dialog or the observability “Traces” page. Both methods let you create a span query to select the spans you want to monitor in the metric.

Use the “Create metric” dialog if you are familiar with creating metrics and filtering metric events and you want to use a similar interface to create metrics from spans. Use the “Traces” page if you are familiar with the observability search syntax or you want a real-time preview of your filter query results.

Span query syntax differences

The “Create metric” dialog uses a graphical query builder to help you filter trace spans, while the “Traces” page uses the observability query search specification.

Both interfaces can create equivalent span filters, but they use different syntax for some conditions. The “Create metric” dialog uses the present token to indicate whether an attribute is present in a span. It does not support the EXISTS operator or its negation, NOT EXISTS.

For example, if you use the “Traces” page to create a search that reads service_version EXISTS, the equivalent filter in the “Create metric” dialog is service_version is present.

Using the Create metric interface

To use the “Create metric” dialog to create a new trace metric:

1. Navigate to the Metrics list.

2. Click Create metric. The “Create metric” dialog appears.

3. Select Traces from the “Event source” drop-down.

4. Use the “Filter to spans” field to create a query that selects the trace spans to use for the new metric.

   The “Filter to spans” field uses these basic controls:

   • Click an empty spot in the field to display the span attributes, operators, or relational functions you can add to the query at that spot.
   • Type characters to filter the available selections you can add. Enter or click a name in the drop-down list to add the value.
   • Use checkboxes in the drop-down list to select multiple values, or use commas to separate multiple, typed values.
   • Click an existing attribute name, value, or operator to change or edit the displayed value.
   • Click the x icon next to a name, value, or operator to delete it.

   Span functions (Any span, Any span 2, Any span 3) can only be added to the top level of the filter query, and cannot be nested within parentheses. To learn more, read Span functions for trace search.

5. (Optional.) Click View traces to view recent spans that match your filter query. This opens a new tab to the Traces page with your filter query applied.

6. In the “Metric definition” section, choose the aggregation type for your metric. The window populates a full metric definition using default values.

   • Use Count or Count distinct units (Percent) to measure matching trace spans during a guarded rollout.
   • Use Sum or Average to create a numeric metric that aggregates a value from the filtered trace spans.

7. If you are configuring a numeric metric:
   i. Click the “Choose a numeric span attribute” field and type or select the span attribute that provides the numeric value you want to measure.
   ii. Select Include units and set the value to 0 or Exclude units that generate no events to choose how to handle units that do not generate a matching span during a rollout.
   iii. Enter a Unit of measure to describe the value you are measuring.

8. Use the “Metric definition” section to configure the metric’s analysis method and success criteria. To learn more, read Analysis method.

9. Enter a Metric Name.

10. (Optional) Add a Description.

11. (Optional) Add any Tags.

12. (Optional) Update the Maintainer.

13. Click Create.

Using the Traces interface

To use the “Traces” page to create a new trace metric:

1. Navigate to the Traces page.
2. Use the Search… field to create a query that selects the trace spans to use for the new metric. Use feature_flag.set.id EXISTS to limit results to trace spans that have the required feature flag span event. To learn more, read Search specification.
3. Review the filter query results in the “Spans” list.
4. Select Create metric from the three-dot menu next to your query. This opens the Create metric dialog and populates it with the query you created.
5. In the “Metric definition” section, choose the aggregation type for your metric. The window populates a full metric definition using default values.
   • Use Count or Count distinct units (Percent) to measure matching trace spans during a guarded rollout.
   • Use Sum or Average to create a numeric metric that aggregates a value from the filtered trace spans.
6. If you are configuring a numeric metric:
   i. Click the “Choose a numeric span attribute” field and type or select the span attribute that provides the numeric value you want to measure.
   ii. Select Include units and set the value to 0 or Exclude units that generate no events to choose how to handle units that do not generate a matching span during a rollout.
   iii. Enter a Unit of measure to describe the value you are measuring.
7. Use the “Metric definition” section to change the metric’s analysis method, analysis units, and success criteria as needed. To learn more, read Analysis method.
8. Enter a Metric Name.
9. (Optional) Add a Description.
10. (Optional) Add any Tags.
11. (Optional) Update the Maintainer.
12. Click Create.

Adding trace metrics to guarded rollouts

You select trace metrics for use with guarded rollouts in the same way you select other LaunchDarkly metrics. To learn more, read Creating guarded rollouts.

Original source

OpenTelemetry in server-side SDKs

This topic explains how to enable OpenTelemetry (OTel) in server-side SDKs, and how to display and use OpenTelemetry data in LaunchDarkly.

Looking for information on OpenTelemetry in client-side SDKs?

For information about using OpenTelemetry in LaunchDarkly client-side SDKs, read OpenTelemetry in client-side SDKs.

About OpenTelemetry

OpenTelemetry is an open source observability framework and toolkit designed to create and manage telemetry data such as traces, metrics, and logs. Because OpenTelemetry is vendor- and tool-agnostic, you can reuse OpenTelemetry instrumentation that already exists in your code to send OpenTelemetry data to LaunchDarkly.

We recommend enabling OpenTelemetry in LaunchDarkly server-side SDKs if you use the Observability, Experimentation, or Guarded rollouts features, or if you use third-party observability tools that support the OpenTelemetry framework.

If you want to implement OpenTelemetry features in new code, we recommend using the LaunchDarkly observability plugins to automate the process of adding flag evaluation metadata and forwarding telemetry data directly to LaunchDarkly endpoints. LaunchDarkly provides observability plugins for many client and server SDKs. To learn more, read Observability SDKs.

Using OpenTelemetry data in LaunchDarkly

You can use OpenTelemetry data in LaunchDarkly in two ways:

1. OpenTelemetry data ingestion: LaunchDarkly provides OpenTelemetry protocol (OTLP) HTTP and gRPC endpoints to support ingesting traces, metrics, and logs from your applications and from third-party application integrations. LaunchDarkly observability features help you search and visualize ingested OTel data, or configure alerts to respond to OTel signals. To learn about configuring OTel ingestion, read Observability SDKs and Observability Integrations. To learn about viewing ingested OTel data, read Observability.
2. LaunchDarkly metrics: LaunchDarkly automatically generates metrics from certain OpenTelemetry traces, and these metrics are available in the Experimentation and guarded rollouts features. When ingested trace data contains feature flag evaluation span events along with certain HTTP span attributes or error span events, LaunchDarkly correlates the data and creates metrics to use with guarded rollouts. To learn more, read OpenTelemetry autogenerated metrics.

You can also use any OTel traces that include feature flag span events to create custom LaunchDarkly metrics for use with experiments or guarded rollouts. To learn more, read Creating metrics from traces.

Sending OpenTelemetry data to LaunchDarkly

To configure and send OpenTelemetry data to LaunchDarkly when you are using LaunchDarkly server-side SDKs:

1. (Optional) Add flag evaluation information to OpenTelemetry spans in your application. To learn how, read the section for your SDK under Server-side SDKs, below.
   • This step is optional but highly recommended, as it means that flag-specific details are available when you review your OTel data in the LaunchDarkly UI.
   • You might choose to skip this step, at least temporarily, if you have already instrumented OpenTelemetry in your application and only want to send your existing OTel data to LaunchDarkly.
   • If you create spans using a LaunchDarkly server-side observability plugin, the SDK adds the required flag evaluation information automatically.
2. Configure your application or OpenTelemetry collector to use the LaunchDarkly OTel endpoints:
   • For HTTP, use https://otel.observability.app.launchdarkly.com:4318 or https://otel.observability.app.launchdarkly.com:443
   • For gRPC, use https://otel.observability.app.launchdarkly.com:4317.
   • If you use a LaunchDarkly server-side observability plugin, the SDK automatically sends telemetry to the default endpoints.
   • You may need to update your infrastructure to make sure your application can reach this domain. To learn more, read Accessing LaunchDarkly by domain.
3. (Optional) Include an active LaunchDarkly SDK key as a resource attribute in your telemetry data.
   • This step is required if you are using OpenTelemetry SDKs. To learn how, read Setting resource attributes, below.
   • This step is not required if you are using LaunchDarkly server-side SDKs. If you use LaunchDarkly SDKs, this is set automatically.
4. Send the OpenTelemetry data to LaunchDarkly. LaunchDarkly supports traces, metrics, and logs. Only traces data generates LaunchDarkly metrics. You can view metrics and logs in the LaunchDarkly UI when you use the Observability features.
   • You can use the OpenTelemetry Collector to pre-filter and aggregate this data. To learn how, read Configuring the collector, below.
   • Alternatively, you can send data directly from your application using OpenTelemetry SDKs.
   • LaunchDarkly server-side observability plugins automatically send telemetry to the default LaunchDarkly endpoints.

Setting resource attributes

[Expandable section]

Configuring the collector

[Expandable section]

Automatic attribute normalization

OpenTelemetry semantic conventions define standard attribute names for common concepts like HTTP requests, database operations, and messaging systems. These conventions evolve over time, and attribute names are occasionally renamed or restructured. For example, http.method was renamed to http.request.method and http.status_code was renamed to http.response.status_code. For more details on how attribute names change across OpenTelemetry versions, see the semantic conventions changelog.

LaunchDarkly automatically normalizes deprecated OpenTelemetry semantic convention attributes to their current equivalents when it ingests your telemetry data. This means you can send data using older attribute names, and LaunchDarkly will create the corresponding current attribute names for you. This normalization applies to all OpenTelemetry data LaunchDarkly ingests, whether from LaunchDarkly SDKs with tracing hooks enabled or from external OTLP endpoints.

This automatic normalization provides several benefits. LaunchDarkly ensures consistent attribute names across your telemetry data, even when services use different OpenTelemetry versions. You can search and filter using the current attribute names regardless of which version your services emit. You can also upgrade your instrumentation gradually without losing the ability to correlate data across services.

When viewing trace attributes in LaunchDarkly, the UI displays indicators showing which attributes have been remapped from deprecated conventions.

Server-side SDKs

LaunchDarkly processes metrics, logs, and traces from your OpenTelemetry data:

• LaunchDarkly processes OpenTelemetry metrics and logs and displays them in the LaunchDarkly UI under Observability
• LaunchDarkly processes two types of data from OpenTelemetry traces:
  • HTTP span attributes, including latency, 5xx occurrences, and other errors, where the span has or overlaps with another span that has at least one feature flag span event. This includes nested spans.
  • Exception span events that occur after a feature flag span event on the same trace. If the exception occurs before the feature flag event, LaunchDarkly does not capture it.

A feature flag span event is defined as any span event that contains a feature_flag.context.key attribute. LaunchDarkly ignores traces that do not include span events with this attribute.

In most server-side SDKs, the OTel traces are specific to the LaunchDarkly project and environment that you specify in the collector configuration, described above. In the .NET (server-side) and Node.js (server-side) SDKs, you can provide the LaunchDarkly client-side ID in the tracing hook. This enables you to send traces for multiple projects and environments using one collector.

The following sections describe, for each supported SDK, how to ensure your spans have compatible feature flag events.

Sending feature flag event data requires feature flag evaluation

The OpenTelemetry tracing hook automatically attaches feature flag event data to your OTel traces for you. Feature flag event data, including the feature_flag.context.key attribute, is only generated when your application uses the LaunchDarkly SDK to evaluate a feature flag.

This feature is available in the following SDKs:

• .NET (server-side)
• Go
• Java
• Node.js (server-side)
• PHP
• Python
• Ruby

.NET (server-side)

Use the .NET (server-side) SDK observability plugin for new applications

We recommend you use the .NET (server-side) SDK observability plugin for new application development or OpenTelemetry instrumentation. The observability plugin provides additional configuration options and automatically adds LaunchDarkly environment and flag attributes to OpenTelemetry spans.

The instructions that follow are provided for reference, for OpenTelemetry applications that were developed before the observability plugin was available.

[Expandable .NET (server-side) code sample]

Go

Use the Go SDK observability plugin for new applications

We recommend you use the Go SDK observability plugin for new application development or OpenTelemetry instrumentation. The observability plugin provides additional configuration options and automatically adds LaunchDarkly environment and flag attributes to OpenTelemetry spans.

The instructions that follow are provided for reference, for OpenTelemetry applications that were developed before the observability plugin was available.

[Expandable Go code sample]

Java

[Expandable Java code sample]

Node.js (server-side)

Use the Node.js (server-side) observability plugin for new applications

We recommend you use the Node.js (server-side) SDK observability plugin for new application development or OpenTelemetry instrumentation. The observability plugin provides additional configuration options and automatically adds LaunchDarkly environment and flag attributes to OpenTelemetry spans.

The instructions that follow are provided for reference, for OpenTelemetry applications that were developed before the observability plugin was available.

[Expandable Node.js (server-side) code sample]

PHP

[Expandable PHP code sample]

Python

[Expandable Python code sample]

Ruby

[Expandable Ruby code sample]

Original source

Analyze your experiments on the same trusted data your business already runs on, so results never come with an asterisk.

Your warehouse is the source of truth for your decisions. A year ago, we made it the source of truth for your experiments, too, bringing warehouse-native experimentation to Snowflake so teams could analyze experiments directly on the data they already trust, with no copies and no second version of the truth.

Since then, adoption has grown steadily, and we've learned a lot from teams running real experiments against their own warehouse data. Today, we're putting those lessons to work, with updates on two fronts:

• Wherever your data lives. Warehouse-native experimentation now runs on BigQuery, Databricks, and Redshift, alongside Snowflake, so you can run it on the warehouse you already use.
• Whatever your analysis demands. It now includes advanced statistical capabilities that were previously available only in hosted experimentation.

Wherever your data lives

Whichever warehouse your organization relies on, you can now experiment directly on your own data. With BigQuery, Databricks, and Redshift joining Snowflake, warehouse native experimentation gives you the same trusted experience, while keeping your sensitive data in your warehouse. LaunchDarkly only receives aggregated, de-identified experiment results to power reporting in our product.

And the workflow stays the same, no matter which warehouse you rely on:

1. LaunchDarkly syncs experiment exposure data into the warehouse via Data Export.
2. Metrics are defined from metric sources, which draw on the tables in your warehouse, and are computed directly against them.
3. Results surface back in LaunchDarkly for analysis and decision-making.

You define your metrics in LaunchDarkly, and they compute against the same governed data your team already trusts. No reconciling, no second version of the truth, and no asterisk on your results.

Whatever your analysis demands

Trusted data is only half of it. You also need the statistical rigor to act on results with confidence. Over the last few months, we've brought the depth of hosted experimentation (where LaunchDarkly stores your data and computes results on our own infrastructure) directly to your warehouse. These are a few of the capabilities we've shipped:

• Sequential Testing: Call experiments the moment they're conclusive, minimizing the false positives that come from peeking early.
• Multiple Comparisons Correction: Test many metrics and variations at once while keeping your false-positive risk under control, even as the comparisons add up.
• Adding metrics post-experiment start: Add metrics on the fly and see results immediately, without committing to a fixed set of metrics upfront.
• Result Segmentation: See how different user segments respond to your hypothesis, not just the aggregate.

Snowflake, our longest-running integration, goes a step further with metric winsorization and windowing for even finer control over how outliers and measurement windows shape your results. We'll also be rolling these features out to other warehouse integrations soon, and going forward, we're aiming to bring new capabilities to every supported warehouse at the same time.

For product teams, this means faster experimentation cycles. For data teams, metrics stay governed in the systems they already manage.

Get started

Warehouse-native experimentation is available today on BigQuery, Databricks, Redshift, and Snowflake. Set it up on the warehouse you already use:

• BigQuery
• Databricks
• Redshift
• Snowflake

New to warehouse-native experimentation? Request a demo and we'll walk you through running your first experiment on your own data.

Original source

LaunchDarkly AI SDK (server-side) for .NET

This AI SDK is in pre-release and not subject to backwards compatibility guarantees. The API may change based on feedback.

Pin to a specific minor version and review the changelog before upgrading.

The LaunchDarkly AI SDK (server-side) for .NET is designed primarily for use in multi-user systems such as web servers and applications. It follows the server-side LaunchDarkly model for multi-user contexts. It is not intended for use in desktop and embedded systems applications.

Currently there is no client-side AI SDK for .NET. If you're interested, please let us know by filing an issue!

LaunchDarkly overview

LaunchDarkly is a feature management platform that serves trillions of feature flags daily to help teams build better software, faster. Get started using LaunchDarkly today!

Supported .NET versions

This version of the AI SDK is built for the following targets:

• .NET 8.0: runs on .NET 8.0 and above (including higher major versions).
• .NET Framework 4.6.2: runs on .NET Framework 4.6.2 and above.
• .NET Standard 2.0: runs in any project that is targeted to .NET Standard 2.x rather than to a specific runtime platform.

The .NET build tools should automatically load the most appropriate build of the SDK for whatever platform your application or library is targeted to.

Getting started

Refer to the SDK documentation for instructions on getting started with using the SDK.

Signing

The published version of this assembly is digitally signed with Authenticode and strong-named. Building the code locally in the default Debug configuration does not use strong-naming and does not require a key file. The public key file is in this repository at LaunchDarkly.pk as well as here:

Public Key:
0024000004800000940000000602000000240000525341310004000001000100f121bbf427e4d7edc64131a9efeefd20978dc58c285aa6f548a4282fc6d871fbebeacc13160e88566f427497b62556bf7ff01017b0f7c9de36869cc681b236bc0df0c85927ac8a439ecb7a6a07ae4111034e03042c4b1569ebc6d3ed945878cca97e1592f864ba7cc81a56b8668a6d7bbe6e44c1279db088b0fdcc3552f746b4

Public Key Token: f86add69004e6885

Learn more

Read our documentation for in-depth instructions on configuring and using LaunchDarkly. You can also head straight to the complete reference guide for this SDK.

The authoritative description of all types, properties, and methods is in the generated API documentation.

Contributing

We encourage pull requests and other contributions from the community. Check out our contributing guidelines for instructions on how to contribute to this SDK.

Verifying build provenance with the SLSA framework

LaunchDarkly uses the SLSA framework (Supply-chain Levels for Software Artifacts) to help developers make their supply chain more secure by ensuring the authenticity and build integrity of our published packages. To learn more, see the provenance guide.

About LaunchDarkly

• LaunchDarkly is a continuous delivery platform that provides feature flags as a service and allows developers to iterate quickly and safely. We allow you to easily flag your features and manage them from the LaunchDarkly dashboard. With LaunchDarkly, you can:
  • Roll out a new feature to a subset of your users (like a group of users who opt-in to a beta tester group), gathering feedback and bug reports from real-world use cases.
  • Gradually roll out a feature to an increasing percentage of users, and track the effect that the feature has on key metrics (for instance, how likely is a user to complete a purchase if they have feature A versus feature B?).
  • Turn off a feature that you realize is causing performance problems in production, without needing to re-deploy, or even restart the application with a changed configuration file.
  • Grant access to certain features based on user attributes, like payment plan (eg: users on the ‘gold’ plan get access to more features than users in the ‘silver’ plan). Disable parts of your application to facilitate maintenance, without taking everything offline.
• LaunchDarkly provides feature flag SDKs for a wide variety of languages and technologies. Read our documentation for a complete list.
• Explore LaunchDarkly
  • launchdarkly.com for more information
  • docs.launchdarkly.com for our documentation and SDK reference guides
  • apidocs.launchdarkly.com for our API documentation
  • blog.launchdarkly.com for the latest product updates

Original source

This topic explains how to use prompt snippets to create, manage, and reuse prompts across AgentControl config variations.

Prompt snippets are reusable, versioned pieces of prompt text that you manage as standalone resources in LaunchDarkly. Snippets let you define prompt content one time and reuse it across configs and variations.

Create and manage snippets in the Library, and reference them inside variation messages to reuse prompt content. The centralized snippets library helps you maintain consistency across config variations and quickly find a snippet you wish to use again. Snippet versioning lets you update prompt content without losing earlier versions of each snippet.

Use prompt snippets to:

• Reduce duplicated prompts across config variations
• Define and reuse prompt content such as tone, formatting, or governance language
• Identify which configs and variations use a specific snippet
• Track updates to prompt content by version
• Maintain prompt consistency across teams

How prompt snippets work

Each snippet has a name, key, and prompt content, along with versioning information and metadata such as tags, description, and maintainer. You reference snippets using their key and version. When LaunchDarkly evaluates a variation, it resolves these references and returns the fully assembled prompt.

These fields determine how you reference, update, and organize snippets:

• The key identifies the snippet and lets you reference it in variation messages
• The version lets you update prompt content without affecting existing variations
• Metadata helps you organize snippets across your project

Snippets are included as part of a config variation’s message body. When LaunchDarkly evaluates a variation, it replaces the reference with the snippet content and returns the fully assembled prompt to your application. A single message can include multiple snippet references along with other prompt content.

Core functions of prompt snippets

Prompt snippets’ core functions include:

• Editing a snippet creates a new version instead of modifying the existing one
• Config variations reference a specific snippet version
• Updating a snippet does not automatically update variations
• Snippets cannot reference other snippets

These behaviors let you update prompt content safely, review changes before applying them, and keep existing variations stable until you choose to update them.

You manage prompt snippets from the Library under AI. The snippets library provides a centralized place to create, view, and update reusable prompts across your project.

View snippets

Use the snippets list in the library to browse, search, and manage snippets across your project.

The snippets list provides a centralized view of all prompt snippets, so you can review existing content, identify reusable snippets, and understand how snippets are organized.

From the snippets list, you can:

• View all snippets in your project in a centralized list
• Search for snippets by name, key, or content
• View snippet details, including versions and usage

Create a snippet

Create a snippet to define reusable prompt content that you can reference across config variations. Use snippets for shared content such as tone, formatting, or common instructions.

To create a snippet from the Snippets tab:

1. Navigate to your project.
2. In the left sidebar, click Agents. The AgentControl menu appears.
3. Click Library.
4. Select the Snippets tab. The snippets list appears.
5. Click New snippet. The “Save snippet” dialog appears.
6. Enter a name for the snippet. The snippet’s key auto-populates based on the name.
7. (Optional) Add a description to help others understand when to use this snippet.
8. In the “Body” field, enter the prompt text you want to save as a snippet.
9. (Optional) Click the tag icon to add tags to organize snippets across your project.
10. Click Create.

New snippets appear in the library’s “Snippets” tab.

You can also create a snippet inline while editing a variation message. To create a snippet this way, read Use snippets in config variations.

Snippet versioning

New snippets start at version 1. When you edit a snippet, LaunchDarkly creates a new version of that snippet instead of modifying the existing one. Previous versions remain available, so you can refer back to them or use them again if needed.

To edit a snippet, click the snippet in the snippets list. This opens the editing page, where you can update the snippet’s name, prompt content, and metadata.

You must manually update config variations to use new snippet versions.

Config variations continue to use the version of a snippet you assigned until you update them to use a different version. For example, if a config variation uses version 1 of a snippet and you update the snippet, creating version 2, the config will continue to use version 1 until you specify otherwise.

Manage snippet usage and versions to understand where snippets are used and control how updates are applied across config variations.

Each snippet includes version history and usage information. Use this information to review how a snippet is used before updating variations to a new version. From the snippet list, you can view all config and variations that reference the snippet and see which version each variation is using. This helps you understand where a snippet is in use before making changes.

From a config variation, you can compare the current snippet version to the latest version and update the variation to use a newer version. Use this workflow to review how changes affect variations and update them individually as needed. This approach lets you apply changes intentionally and keep existing variations stable until you update them.

Compare snippet versions

You can view and compare versions of a snippet from the snippets list. Here’s how:

1. From the snippets list, find the snippet with versions you wish to view.
2. Click the version number in the “Versions” column. The “Version history” window appears.
3. Click Compare and select two versions to see a code diff of the changes between them.

Delete a snippet

Delete a snippet when you no longer need it and it is not used by any config variations. Here’s how:

1. From the snippets list, find the snippet you wish to delete.
2. Click snippet in the snippets list. The snippet detail page appears.
3. Click Delete. The snippet is deleted.

Before you delete a snippet, remove any references to it from config variations. LaunchDarkly prevents you from deleting snippets that are still in use.

Use snippets in config variations

You can use snippets in config variations in two ways:

• Attach existing snippets to a variation
• Create new snippets directly in variation messages

Attach snippets to a variation

Attach snippets from the variation editor to include reusable prompt content in a variation message.

To attach snippets to a variation:

1. Navigate to your config.
2. Select the Variations tab and open a variation.
3. There are three ways to use snippets in messages:
   • Click the “Load prompt snippet” button in the variation message body toolbar to open search and find a snippet.
   • In the variation message body, use the {{snippet.}} selector to open search and find a snippet.
   • Reference a snippet directly using {{snippet.example-snippet-key#version-number}}.
4. Click Review and save.

Create inline snippets in messages

You can also create a snippet directly while editing a variation message.

To create snippets in a message:

1. Open a variation and begin editing the message content.
2. Click “Save as” in the variation message body toolbar.
3. Choose to either save the message body text as a New snippet or New version of existing snippet.

Behavior for in-use variations

You can use prompt snippets in variations that are part of active experiments or guarded rollouts.

If a variation that uses a snippet is part of an active experiment or guarded rollout, updating the snippet version may be blocked. LaunchDarkly blocks these updates to prevent changes that could affect experiment results or rollout behavior. If an update is blocked, LaunchDarkly provides details about the active usage, including which experiment or rollout is affected. Use this information to determine when to update the variation after it is no longer part of an active experiment or rollout.

Audit logs and access control

Prompt snippets use the same access control model as AgentControl configs, so you can manage access using existing roles and permissions. Access to create, update, and delete snippets follows the same permissions as other AgentControl resources.

LaunchDarkly records changes to snippets and their attachments in audit logs. These records show when snippets are created, updated, or attached to variations.

Permissions align with existing AgentControl actions. Audit log entries are created for snippet creation, updates, and changes to snippet attachments on variations.

Use audit logs to review changes to snippets and their attachments. This helps you track how snippets change over time and maintain control over changes across AgentControl config variations.

Original source

AI insights

This topic explains how to use AI insights to understand the impact of AgentControl configs by analyzing metrics across your project. AI insights provides a project-level view across configurations, models, and targeting rules. Use it to identify changes, compare configurations, and determine which configurations, models, or variations are driving results and their impact on performance and outcomes.

Use AI insights to:

• Detect changes in cost, usage, and quality metrics
• Identify regressions after updates
• Compare performance across configs, models, and providers
• Analyze trends over time and understand how changes affect performance
• Investigate specific configs and recent changes
• Establish consistent, organization-wide practices for evaluating AI model performance

To analyze performance for a single config, use the Monitoring tab.

AI insights page

The Insights page provides a unified, aggregated view of metrics across your configs. It includes time series charts, summary metrics, and a configuration-level table so you can review performance, identify changes, and understand their impact. All components reflect the selected time range and filters.

Use the controls at the top of the page to select a metric view, group results, filter configurations, and adjust the time range.

To open the Insights page:

1. Navigate to your project.
2. In the left sidebar, expand AI, then select Insights.

Use this page to monitor performance and investigate changes across your configs. A typical workflow includes:

1. Use the trends view to analyze changes over time and understand how updates affect performance.
2. Review quick stats to identify changes in key metrics.
3. Use the configs and variations table to compare configurations and identify which require further investigation. This helps you understand what changed and decide whether to act.

Alerts highlight changes in key metrics so you can identify where to investigate without reviewing each configuration individually.

Trends

The trends view displays metrics as time series charts so you can compare performance and understand how metrics change over time across configurations. You can group results by config, model, provider, or agent graph, including multi-agent workflows. You can analyze metrics such as token usage, latency, satisfaction, error rate, and evaluation scores over time.

Use the trends view to track changes over time and understand their impact on performance. Apply filters to focus your analysis on specific configurations or models.

You can also review changes to prompts, models, and targeting rules alongside performance metrics to understand how updates affect cost, latency, or satisfaction.

Quick stats

Quick stats summarize key metrics, including active configs and experiments, average satisfaction, and cost.

Use quick stats to identify changes in these metrics and determine where to investigate further. These changes may reflect shifts in performance and impact.

Configs and variations

The configs and variations table shows metrics for each config, including generations, token usage, satisfaction, latency, error rate, model and provider, and experiment status.

Use this table to compare configurations and identify differences in metrics that require further investigation.

Alerts

AI insights includes system-generated alerts for changes in key metrics. Use alerts to determine which configurations are driving changes and require further investigation.

Alerts provide a proactive way to monitor performance by highlighting configurations with recent changes so you can focus your investigation. Each alert includes the affected config and details about the change.

Instrumentation requirements

AI insights depends on metrics recorded from your application.

To populate insights, use a LaunchDarkly AI SDK to evaluate configs and record generation metrics such as latency, token usage, success, and error. You can also record evaluation metrics using judges.

If your application does not record metrics, the Insights page may not display data.

Choose a view

Use the following guidance to select the appropriate view:

• Use AI insights to monitor metrics and identify changes across configurations.
• Use the Monitoring tab to analyze performance for a single config and its variations.

These views support different levels of analysis, from investigating a single config to understanding patterns across configurations.

Original source