ABsmartly Release Notes

The main focus of this release is the introduction of our new ownership model, specifically team-level roles and permissions. We’ve also included a number of small improvements and bug fixes to improve usability and platform flexibility.

Team-Level Roles and Permissions

You can now assign users to teams and grant them team-level roles — such as Team contributor, Team viewer, or Team admin — that define how they interact with experiments, metrics, goals, templates, and features owned by that team.

This release is an intermediate step toward the full rollout of our new ownership model. Unless you explicitly add users to teams and assign them roles, there should be no changes in behavior and everything will continue working as before.

If you encounter any issues or inconsistencies related to roles and permissions, please contact us.

What's New

We have introduced four new built-in roles:

• Team Viewer
  Can view all experiments, features, templates, metrics, and goals owned by the team.
• Team Contributor
  Can do everything a Team Viewer can, plus create/edit experiments, features, templates, metrics, and goals within the team and manage the life cycle (start, stop) of those experiments and features.
• Team Admin
  Can do everything a Team Contributor can, and also manage the team — including adding/removing members, assigning roles, editing metadata, etc.
• Base User
  This will replace the current default "User" role. Following the principle of least privilege, this new role provides read-only access to most components, but does not include access to experiments, features, templates, metrics, or goals — which will be managed at the team level.

These roles are immutable, but custom roles (e.g. "Metric Owner", "Experiment Reviewer") will be supported in future releases to meet your unique needs.

All existing global roles remain valid and can be assigned at the Global Team level, which represents your organisation. Team role permissions are inherited: roles granted at any level apply to all child teams.

If you haven't started yet, you can begin managing your team hierarchy at: /settings/teams

If you have any questions or want to learn how to get the most from this model, feel free to reach out!

Other Notable Improvements and Fixes

• Ignore exposure events
  You can now tag exposure events with a __ignore attribute to have them excluded from experiment analysis. This is useful for excluding internal users from your experiments.
• Experiment export fix
  Fixed an issue where re-exporting experiment data for an existing export request would break the export process.
• Metrics list UI cleanup
  Improved the layout of the metrics list page to avoid overly long description text dominating the screen.

Upcoming Change: Fair Usage Policy Update

In our next release, we plan to introduce a 2KB limit on the JSON payload sent with goal and exposure events. This change is needed to keep a better control over cost and performance of the ABsmartly platform. Events whose payload is above the threshold will not be processed by the ABsmartly platform. While this limit is way over what most of our customers are currently using, we will query past events and reach out to you separately if we see that you have used larger payloads in the past so we can better prepare for this upcoming change.

If you have any concerns or use cases that require larger payloads, please let us know as soon as possible.

Questions or Feedback?

As always, if you have questions about this release or anything else, don't hesitate to get in touch.

This release includes a number of small but meaningful improvements and bug fixes. While the team continues to work on major backend updates, this update focuses on performance optimisations and fixes that improve clarity and usability.

Performance Improvements

To improve platform performance and reduce unnecessary load, we have made the following changes:

• Feature flag: We no longer fetch data older than 30 days by default
• Long-running experiments: We have reduced how frequently data is updated for experiments running longer than 60 days

These changes help improve performance without impacting the accuracy or freshness of recent results.

Notable Bug Fixes

• Table sorting restored: Sorting on list tables has been re-enabled after being unintentionally removed in a previous release
• Improved clarity for GST experiments: To avoid confusion during interim analyses, the GST data switch is now disabled until the experiment is complete. Previously, we showed non-GST-adjusted data when no boundary had been crossed, which led to misinterpretation.
• Caching issues on the Activity tab fixed: We addressed some caching issues on the activity tab where data would not update until the page was refreshed.
• Add to favourites fixed: Resolved an issue where adding experiments to favourites did not always work.

Please let us know if you have any questions or feedback — and stay tuned for more significant changes coming soon!

This release includes a number of small but meaningful improvements and bug fixes. While the team continues to work on major backend updates, this update focuses on performance optimisations and fixes that improve clarity and usability.

Performance Improvements

To improve platform performance and reduce unnecessary load, we have made the following changes:

• Feature flag: We no longer fetch data older than 30 days by default
• Long-running experiments: We have reduced how frequently data is updated for experiments running longer than 60 days

These changes help improve performance without impacting the accuracy or freshness of recent results.

Notable Bug Fixes

• Table sorting restored: Sorting on list tables has been re‑enabled after being unintentionally removed in a previous release
• Improved clarity for GST experiments: To avoid confusion during interim analyses, the GST data switch is now disabled until the experiment is complete. Previously, we showed non-GST-adjusted data when no boundary had been crossed, which led to misinterpretation.
• Caching issues on the Activity tab fixed: We addressed some caching issues on the activity tab where data would not update until the page was refreshed.
• Add to favourites fixed: Resolved an issue where adding experiments to favourites did not always work.

Please let us know if you have any questions or feedback — and stay tuned for more significant changes coming soon!

This release includes a number of small but meaningful improvements and bug fixes. While the team continues to work on major backend updates, this update focuses on performance optimisations and fixes that improve clarity and usability.

Performance Improvements

To improve platform performance and reduce unnecessary load, we have made the following changes:

• Feature flag: We no longer fetch data older than 30 days by default
• Long-running experiments: We have reduced how frequently data is updated for experiments running longer than 60 days

These changes help improve performance without impacting the accuracy or freshness of recent results.

Notable Bug Fixes

• Table sorting restored: Sorting on list tables has been re-enabled after being unintentionally removed in a previous release
• Improved clarity for GST experiments: To avoid confusion during interim analyses, the GST data switch is now disabled until the experiment is complete. Previously, we showed non-GST-adjusted data when no boundary had been crossed, which led to misinterpretation.
• Caching issues on the Activity tab fixed: We addressed some caching issues on the activity tab where data would not update until the page was refreshed.
• Add to favourites fixed: Resolved an issue where adding experiments to favourites did not always work.

Please let us know if you have any questions or feedback — and stay tuned for more significant changes coming soon!

This release includes a number of small but meaningful improvements and bug fixes. While the team continues to work on major backend updates, this update focuses on performance optimisations and fixes that improve clarity and usability.

Performance Improvements

To improve platform performance and reduce unnecessary load, we have made the following changes:

• Feature flag: We no longer fetch data older than 30 days by default
• Long-running experiments: We have reduced how frequently data is updated for experiments running longer than 60 days

These changes help improve performance without impacting the accuracy or freshness of recent results.

Notable Bug Fixes

• Table sorting restored: Sorting on list tables has been re-enabled after being unintentionally removed in a previous release
• Improved clarity for GST experiments: To avoid confusion during interim analyses, the GST data switch is now disabled until the experiment is complete. Previously, we showed non-GST-adjusted data when no boundary had been crossed, which led to misinterpretation.
• Caching issues on the Activity tab fixed: We addressed some caching issues on the activity tab where data would not update until the page was refreshed.
• Add to favourites fixed: Resolved an issue where adding experiments to favourites did not always work.

Please let us know if you have any questions or feedback — and stay tuned for more significant changes coming soon!

This release brings several UX improvements and bug fixes aimed at making experimentation and collaboration smoother and more reliable. We’ve made owner management more flexible, improved graph interactions, and fixed key issues around estimations and data tables.

Improved Owner Filters and Flexibility

• An owner can now be a single user or a team.
• The Owner filter now accepts both users and teams when filtering, making results more inclusive.

Bug Fixes and Usability Improvements

• MDE and Max Runtime Automatic Estimation Refresh: Estimates now update automatically when the standard deviation changes. We have also improved estimation precision.
• Graph Zoom Behavior: Y-axis now updates correctly when zooming in, giving you a clearer view of trends.
• GST Estimate Accuracy: The next GST analysis estimate now correctly respects the minimum time between analyses.

Please let us know if you have any questions or feedback — we're always happy to hear from you.

This release introduces a brand-new notification system to help teams stay on top of what matters, along with the official launch of the improved Metrics Overview tab. These updates are designed to make your workflow more informed, timely, and collaborative.

New Feature: Built-In Notifications

Never miss a key event again. Our new built-in notification system ensures you're always up to date with what's happening on the platform.

• Team assignments: Get notified when you're added to or removed from a team
• Experiment/feature lifecycle updates: Get alerts when an experiment starts, stops, is put Full On, or when a feature is turned on/off.

More alert types will be added in upcoming releases — stay tuned!

New Feature: Impact per Day Estimates

We've added new Impact per Day estimates to all relevant metric types in the experiment overview to help you better understand the real-world significance of your experiment results. This gives you a clearer picture of the daily impact your changes could have when rolled out, making it easier to evaluate their practical value.

Metrics Overview Tab Out of Beta

Thanks to all your feedback, the new Metrics Overview tab is now officially out of beta.

• Cleaner layout and better tooltips and timestamps
• Consolidation of GST and Fixed Horizon experiment results in a single table
• Display Impact per Day for all relevant metrics
• More actionable information at a glance

These changes are designed to help you make faster, more informed decisions when reviewing experiment results.

Please let us know if you have any questions or would like to discuss future features with us.

Our latest release introduces new features to support better collaboration, data access, and metric/goal discoverability. With foundational Team Management features and a faster way to download experiment data, this update sets the stage for a more streamlined and scalable experimentation experience.

New Feature: Team Management (Foundations)

Teams are now much more than just metadata. You can properly define and manage teams within the platform. This update allows you to map your internal organisational structure directly into the product by:

• Creating and managing teams
• Assigning users to teams
• Structuring teams hierarchically (e.g. parent and child teams)

This is the first step toward upcoming improvements around ownership, permissions, and team-level collaboration. Go to Settings > Teams to get started with managing your team and inviting your team members. If you already have teams defined in ABsmartly, you can simply move them to the right place in your org structure. Check our wiki page for more information on creating and managing teams in ABsmartly.

New Feature: One-Click Experiment Data Export

We added an Export data button to the experiment overview page, allowing you to export participant exposures and goal event data for that experiment with a single click.

• No need to navigate through the events page or APIs
• Directly download all experiment-related data with a single click
• Makes it easier for teams to access raw data for custom analysis or deep dives

From the experiment overview page:

• Click on ... and select Export data.
• You will get a confirmation that the request is in progress. The process might take a while depending on the size of the experiment.
• You will get a notification on the experiment activity page once the export file is ready.
• The export file will be available for 30 days once it has been created.
  Only users with the "Events:List" permission can trigger the export and download the file.

Improved Metrics & Goals List Pages

Improve searchability and filtering (archived vs. unarchived)

• Improve searchability and filtering (archived vs unarchived)
• Help you quickly identify and select the metrics and goals that matter

Other Fixes & Improvements

This release also includes several smaller enhancements and bug fixes to improve overall platform stability and performance.

Feedback and Next Steps

As always, we value your feedback, so please share with us what you would like to see improve in the future.

This release focuses on backend improvements and foundational work to support future features. While there are few visible changes for you today, these updates improve platform stability and pave the way for upcoming capabilities. We still managed to squeeze a few improvements to the Velocity report and to the Experiment Dashboard redesign.

Velocity Report

• View the list of experiments: Our most‑requested Velocity Report enhancement: you can now click any chart in the Velocity Report to view the underlying experiments.
• Early stop vs Early full-on: The Experiments not completed graph now shows the breakdown between the early full-on and early stops.

Experiment Dashboard Redesign

Our new Experiment Dashboard redesign is still in progress and available for preview. The main goal is to consolidate how Fixed Horizon and Group Sequential experiments are shown and to make it easier for everyone to make quality decisions. In this release, we added GST data to the metrics table. Please have a look and let us know what you think.

• How to see it? Within the dashboard of any existing experiment you will have a 'Load new UI' button which will enable you to toggle between the old and the new view.
• What’s next? Give us your feedback below or send it to [email protected].

Feedback and Next Steps

As always we value your feedback so please share with us what you would like to see improve in the future. We have some really exciting new features coming your way in the next few releases so stay tuned.

In this release, we focused mainly on improvements to the Velocity and Decisions Report. All those improvements are driven by feedback we received from you, so keep them coming. Here are some of the most important updates.

Velocity Report

• Aggregate data per quarter: To help you run your quarterly experimentation reports we made it possible to aggregate the data per quarter (previously you could only aggregate per year, month or week).
• Aborted reasons report: We introduced a new Aborted Reasons report. This report breaks down all aborted experiments (experiments stopped early) by their aborted reason which is captured when the experiment is stopped. This new report will make it easier to understand bottlenecks and current issues in your experimentation program.
• Experiment completed: We fixed an issue where we were over-reporting the number of completed experiments.

Stay tuned: One of the main feature requests we received for the Velocity Report is to be able to open the list of experiments in each report. We are hoping to introduce this feature in one of our upcoming releases.

Decisions Report

Earlier this month we launched the new Decisions Reports (Beta). In this release we are introducing a few improvements to provide better insights into each decision.

• Secondary & Guardrail metrics: Like with the primary metric, you can now see the experiment results on the secondary and guardrail metrics. This helps understand the rationale behind each decision.
• Health checks: We are now surfacing any health check violations in the reports so you can get better insights into the quality of each decision.

Feedback & Next Steps

As always we value your feedback so please share with us what you would like to see improve in the future. We have some really exciting new features coming your way in the next few releases so stay tuned.