Skip to content
Frontier Leaf FL markFLFrontierLeaf

A field journal of new AI developments.

Model updates

AWS VP Colleen Aubrey advises using existing business metrics to measure AI success

Enterprise leaders should assess AI initiatives through traditional business outcomes rather than creating new artificial intelligence-specific key performance indicators.

Depth
1 outlet · newest source 232h ago
AWS VP Colleen Aubrey advises using existing business metrics to measure AI success

Key points

  • AWS executive Colleen Aubrey argues that AI success should be measured using existing business metrics.
  • Amazon Connect serves as a primary example of applying deep operational experience to build successful AI applications.
  • The Connect family expanded into Customer, Talent, Health and Decisions products based on lessons from its initial voice quality focus.

Colleen Aubrey, senior vice president of Applied AI Solutions at AWS, stated that enterprises should stop inventing new metrics for artificial intelligence initiatives. According to a report by SiliconANGLE, she argues that the same traditional business metrics used for other IT projects are sufficient for measuring AI success.She suggests that companies should look at their regular business numbers to see if AI projects are working well.Aubrey contends that distinct AI KPIs are unnecessary, advocating for the integration of AI performance measurement into standard business metric frameworks.

Evidence: reportedAWS’ Colleen Aubrey: Stop inventing new AI metrics — it’s your business metrics that matter

aifleet-siliconangle

The evidence

Aubrey possesses more than twenty years of experience at Amazon, where she previously built customer-facing products for millions of sellers and advertisers. She now applies these lessons to help enterprises move their AI programs out of pilot mode. Her approach involves discarding the concept of a separate AI return-on-investment calculation in favour of standard business outcome tracking.

The decision

The core implication for IT leaders is to align AI measurement with established operational goals. Aubrey noted that she worries when people attempt to create new metrics to measure AI impact. She emphasised that it is the existing business metrics that truly matter for determining whether an initiative has been successful.

The takeaway

Organisations face a challenge in deciding what to measure when determining if an AI initiative was successful. Aubrey suggests that understanding AI ROI requires thinking in terms of business metrics rather than creating new artificial intelligence-specific key performance indicators. This approach mirrors the playbook she built for the Amazon Connect family of products.

Building on this

Developers can build applications by leveraging the documented success pattern of Amazon Connect. The team achieved success by choosing a sharp wedge, focusing on outstanding voice quality on the AWS backbone rather than chasing feature parity. They also utilised deep operational experience from running Amazon’s customer service function. This blueprint now supports the expanded Connect family, including Connect Customer, Connect Talent, Connect Health and Connect Decisions. Developers should focus on knowing their customers and leveraging operational experience rather than building broad feature sets.

Why it matters

  • IT buyers can avoid the cost and complexity of developing new AI-specific measurement frameworks by using existing business metrics.
  • The Connect example demonstrates that focusing on a specific operational strength, such as voice quality, can be more effective than pursuing feature parity.
  • Enterprises moving out of AI pilot purgatory can apply the lessons from Amazon’s twenty-year cloud infrastructure experience to their own AI programs.
  • The expansion of the Connect family into Customer, Talent, Health and Decisions shows how a single successful application can scale into multiple verticals.

Topics: AWS, AI metrics, Colleen Aubrey, Amazon Connect, enterprise AI

Updated

Then / now / next

Then

Many organisations struggled to decide what to measure for AI initiatives.

Now

AWS executive Colleen Aubrey advises using existing business metrics to determine AI success.

Next

IT leaders can apply the Connect playbook to move their AI programs out of pilot mode.

Story timeline

  1. Then Many organisations struggled to decide what to measure for AI initiatives.
  2. 06 Nov 2026 AWS’ Colleen Aubrey: Stop inventing new AI metrics — it’s your business metrics that matter
  3. Now AWS executive Colleen Aubrey advises using existing business metrics to determine AI success.
  4. Next IT leaders can apply the Connect playbook to move their AI programs out of pilot mode.

For readers

Business

  • Use existing business metrics to measure AI success instead of creating new KPIs.
  • Focus on operational strengths rather than chasing feature parity with competitors.

Developers

  • Leverage deep operational experience to build successful AI applications.
  • Start with a sharp wedge, such as voice quality, before expanding features.

Investors

  • AI initiatives should be evaluated based on traditional business outcomes.
  • Amazon Connect's success demonstrates the value of operational expertise in AI.

Ask the article

Local keyword search of this article; no generated answers.

Did this article give you the evidence you needed?

No vote saved on this device.
Your choice stays on this device. No public tally.

Questions readers ask

What metrics should enterprises use to measure AI success?

Enterprises should use their existing business metrics rather than inventing new artificial intelligence-specific metrics.

What was the key to Amazon Connect's initial success?

The team focused on outstanding voice quality on the AWS backbone and utilised deep operational experience from Amazon's customer service function.

How many years of experience does Colleen Aubrey have at Amazon?

Colleen Aubrey has more than twenty years of experience at Amazon.

Connected stories

This storyDatabricks Genie agent executes unsupervised multi-step tasks using platform ontologyDatabricks Genie agent executes Dartmouth provost resigns over undisclosed AI use in academic writingDartmouth provost resigns over uHugging Face releases Granite 4.1 LLMON preview models to separate instructions from dataHugging Face releases Granite 4.Apollo Economist: AI Fails to Boost Profits or Productivity for Non-Tech FirmsApollo Economist: AI Fails to Bo

Reading receipt

  • AWS executive Colleen Aubrey argues that AI success should be measured using existing business metrics.
  • Amazon Connect serves as a primary example of applying deep operational experience to build successful AI applications.
  • The Connect family expanded into Customer, Talent, Health and Decisions products based on lessons from its initial voice quality focus.

Still open: How specifically should different industries adapt the Connect playbook to their unique operational contexts?

Still open: What specific business metrics does Aubrey recommend for different types of AI applications?

Time spent: 0:00 · Depth: Full

Sources

  1. SiliconANGLE AI

Related stories

Background reading