
Key points
- AWS executive Colleen Aubrey argues that AI success should be measured using existing business metrics.
- Amazon Connect serves as a primary example of applying deep operational experience to build successful AI applications.
- The Connect family expanded into Customer, Talent, Health and Decisions products based on lessons from its initial voice quality focus.
Colleen Aubrey, senior vice president of Applied AI Solutions at AWS, stated that enterprises should stop inventing new metrics for artificial intelligence initiatives. According to a report by SiliconANGLE, she argues that the same traditional business metrics used for other IT projects are sufficient for measuring AI success.reportedShe suggests that companies should look at their regular business numbers to see if AI projects are working well.Aubrey contends that distinct AI KPIs are unnecessary, advocating for the integration of AI performance measurement into standard business metric frameworks.
Evidence: reported
AWS’ Colleen Aubrey: Stop inventing new AI metrics — it’s your business metrics that matteraifleet-siliconangle
The evidence
Aubrey possesses more than twenty years of experience at Amazon, where she previously built customer-facing products for millions of sellers and advertisers. She now applies these lessons to help enterprises move their AI programs out of pilot mode. Her approach involves discarding the concept of a separate AI return-on-investment calculation in favour of standard business outcome tracking.
The decision
The core implication for IT leaders is to align AI measurement with established operational goals. Aubrey noted that she worries when people attempt to create new metrics to measure AI impact. She emphasised that it is the existing business metrics that truly matter for determining whether an initiative has been successful.
The takeaway
Organisations face a challenge in deciding what to measure when determining if an AI initiative was successful. Aubrey suggests that understanding AI ROI requires thinking in terms of business metrics rather than creating new artificial intelligence-specific key performance indicators. This approach mirrors the playbook she built for the Amazon Connect family of products.
Building on this
Developers can build applications by leveraging the documented success pattern of Amazon Connect. The team achieved success by choosing a sharp wedge, focusing on outstanding voice quality on the AWS backbone rather than chasing feature parity. They also utilised deep operational experience from running Amazon’s customer service function. This blueprint now supports the expanded Connect family, including Connect Customer, Connect Talent, Connect Health and Connect Decisions. Developers should focus on knowing their customers and leveraging operational experience rather than building broad feature sets.
Why it matters
- IT buyers can avoid the cost and complexity of developing new AI-specific measurement frameworks by using existing business metrics.
- The Connect example demonstrates that focusing on a specific operational strength, such as voice quality, can be more effective than pursuing feature parity.
- Enterprises moving out of AI pilot purgatory can apply the lessons from Amazon’s twenty-year cloud infrastructure experience to their own AI programs.
- The expansion of the Connect family into Customer, Talent, Health and Decisions shows how a single successful application can scale into multiple verticals.
Topics: AWS, AI metrics, Colleen Aubrey, Amazon Connect, enterprise AI
Updated
Then / now / next
Many organisations struggled to decide what to measure for AI initiatives.
AWS executive Colleen Aubrey advises using existing business metrics to determine AI success.
IT leaders can apply the Connect playbook to move their AI programs out of pilot mode.
Story timeline
- Then Many organisations struggled to decide what to measure for AI initiatives.
- 06 Nov 2026 AWS’ Colleen Aubrey: Stop inventing new AI metrics — it’s your business metrics that matter
- Now AWS executive Colleen Aubrey advises using existing business metrics to determine AI success.
- Next IT leaders can apply the Connect playbook to move their AI programs out of pilot mode.
For readers
Business
- Use existing business metrics to measure AI success instead of creating new KPIs.
- Focus on operational strengths rather than chasing feature parity with competitors.
Developers
- Leverage deep operational experience to build successful AI applications.
- Start with a sharp wedge, such as voice quality, before expanding features.
Investors
- AI initiatives should be evaluated based on traditional business outcomes.
- Amazon Connect's success demonstrates the value of operational expertise in AI.
Ask the article
Local keyword search of this article; no generated answers.Did this article give you the evidence you needed?
Your choice stays on this device. No public tally.Questions readers ask
What metrics should enterprises use to measure AI success?
Enterprises should use their existing business metrics rather than inventing new artificial intelligence-specific metrics.
What was the key to Amazon Connect's initial success?
The team focused on outstanding voice quality on the AWS backbone and utilised deep operational experience from Amazon's customer service function.
How many years of experience does Colleen Aubrey have at Amazon?
Colleen Aubrey has more than twenty years of experience at Amazon.
Connected stories
Reading receipt
- AWS executive Colleen Aubrey argues that AI success should be measured using existing business metrics.
- Amazon Connect serves as a primary example of applying deep operational experience to build successful AI applications.
- The Connect family expanded into Customer, Talent, Health and Decisions products based on lessons from its initial voice quality focus.
Still open: How specifically should different industries adapt the Connect playbook to their unique operational contexts?
Still open: What specific business metrics does Aubrey recommend for different types of AI applications?