Making deliberate AI investments, matching capabilities to people’s needs, and validating the business value they deliver.
Listen to a short audio summary (5 min)
Prefer reading? The full article continues below.
In my recent articles, I explored what changes as the Digital Workplace becomes an AI Workplace, and how work and people’s roles evolve in AI-enabled service management.
Both lead to a practical question:
How do we make sound AI investments, and know whether they are delivering value?
Adoption figures are a useful starting point. But they are only the beginning.
Licences assigned show access. Prompts submitted show activity. Sustained use shows adoption. Better work outcomes show improvement. Business value requires evidence—and someone accountable for validating it.
This article reflects my perspective as a Digital Workplace and service management leader. It offers a practical approach to investment and measurement; outcomes will depend on the organization, its workflows and how the capabilities are implemented.
Business Value Starts Before the Investment
Value is usually discussed after a rollout. It starts earlier.
Before buying or enabling an AI capability, we need to ask:
- What business problem are we trying to solve?
- Which users and workflows need support?
- Can existing capabilities already meet the requirement?
- What improvement do we expect?
- Who owns the outcome, and how will it be verified?
New AI features arrive constantly. Each should trigger an evaluation, not an automatic investment.
A capability can be impressive without addressing a priority business need. It may also overlap with something the organization already has.
The governance approach I discussed in Building an Agentic Center of Excellence applies here: involve the existing governance structure and relevant owners, assess the requirement, and make a deliberate decision.
The investment should follow the need.
The Right Capability and License for the Right User
Someone handling complex, frequent workflows may need different capabilities from a person who uses AI occasionally. Role, task frequency, required features, existing entitlements and approved data access all matter.
Paid AI licences do not necessarily need to go to the entire organization.
Start with users and workflows where there is a clear requirement and a credible opportunity to improve outcomes. Choose the capability and licensing option that meet that need.
A targeted pilot can establish who benefits, what support they need, and whether wider investment is justified.
The approval should connect the user, the capability and the expected outcome.
Move Up the Measurement Ladder
Activity is often the easiest starting point. The harder question is how far up the ladder the evidence actually reaches.
| Stage | What it tells us | Example evidence |
|---|---|---|
| Activity | People have access and are trying AI | Licences assigned, active users, interactions |
| Adoption | AI is becoming part of relevant work | Sustained use in defined workflows |
| Work outcome | The work is improving | Quality, handling time, effort, rework |
| Business value | The improvement supports organizational goals | Capacity used, service improvement, validated savings or risk reduction |
Each stage answers a different question.
An assigned licence shows access. Repeated use suggests adoption. Neither establishes whether the work is better.
High usage may reflect useful support. It may also reflect experimentation, repeated attempts or outputs that require correction.
To connect adoption with value, establish a baseline: how the work performed before the change, and what improvement would justify the investment.
Tools Help Build the Evidence
In my earlier Analytics Hub article, I explored resources for understanding Copilot adoption, licence utilization and potential impact. That discussion also included Copilot Analytics Labs and Microsoft365 Analytics Insights – Copilot Adoption.
These resources can help teams investigate usage patterns, identify adoption gaps and inform licence reviews.
Microsoft’s Copilot Dashboard in Viva Insights provides readiness, adoption, impact and sentiment insights. The measurement ladder in this article is my framework for connecting those signals with workflow outcomes and business value.
Microsoft’s own documentation describes Copilot assisted hours as a general estimate, using activity data and research-based assumptions. Its current methodology applies six-minute assistance factors to search or summary actions and creation actions; meeting-related assistance is calculated differently.
These are broad approximations, not direct measurements of the time each employee saved. Microsoft also notes that seasonality, role shifts and organizational changes can influence metric changes.
The important step is connecting telemetry with actual work.
Combine tool-generated insights with workflow measures, employee feedback and validated costs. Check metric definitions and known data issues before relying on a baseline.
Tools help assemble the evidence. Accountable owners determine whether it supports the investment decision.
Who Owns the Evidence?
An AI business case needs a clear answer to this question.
Someone must establish the baseline, validate the results and decide whether to scale. Several groups contribute:
| Contributor | Responsibility |
|---|---|
| Business or process owner | Defines the expected outcome and confirms its relevance |
| Service or platform owner | Coordinates measurement, adoption and ongoing optimization |
| Employees | Validate the practical benefit and review effort |
| Finance | Validates financial benefit claims where applicable |
| AI CoE, security and governance teams | Support evaluation and appropriate controls |
Several teams contribute evidence, but a named business or service owner should remain accountable for the outcome and the decision to scale.
One distinction matters: time saved does not automatically become financial savings.
It may create capacity, reduce employee effort or improve service responsiveness. Each is a potential benefit, but it should be measured and described accurately.
If capacity is redirected into resolving recurring problems or improving knowledge, show that connection. If the benefit is reduced workload, evaluate it directly rather than converting every saved minute into a cash-saving claim.
An Illustrative Example: AI-Assisted Service Desk Work
This example is hypothetical. The directional changes illustrate how to assess a pilot; they do not represent measured or expected results.
Consider a common category of collaboration tickets where AI could help analysts retrieve knowledge and prepare responses.
1. Establish the baseline. Record active handling time, end-to-end resolution time, reopen rates, quality and analyst effort. Compare tickets of similar categories and complexity.
2. Select the right users and capability. Choose analysts who regularly handle the workflow and identify the capability they need.
3. Run a limited pilot. Define its scope, duration, review requirements and success criteria.
4. Measure more than speed. Track quality, rework, analyst effort and the experience of people receiving support.
5. Validate the benefit. Include relevant costs and the time spent checking and correcting AI output.
6. Decide. Expand, adjust or stop based on the evidence.
The pilot might produce a mixed result:
| Measure | Before | During pilot |
|---|---|---|
| Active handling time, including review | Established baseline | Lower |
| Reopened tickets | Established baseline | Slightly higher |
| AI-output review effort | No AI-specific review | Additional effort within handling time |
| Analyst experience | Established baseline | Mixed feedback |
Faster handling is promising, but additional reopened tickets could reduce the overall benefit. Review effort should be included in handling time and tracked separately for understanding, without counting it twice.
The next step may be to improve knowledge sources, training or the review process before expanding access. Other changes during the pilot—such as staffing or ticket complexity—should also be considered before attributing the result to AI.
A pilot is useful when it improves the decision, including a decision to adjust the approach.
Review, Optimize and Scale
Licence optimization starts at approval and continues throughout the service lifecycle.
As needs evolve, review whether licences remain assigned to the right people and whether outcomes justify renewal or expansion.
Do users need training? Has the workflow changed? Would another approved capability meet the requirement more effectively?
When usage is low, investigate before reassigning licences. The cause may be missing skills, access barriers, an unclear use case or a workflow that gains little from AI.
Frequent usage should also be assessed against outcomes before expanding investment.
Licence and consumption costs belong in the evaluation, alongside implementation, support and human review. My AI FinOps article explores cost management in more detail.
Here, the focus is whether those costs are justified by verified benefits. Where financial ROI is claimed, use validated financial benefits and relevant costs over a defined period. Report service quality, employee experience and risk benefits separately unless there is a defensible basis for monetizing them.
Closing Thoughts
Digital Workplace leaders need to connect AI investment with user needs, service outcomes and organizational priorities.
That means looking beyond deployment and usage to understand whether the capability improves work—and whether that improvement justifies continued investment.
Before asking how many people are using AI, ask which people need which capabilities, what outcome the investment should improve, and who will verify that it did.
Define the need. Approve deliberately. Validate the benefit. Use the evidence to optimize or scale.
Related Reading
- The Digital Workplace Is Becoming an AI Workplace: But the Foundations Haven’t Changed
- AI-Enabled Service Management: Automating Work, Evolving People’s Roles
- Building an Agentic Center of Excellence: Do We Really Need Another CoE?
- Analytics Hub: Microsoft’s Free Toolkit to Measure Copilot Adoption, Impact & ROI
- FinOps for AI: Understanding Tokens, Copilot Credits and the Real Cost of AI
Reference
- Microsoft Learn: Microsoft Copilot Dashboard in Viva Insights — Official documentation covering dashboard metrics, assisted-hours methodology, interpretation limitations and known data issues.
No comments:
Post a Comment