09/04/2025
Table of ContentsAs AI continues to reshape the workplace, organizations are grappling with a fundamental question: How do we know if our AI investments are actually paying off? In a Tech Edge episode, William Belet, product strategy lead for tyGraph at AvePoint, delved into the critical importance of measuring Microsoft 365 Copilot success and what that data reveals about organizational transformation.
It’s not only about tracking usage but more about understanding the human story behind AI adoption. This blog covers why telemetry is an essential foundation of AI adoption to gather data and understand Copilot’s value to your business.
The Foundation: Why Measurement Matters More Than Ever
William’s insights from working with organizations implementing Copilot revealed fascinating patterns that every business leader should understand. He made a compelling point that resonated with my own experience:
“Having great telemetry is really important as part of the AI process because when it comes to having really great data, you’ve got to have an intimate understanding of people’s usage.”
It isn’t just about checking boxes or proving ROI to executives, but also understanding the ways AI transforms work itself. William explained that measuring Copilot usage is no longer simply a matter of whether everybody started using AI — it’s about identifying the specific scenarios where it’s driving real business value and telling that story convincingly.
In my role overseeing information management strategies, I’ve seen firsthand how organizations that skip the measurement piece often struggle to scale their AI initiatives effectively. They may see pockets of success, but without the data to understand why certain approaches work and others don’t, they can’t replicate or expand those wins.
The Tale of Two User Types: Natural Adopters vs. Everyone Else
Two distinct camps emerge during Copilot rollouts: “You’ll have folks who are self-starters; they’re excited about what the technology can bring,” says William. These natural adopters often have previous experience with AI tools like ChatGPT or Gemini, so when Copilot arrives, they immediately latch onto it and extract value right away.
Then, there’s another group wherein technology or change does not come in naturally — and this is where it gets interesting. William made a crucial point that many organizations miss: “Delegation is a skill that a lot of people don’t always have time to practice.” This struck me as particularly insightful because it reframes AI adoption challenges. It’s not only adopting a new technology and developing a fundamentally different approach to work, but also learning prompts and effectively delegating tasks to an AI assistant.
This observation has significant implications for training and development strategies to enable employee engagement and AI implementation. Organizations that recognize this challenge early – and have a team in charge of change management and digital transformation – can support more targeted interventions to help struggling users bridge that gap.
Privacy and Trust: The Delicate Balance
William noted significant hesitancy about what the telemetry data might reveal within the company. Data exposure repeatedly came up as an organizational challenge, touching on something I encounter frequently: the tension between gaining insights and maintaining employee trust.
However, AvePoint’s approach in handling its Copilot program demonstrates a crucial principle: you can gain valuable insights without compromising individual privacy. Even though they use their own analytics tool — tyGraph, they treat themselves like any other customer, choosing high levels of anonymization. Organizing, securing, and properly ensuring data won’t expose inappropriate information is a valuable organizational maturity exercise.
It forces companies to “clean house” and establish good data governance practices that benefit them far beyond their AI initiatives.
Different Roles, Different Value: The Customization Challenge
It’s one thing to have the data, but another to know what to do with it moving forward. William’s explanation of how his role in product strategy generates completely different value from Copilot compared to someone in sales was particularly enlightening. He described using Copilot to write acceptance criteria for developers and generate content to motivate component building, while sales team members would have entirely different use cases and value propositions.
This diversity in application presents both an opportunity and a challenge. The opportunity is that Copilot’s versatility means it can drive value across virtually every business function. The challenge is that measuring and supporting this diverse range of use cases requires a more sophisticated approach than one-size-fits-all training or evaluation metrics.
From my perspective, managing information across diverse business units reinforces the need for role-specific measurement frameworks. You can’t effectively gauge success using the same metrics for a product strategist and a sales representative — their value drivers are fundamentally different.
