In a recent episode of the podcast "You Should Know," hosted by Ryan Leary and William Tincup, John Phillips, Group Vice President of Employee Experience at ServiceNow, made a compelling case that the way organizations measure AI adoption is fundamentally flawed. Phillips argues that counting how often employees use AI tools misses the point entirely. Instead, he insists, the focus should be on whether jobs get done faster, with less friction, and with better outcomes for both employees and the business. This conversation comes at a time when chief human resources officers (CHROs) are under increasing pressure to demonstrate AI productivity gains across increasingly fragmented technology stacks.
Phillips painted a stark picture of the current AI landscape, describing it as a "train wreck of productivity." He noted that every system of record now ships its own AI agent, creating chaos for practitioners. "We're watching this like train wreck of productivity," he told the hosts. The problem, he explained, is that these agents do not communicate with each other, leading to a disjointed and inefficient user experience. He cited customers arriving with eight purchased AI tools plus one they built themselves, none of which talk to each other.
Phillips predicts a fast pivot in how AI success is evaluated. "We're going to quickly stop talking about AI adoption as tool usage, and we're going to start looking at the outcomes and jobs to be done," he said. This shift is crucial because measuring tool usage alone does not capture the real impact on work performance. For example, an AI tool might claim to save employees 23 hours a month, but if those savings are not realized in practice, the metric is meaningless. Phillips emphasized the importance of the two-sided value exchange between employee and employer, and questioned what actually happens to the time saved.
The conversation also delved into the collapse of work boundaries post-COVID, burnout, and the internal dialogue of "am I enough." Phillips highlighted the need for hyper-personalization in employee engagement, arguing that discretionary effort is a truer metric than traditional engagement surveys. Tincup, who has long criticized engagement surveys, pushed Phillips on this point, and Phillips agreed that discretionary effort—going above and beyond what is required—is a more meaningful indicator of employee engagement.
ServiceNow's approach, according to Phillips, is to layer an agentic companion across existing systems rather than ripping and replacing them. He described the company's AI control tower vision as an agentic overlay that stitches together 15 large language models (LLMs) and 100 systems. This approach aims to solve the fragmentation problem by providing a unified interface that can orchestrate work across disparate tools.
Phillips also shared his philosophy shaped by time spent in refugee camps, arguing that "skills and talent is universal and opportunity is not." This perspective underscores the importance of democratizing access to AI tools and ensuring that all employees, regardless of background, can benefit from the productivity gains AI promises.
Leary shared a personal anecdote about applying to Home Depot and never receiving so much as an acknowledgment email, illustrating the disconnect between the promise of AI-driven efficiency and the reality of many organizations' talent acquisition processes. The hosts and Phillips agreed that AI has the potential to fix such broken processes, but only if organizations stop measuring adoption and start measuring outcomes.
The episode, which is part of the WRKdefined Podcast Network, is available now wherever podcasts are heard.
