AI Adoption Is a Vanity Metric
Chris Garrett explains why AI success depends on governance, verification, and human judgment—not adoption metrics or automation alone.
Chris Garrett
VP of Managed Technology and Security at Novatech
AI Impact: AI is changing leverage, influencing decisions, and forcing leaders to focus on real outcomes.
Workflow Improvements: AI aids in security alert triage, log analysis, and documentation, improving efficiency without replacing humans.
Governance Necessity: AI governance must precede deployment to prevent unintentional data access and ensure responsible usage.
Capability Gap: AI offers speed, not capability, and firms need to avoid over-reliance on AI-generated solutions.
Junior Role Evolution: AI shifts junior roles from volume tasks to judgment training, requiring a reevaluation of development paths.
Chris Garrett is VP of Managed Technology and Security Operations at Novatech, a managed service provider. He's also the author of Scale or Sell.
We sat down with Chris to learn from his AI workflows and to get a sense of how AI is affecting the industry. Here's what he had to say.
Translating Systems to Dollars
I run Managed IT and Security at Novatech, a managed service provider owned by Perpetual Capital Partners. The title says VP. The work says something plainer. I am accountable for whether technology earns the business money or quietly drains it.
Continue Reading for Free
Create a free account to finish this article, plus get ongoing access to timely insights and practical resources.
Have an account? Log In
The path here was not a straight line. I came up through the operational layer, the place where systems break, and someone has to own the fix. I worked the help desk, then infrastructure, then security. Those are the rooms where the gap between what a vendor promises and what a network does becomes your problem at 2 a.m. That foundation taught me the lesson that has held ever since. Technology is not the point. The outcome is that revenue is protected, risk is priced correctly, and downtime is erased before a client feels it.
Then, I learned to translate. Engineers speak in systems and executives speak in dollars, and most technology leaders pick one language and lose the other audience. I built a career standing in that seam, turning latency, exposure, and architecture into the language of margin and continuity: IT as a multiplier, never a cost center. That framing changed the conversations I joined and the decisions I shaped.
The ground is moving again. AI is not the next tool in the stack. It changes what leverage means, who holds it, and how fast capability compounds. Most organizations are answering with noise. They run pilots that prove nothing and write policies to manage anxiety rather than risk, while vendors sell inevitability to buyers who have not done the arithmetic.
My interest is narrower and harder to fake. Where does AI move the actual numbers? What does it expose that was already fragile? What competence does it demand that most teams do not yet have? I treat it the way I treat everything else: evidence over hype, outcomes over consensus, the systems layer over the surface story.
Complexities of the Job
Novatech is a managed service provider owned by Perpetual Capital Partners since September 2024. We are a post-acquisition, profitable services business with a mandate to scale margin and security revenue while maintaining the delivery quality that earned our client base.
The division operates alongside Managed Technology, Managed Office, and Field Dispatch, performing work that keeps client operations running and defended. My team is 68 engineers and analysts across service desk, infrastructure, and a dedicated security practice, supporting 400 client environments.
The architecture is not a single product; it is multi-tenant. We operate dozens of distinct client environments, each with its own mix of on-premise infrastructure, Microsoft 365, Azure, AWS, and Google Cloud, plus hybrid networks we did not always design but now own. This variance creates complexity. A routine change for one client poses a production risk for another. The discipline involves managing this fleet to a single standard of reliability and security posture without forcing every client into the same box.
Our deployment model is managed and remote-first, supported by Field Dispatch for work requiring on-site presence. We continuously monitor, patch, and defend. We measure ourselves on uptime, response time, and how quickly we contain security events before clients experience them.
How AI Transforms Technical Decision-Making Processes — But Does Not Own Them
My core philosophy is "Wrap the Engineer in AI" to get the best of both worlds. No headcount reduction; instead, the team does more and better work with assistive tooling.
AI now powers high-volume work where a human reviewer was always the bottleneck and never the value. For example, security alert triage first runs through an AI layer because a SOC analyst reading ten thousand low-signal events by hand is expensive and performs worse than the model.
We use AI similarly for log analysis during investigations, for first-pass documentation and runbook drafting, and for summarizing a client's environment before a review. This ensures the engineer walks in informed instead of cold. In all these cases, AI accelerates the read; it does not make the call.
The call stays human, and the line is deliberate. Architecture decisions for a client environment remain human because the model optimizes for the technically clean answer. A real decision weighs budget, the client's risk tolerance, and what their team can actually operate after we leave.
Incident response remains human at the command level. AI can surface what happened, but containment choices carry business consequences, legal exposure, and client trust. I will not hand judgment under pressure to a tool that cannot be held accountable for the outcome. Final security review before sign-off remains human for the same reason. The model excels at finding known patterns but is blind to novel issues that truly matter.
A Real-World Security Alert Triage Workflow
I will not hand judgment under pressure to a tool that cannot be held accountable for the outcome…The judgment remains with the person.
Let's dive deeper into the security alert triage, because it runs daily and clearly shows where AI works and where a human takes over.
The process starts with a volume that no human can or should consistently process manually. Across the client fleet, our monitoring generates tens of thousands of events daily, most of them noise. Before AI, an analyst worked a queue, and the real signal remained buried under failed logins and benign anomalies. Important issues received late attention because the analyst had to wade through noise to find them.
The AI layer first processes that raw stream. It clusters related events, eliminates obvious false positives against known-good patterns, correlates activity across a client environment that a human would consider unconnected, and ranks the remaining items by their resemblance to a real intrusion. The analyst does not receive a firehose of data. Instead, they receive a short, ordered list with attached context, named affected systems, and a spelled-out reason for each item's surfacing. The model performs the initial reading but makes no decisions.
The analyst opens that list, and human work begins. They investigate the suspicious thread, and AI assists again by summarizing relevant logs and the timeline, allowing the analyst to reconstruct events in minutes instead of an hour. The judgment remains with the person. Is this a real compromise, or a sanctioned change that appears suspicious? Does the pattern match something novel that the model would not flag? The model accelerates evidence review; the analyst makes the determination.
When a real event occurs, containment remains a human command decision. The analyst, or the senior on-call for serious incidents, decides whether to isolate a host, disable an account, or pull a client offline. These choices carry business costs, legal exposure, and client trust, and no tool can be held accountable for those decisions. AI can tell you what happened, but it does not choose our response under pressure.
After containment, AI drafts the incident record, timeline, actions taken, and client-facing summary. This draft saves significant time compared to starting from a blank page. It does not ship until an engineer corrects and signs it. A polished wrong report is worse than no report, and an AI-drafted procedure of record without a human owner creates a liability we decided would never happen on our watch.
The loop closes with a step most teams skip. We sample what the AI dismissed, not only what it escalated, and feed these misses back to sharpen the next pass. This sampling discipline prevents automation bias from setting in. A tool that makes analysts faster also makes them complacent if no one checks the items it discards.
Why Businesses Must Clean Their Permissions Before Integrating AI
I also implemented a governance gate on Microsoft Copilot.
Before, Copilot appeared to be a productivity upgrade, and clients treated it as a simple switch to flip. The problem is that Copilot inherits a user's existing access. Most of our clients had years of permission sprawl: open SharePoint sites, shared drives nobody had pruned, and files that were technically reachable but practically invisible. The moment you give an employee a tool that searches everything they can reach, invisible content becomes a search result. A junior employee could ask a plain-language question and surface salary data or a board deck never meant for them. Nobody had addressed the permissions, but the exposure was real and had been sitting there for years.
I built a mandatory readiness assessment that runs before we enable Copilot for anyone. We map effective access, find over-shared content, and remediate the permission model first. The client signs off on what their own people can reach before a single AI license goes live. We turned a feature request into a governed deployment with a hard prerequisite.
How AI's Gains Come With Discipline Costs
On the security side, the alert triage layer cut the volume that analysts manually touch by roughly 70 percent. Mean time to acknowledge a real alert dropped from hours to minutes during off-peak windows because analysts no longer dig through noise to find important events. This is our cleanest win. The cost of being wrong is low; a human checks the short list anyway, and the model excels at pattern-matching, which humans do poorly day in and day out.
The CoPilot readiness work produced a different kind of result. We found exposed data in every assessment we have run, problems that predated the AI and would have stayed buried. That is good security and good revenue.
The drafting wins are real but smaller than the hype. AI-assisted documentation and runbook drafts cut the time to a first pass by more than half. An engineer still needs to correct the work, so the saving is in the blank-page problem, not the final product.
The failure mode that cost us the most was automation bias. Early on, a couple of analysts started trusting the AI triage too far and waved through things they should have opened. We caught it in review, but it taught me that a tool making people faster also makes them lazier if you let it, and the fix was process, not technology. We now sample what the AI dismisses, not only what it escalates.
In short, AI bought us speed everywhere and judgment nowhere. Every gain came with a discipline cost to prevent people from trusting it beyond what it earned.
How AI Improves Output, But Not Capability
Here's something no one wants to discuss: AI did not make people more capable. It let them rent the appearance of capability and pretend it was the real thing.
Call it "renting a voice". A person produces an artifact they could not have produced alone — code, an architecture rationale, a security policy, or a client-facing analysis, for example — and the artifact reads as fluent and authoritative. The fluency is real. The words borrow the underlying competence, which disappears the moment the model stops typing. The person owns the output, but not the understanding. That gap is invisible until something breaks, and then that gap is the loudest thing in the room.
Here's an example: A junior engineer ships an AI-generated fix that works in the demo, then cannot explain it during an incident three weeks later. So, a senior engineer reverse-engineers a solution nobody on the team authored. Front-loaded speed led to back-loaded cost, and the cost was higher because the fluent surface delayed the moment anyone checked the substance.
The failed expectation was leverage. The promise was that AI would multiply skilled people. What it does is flatten the visible difference between someone who knows and someone who can prompt, and that flattening is expensive because the two produce identical-looking work and wildly different outcomes under load.
The only durable use of AI, then, is when someone who already has the competence uses AI to work faster, because that person can tell when the output is wrong. Hand the same tool to someone borrowing the competence outright, and you have not added capacity. You have added confident output with no one accountable behind it.
Why Junior Teammates Are Still Necessary with AI
This is the model I used to build every service organization I ever ran: Stack junior capacity under senior oversight, let volume work flow down to the cheapest competent hands, and grow the pyramid as demand grows.
AI ate the bottom of the pyramid. The work I used to staff with juniors — first-pass triage, log reading, runbook drafting, routine tickets — is the exact work the model now does faster and cheaper than a person learning on the job. The staffing math that depended on a wide base of inexpensive volume work no longer holds because the volume work no longer needs a person.
But the junior tier was never only cheap labor. It was the training ground. People became the seniors I rely on by grinding through thousands of low-stakes tickets and building judgment one mistake at a time. Remove that rung, and you have not only changed this year's cost structure. You have stopped manufacturing the senior engineers you will need in five years, and you will not feel that gap until it is expensive to fix.
One way to avoid getting to that point is with narrower AI rollouts, earned tier by tier. Same gains without quietly defunding the pipeline that produces the seniors who make this safe.
I didn't do that, so I am rebuilding the ladder. We keep juniors, but not to absorb volume the model already handles. They are there to develop the one thing AI cannot give them: the judgment to know when its confident output is wrong. That means deliberate exposure to hard problems early, senior time spent teaching instead of delegating, and a smaller, sharper base than the old economics would have built.
Why Adoption Is a Vanity Metric
Rollout fails at the human layer, not the technology layer. I had to walk back almost everything because I got that backward.
I treated the launch as a capability problem. Pick the tools, prove they work, give people access, measure adoption. Adoption climbed, and I read it as success. I was actually watching people get comfortable trusting output they had not learned to question, and comfort is an early symptom of failure, not proof of success.
Speaking of which, I should have refused to measure adoption. It is a vanity number, and it pointed me in the wrong direction. The metric that mattered was whether people verified the output before trusting it, and that is harder to count, so I tracked the easy thing instead. Measuring usage rewarded the exact behavior that caused the thrash: people leaned on the tool more, faster, with less scrutiny.
I would measure verification and rework now, and I would never again let a usage chart stand in for improved work.
Why CTOs Must Tie Every AI Decision Back to Business Numbers
Here's my advice:
- Governance first. It's not exciting, and it won't hype the FOMO crowd like the latest frontier models, but it's an absolute necessity. Govern the entire surface before you deploy and scale it. Most serious issues I have seen came from CTOs who deployed AI for velocity and wrote governance after the fact. Reverse that order.
- Stop buying maturity and start buying verifiable capability. Half the AI in vendor pitches wraps an API, sold as owned competence. Make every vendor show you the thing working on a problem outside its demo, and make your own teams do the same.
- Protect the people who already know. Leverage is real only in the hands of someone who can tell when the model is wrong. Invest in that judgment, measure it, and refuse to let prompt fluency stand in for it on your org chart.
- Tie every AI decision to a number the business already cares about. Protect revenue, price risk, and return hours to people who do work that matters. CTOs who frame AI as a technology initiative will fight for budget every quarter. Those who frame it as margin and risk will get asked for their opinion in rooms that used to make decisions without them. The technical case rarely moves an executive; P&L math does.
- Practice discipline. Deploy AI where a person still owns the result, keep it out of the seat where the result is unforgiving, and audit what it dismisses, not only what it escalates. The market is rewarding speed right now and will spend the next few years repricing teams that mistook fluency for competence. Build so you are not one of them.
Here’s my advice: Governance first. It’s not exciting, and it won’t hype the FOMO crowd like the latest frontier models, but it’s an absolute necessity. Stop buying maturity and start buying verifiable capability. Protect the people who already know. Tie every AI decision to a number the business already cares about. Practice discipline.
Follow Along
You can follow Chris Garrett's work on LinkedIn. And check out his latest book, Scale or Sell.
More expert interviews to come on The CTO Club!
