Where Coding Agents Help, and Where They Fail
Pedro Alves explains where AI coding agents speed up engineering, where they introduce hidden risks, and why experienced developers remain essential.
Pedro Alves
CTO at Thoth AI
Coding agents: Coding agents can accelerate implementation, but human judgment must guide architecture, quality, and production decisions.
Hidden bugs: AI-generated code often runs successfully while concealing logical defects that require experienced reviewers to detect.
Creative gaps: Models handle well-defined translation tasks, but people must supply unconventional insights, connections, and core decisions.
Team leverage: Organizations gain more by scaling experienced professionals with AI than by trying to elevate inexperienced teams.
Central control: A small AI-focused team can create reusable workflows, reduce duplicated experiments, and improve adoption across departments.
Pedro Alves has been working extensively with AI for 25 years. He is currently CTO at Thoth AI and the creator of Last Week in AI.
We sat down with Pedro to discuss coding agents — and why it’s so important to know what they can and cannot do. Here’s what he told us.
Before “AI” Was Something People Put on Slides
I'm Pedro, CTO of Thoth AI. I was working with AI before it was something people put on slides — back in 2001, as a computer-science undergrad, I built a little simulated world full of agents and let them learn to outplay each other, a kind of digital game-of-life where the whole point was watching them get better. That was 25 years ago, and I'm still chasing the same question: How do you get a system to improve?
Continue Reading for Free
Create a free account to finish this article, plus get ongoing access to timely insights and practical resources.
Have an account? Log In
When grad school came around, everyone assumed I'd do a PhD in machine learning. I didn't. It felt too narrow, too theoretical. I wanted to learn the craft against the richest, messiest data I could find, so I went into computational biology: proteomics for my master's, genomics for my PhD. I picked up machine learning, statistics, and data science by applying them — gene-gene interaction networks, feature engineering, neural nets, and ensemble methods — to problems where being wrong cost something.
That's where I learned the lesson that's shaped everything since. In medicine, I built disease-progression models, and the standard scorecard — overall accuracy — was useless. A "good" model by that measure just reconfirmed the cases doctors already knew about, when the entire value was in the cases they'd miss. So instead of optimizing accuracy, I optimized partial AUC — essentially the model's skill on the cases that mattered. The metric someone hands you is almost never the real goal.
From there, I toured various industries: those medical models, a stint consulting for a pro soccer team, then Silicon Valley, and years in computer vision — back when you still had to innovate on how you trained and even initialized a network's weights to get detection and classification working at all. I worked with data from social networks, retail, fashion, and video. I spent a year building trading algorithms with genetic algorithms and particle swarm optimization — nonparametric optimization methods I'd written papers on in grad school. Eventually, I started my own company, building AI tools for people who weren't data scientists.
Breadth connects all of it. After a dozen fields, I keep seeing the same problem in different costumes — a trick obvious in genomics turns out to be the unlock in computer vision. I care more about how a problem is framed than how it's solved, because the framing is where the leverage hides.
This is what brings me to Thoth. On paper, we're a data-annotation company. But I think the problem the whole space solves is the wrong one. Nobody wants annotated data — data is just the intermediate step. They want a better model. So, I'm pushing toward a future where a system figures out what data your model needs to improve and handles that part for you, so you can focus on the model instead of the labeling.
Two Sides of the Business
Structurally, Thoth AI has two sides. One side is the data operations engine, which runs internationally and at scale. It's bigger than a typical startup but still much smaller than the giants in the space, so I'd call it medium-sized. The other side, the part I am building, is a brand new research and innovation team here in the Bay Area — currently five of us, with more joining over the next few months. There, we pursue the rest of the model improvement: everything beyond labeling.
Our research has two main thrusts. The first is robotics: generating embodied data, evaluating it, and using active learning to automatically generate the best possible training data, so robots can learn their tasks on the least amount of data possible. Most of it uses egocentric data (essentially first-person footage), and we focus much of our work on producing the highest-quality egocentric data possible. The second is reliability and efficiency in language models: understanding what causes hallucinations and what they look like, and on the efficiency side, recognizing when a smaller model can handle a query, then reformulating or breaking it into pieces so a simpler, cheaper model can answer it. All of this is research today, aimed at products we will launch from this team.
Regarding stage and size: the data side is an established, medium-sized international operation, while the US research team is deliberately early. Five of us are in our San Mateo office now, with one or two more hires joining over the next few months. That is our current status.
How Heavy Testing Creates Reliable Outcomes
Many rush to automate every task and function with AI, but I am cautious. AI makes building a demo trivial. Building something reliable for daily use is different. When you use these tools for real, you hit edge and corner cases where automation almost works, but not quite. So, I test heavily before trusting anything in a real workflow.
This year, I built AI tooling for our weekly CEO review meeting that cleared that bar. It emails area owners to collect updates, automatically compiles responses, drafts reports, summaries, and the agenda, and tracks results over time.
Before, all of that was manual. Someone chased people for updates, manually stitched responses together, wrote summaries and the agenda, and tracked outcomes week to week. It worked, but it was slow, and the agenda easily drifted toward what was freshest rather than what mattered most.
Prep now mostly runs itself. But that's not the real win. The bigger improvement is quality: the meeting is sharper and more on point, and the right things reliably reach the CEO. So it's less about time saved and more about directing the CEO's attention to the decisions that truly matter.
Why Judgment Remains Human While AI Accelerates Coding
AI now powers coding itself, almost everywhere. My researchers, engineers who productionize our work, and I all use AI coding tools. Human judgment, not a specific task category, determines how code gets built, and this standard shifts with the stakes.
On the research side, the goal is usually to get something running, often by setting up a GitHub repo to test an idea's viability. Nothing heads to production, so I am lenient about its writing. If it works and answers the question, that's enough.
Production is different. How we build something matters as much as whether it runs, so we use AI more carefully. Senior engineers own the architecture, quality, and reviews. AI accelerates them, but it does not make those calls. That line remains human.
The team's seniority helps. They use AI differently from less experienced individuals. They already know what good looks like, so the model amplifies their existing judgment rather than substituting for judgment they are still developing. It speeds them up without replacing the essential part.
How to Use Coding Agents Effectively
Almost every project has one unique thing it needs: a non-obvious insight or decision the problem calls for…The part that makes a project work is usually not average, and the AI cannot arrive at it on its own. That piece is mine to find. Then, the AI builds everything downstream of it faster and better than I could alone.
Overall, the effectiveness of coding agents comes from disciplining myself about the technology's strengths and weaknesses.
I think about it like painting a canvas. The goal is always the same: clarify the picture in my head, then use AI to translate that intent into working code as fast as possible. Its success depends on one variable: how much creative work I keep versus handing off.
AI shines in the first tier. I can see the whole painting and lay down broad strokes, and the AI fills in the details. I know exactly what needs to be built, and translating intent to code is essentially formulaic. This is where AI is a real force multiplier, and most of my day-to-day work falls here.
The second tier is useful but has higher variance. It's similar, but I intentionally leave a few canvas regions open and tell the model to paint something there. It requires some creativity, but my direction still bounds it. I treat that output as a draft, not an answer.
I caution people about the third tier. This involves handing the AI the creative core: the project definition or key design decisions. It looks impressive to someone not deep in the domain, but it underperforms far more often than it succeeds because the model fills a gap I should have filled.
That gap is the whole point. Almost every project has one unique thing it needs: a non-obvious insight or decision the problem calls for. By default, the model provides the average solution, the most common pattern from its training. The part that makes a project work is usually not average, and the AI cannot arrive at it on its own. That piece is mine to find. Then, the AI builds everything downstream of it faster and better than I could alone.
How Coding Agents Create Hard-to-Find Bugs
The clearest good result of coding agents is speed. Coding is faster, which increases our deployment frequency, and it's not just raw speed. Things that were complicated to set up are much easier now. That part is real, and it matters.
A more interesting result is how our bugs changed. The defect rate hasn't gone up, but the types of defects have changed. AI ensures code runs. It almost always checks this before handing it back to you. So, the simplest bugs — those that just throw an error or won't run — show up far less often. What remains are logical bugs, cases the model didn't think through, and these are harder to catch precisely because the code runs cleanly.
That's where the downside lies. These bugs hide better. If someone lacks experience or leans too heavily on the tool, they see the code running and assume it's correct when it isn't. The danger isn't more bugs; it's false confidence. You need someone who knows what to look for to catch things that pass the "it runs" test but are still wrong.
Why AI Should Be Used to Scale Experienced People Instead of Improving Weak People
What surprised me most was where the leverage comes from. Intuitively, one might think AI lowers the experience bar, allowing you to staff cheaper, more junior teams and let the tooling level them up. The opposite proved true. Experience matters more now, not less.
It comes down to reliability. Using AI to turn one senior engineer into five senior engineers is far more reliable than turning a junior engineer into a senior one. AI multiplies judgment you already have; it does not manufacture judgment you don't.
So, I scale my strongest people out; I don't try to scale my weakest people up. A few experienced people, each running at several times their old output, beat a larger mixed team every time.
Why Humans Must Bring the Creativity — Not AI
AI clearly has not delivered on the creative and connective aspects of research. When I do technical work with AI assistance, a human still needs to connect the dots: to figure out how to extend someone else's work, to link two papers in a way that makes sense, and to make the logical leaps you'd take for granted. That part still requires a person.
This is not surprising when you remember what these models are. They are probability machines. They output the text most likely to please the reader. We are getting better at training them on other kinds of tasks, but it becomes genuinely hard when you try to teach creativity itself, because building the training signal is difficult.
Consider datasets. A dataset for what a dog looks like is easy. You collect many pictures of dogs. A dataset for creativity in drawing dogs presents a different problem. For example, consider someone who adds wings to a dog. The point of that example is not the wings. It is the abstract move behind them: taking something that does not belong and putting it where it is not expected. To get a model to understand that, you need a huge number of examples so it learns that the wings do not matter; the creative gesture does.
Even if it gets there, you hit the next wall. Now, take that same creativity and apply it to a mathematical equation or a piece of code from a paper, using it to extend someone's algorithm. That kind of transfer is where it falls apart.
People lean on machines for exactly the part where they are weakest. The irony is that the fix is cheap. A pinch of human creativity goes a long way. Do not lean too hard on the machine for that piece, and you get remarkably far.
Why CTOs Must Redesign How AI Capabilities Move Through Organizations
This won't be the popular piece of advice right now, but I think it's the prudent one, and most likely to pay off: Be more deliberate than the moment pushes you to be.
Right now, companies are spending enormous amounts of money on a simple assumption: everyone can automate part of their job with AI. So the directive goes out to everybody. Engineers, data people, sales, marketing, the whole company. Get an account, automate something, build your own tools, go. The result is a huge amount of wasted effort: duplicate work, projects that go nowhere, tools nobody uses. Companies burn a lot of money on effort, time, and tokens that produce nothing durable.
Put a real process around it. Centralize it. Have a small central team whose job is to make every other team better with AI. They develop workflows, reusable context, standards, and effective patterns, then push them out so individual engineers aren’t each paying the discovery cost. This is the structural version of what I said earlier about concentrating leverage in your strongest people: a few AI-fluent engineers lifting the whole organization.
This yields two results simultaneously. Utilization goes up because teams run proven patterns instead of guessing. And you claw back a large amount of time quietly lost to experimentation that never needed to happen more than once.
This won’t be the popular piece of advice right now, but I think it’s the prudent one, and most likely to pay off: Be more deliberate than the moment pushes you to be…Put a real process around it. Centralize it. Have a small central team whose job is to make every other team better with AI.
Follow Along
You can follow Pedro Alves's work on LinkedIn or subscribe to his newsletter, Last Week in AI. And check out Thoth AI.
More expert interviews to come on The CTO Club!
