RT Robert Truesdale

What AI Agents Are Good At (And Where They Still Need a Leash)

I've been running AI agents in production since 2026, and I'm done with the hype cycle. These tools can actually do useful work—but they can also eat your afternoon in ways you didn't think possible. Here's what I've learned about where they shine and where they'll bite you.

The Stuff Agents Do Well

Let me be specific. I'm not talking about chatbots that summarize meetings. I'm talking about autonomous agents that take action—writing files, calling APIs, running scripts, making decisions without you in the loop.

Where they earn their keep:

  • Scanning and summarizing logs at scale. I have an agent that pulls logs from 40+ servers every morning, looks for error patterns I've trained it on, and gives me a three-paragraph summary. It used to take me an hour. Now it takes four seconds. The key is that I've locked down what it can actually do—it reads and summarizes. It doesn't delete anything, doesn't restart services, doesn't email anyone. Read-only with a summary output.
  • First-pass code review. My agent scans pull requests for common mistakes—hardcoded credentials, missing error handling, obvious security issues. It's not replacing a human review. It's doing the boring part faster. It flags maybe 30% of what a senior dev would catch, but it catches it consistently, and it never gets tired.
  • Generating boilerplate at scale. Need to spin up 20 Terraform modules with the same structure? An agent can do that. It's not creative work. It's pattern application, and that's exactly what these tools are good at.
  • Content triage and categorization. I run a technical blog with about 800 articles. My agent helps tag them, checks for broken internal links, and suggests which older posts might need updates based on search data. It's a research assistant, not an editor.

That's the honest list. Agents are good at applying patterns to large volumes of data, doing repetitive tasks consistently, and handling the "read this, tell me what's interesting" work that burns human time.

Where They Still Need a Leash

Here's where I stopped being impressed and started being careful.

They will confidently do the wrong thing. I had an agent tasked with cleaning up old S3 buckets. It found a bucket labeled "temp-archive-2026" and deleted it. Turns out a dev was actively using it. The agent didn't know. It followed the pattern it was given. There's no undo button in S3.

This is the core problem. Agents optimize for completing their task. They don't know what's important to your business. They don't know that "temp-archive-2026" is actually critical because someone made a naming mistake.

They hallucinate tool use. I've seen agents try to call APIs that don't exist, use parameters that don't apply, and report back that they "successfully completed" tasks they never actually performed. You have to verify everything. And verification takes time—so you lose the speed advantage unless you're very careful about scope.

They don't know when to stop. Give an agent a vague goal like "improve the codebase" and it'll go wild. I've seen agents generate thousands of lines of code that no one asked for, create new files that conflict with existing ones, and generally make a mess while feeling productive. The tighter the scope, the better. Loose goals produce loose results.

Context window limits are real. Throw a 500-page architecture document at an agent and ask it to find inconsistencies. It'll miss things. Not because it's stupid—because it can't hold the whole picture at once. You need to chunk your inputs and verify outputs.

Failure Modes I Actually Encountered

Let me give you the real maintenance picture.

The "it worked yesterday" problem. My log-scanning agent broke in March 2026. One of the servers changed its log format slightly. The agent couldn't parse it, and instead of flagging an error, it returned an empty summary. I didn't notice for two days because I was trusting the output. Now I have a sanity check—it verifies it processed at least X records or it alerts me. Empty success is the most dangerous failure mode.

The permission drift issue. Agents run with whatever credentials you give them. I gave one agent write access to a documentation repo. Six weeks later, I realized it had been making commits under its own identity that didn't go through code review. I had to audit 200 commits. The lesson: agents should have the minimum permissions required, and their changes should still go through your normal review process.

The dependency hell. Agents often need specific versions of libraries, specific API access, specific environment variables. When something changes on your end—and something always changes—the agent breaks silently. I now treat every agent like a petri dish: interesting to watch, but I expect it to die without constant attention.

What Actually Works

After two years of this, here's what I'd tell someone starting in 2026:

1. Treat agents as junior team members, not contractors. A contractor delivers a result and leaves. An agent needs supervision, feedback, and clear boundaries. If you wouldn't leave a junior dev alone with a task, don't leave an agent alone with it.

2. Build guardrails, not just prompts. If your agent can delete something, add a confirmation step. If it can modify files, route its changes through a review pipeline. The prompt is the first line of defense; the architecture is the second.

3. Start read-only, then add privileges slowly. Let the agent summarize before you let it edit. Let it flag issues before you let it fix them. Every new capability is a new failure mode.

4. Budget for maintenance. I spend about 2-3 hours per week keeping my agents running—fixing broken parsers, updating their knowledge bases, reviewing their outputs. This is not a "set it and forget it" technology. If someone tells you otherwise, they haven't used it in production.

What I Would Do First

If you're thinking about adding an agent to your workflow, don't start with your most critical system. Don't start with "automate our database migrations" or anything that could cause real damage.

Start with something bounded and verifiable:

  • Summarize a specific log file and output to a text file you can check
  • Scan a specific directory for a specific pattern and report what it finds
  • Take meeting notes in one channel and post a summary to another

Pick something where you can verify the output in 30 seconds. Get comfortable with what "good" looks like. Then expand slowly.

The agents aren't ready to run your infrastructure. But they can make your life easier—if you keep the leash tight and your expectations realistic.