LLMs in the Real World: When They Work and When They Don’t
The honest assessment of limitations, sustainability, and informed adoption
Introduction
The previous article showed the daily workflows where LLMs integrate smoothly—code analysis, problem-solving, documentation, research. Those patterns work, and they’re valuable.
But if I only showed you the wins, I’d be no better than the hype cycle I started out skeptical about.
This article is the reality check: where these tools actually break down, which integration patterns survive long-term contact with real engineering work, and the honest assessment of whether this whole approach is worth maintaining.
If you’ve been following from skepticism through experimentation to daily integration, this is where we complete the journey with clear-eyed evaluation of both capabilities and constraints.
The Real Question: Better, Not Faster
Before diving into limitations and integration patterns, let’s establish the evaluation framework.
The constant drumbeat around LLMs focuses on productivity: “How much time did you save? How much faster can you ship features?” That’s the wrong question—or at least, it misses what makes senior engineers valuable.
We don’t get paid well because we can type fast. We get paid well because we type the correct symbols in the correct order. The hard parts of engineering have never been about typing speed:
Understanding complex systems well enough to modify them safely
Making architectural decisions that won’t paint you into a corner
Writing documentation that actually helps the next person
Learning new technologies deeply enough to make informed choices
Refactoring legacy code without introducing subtle bugs
The question isn’t “does this make me faster?” It’s “does this make me better at the things that are actually hard?”
That’s the lens for everything that follows. Limitations matter when they threaten craftsmanship gains. Integration patterns stick when they enable more capable engineering. The evaluation isn’t about time saved—it’s about capability expanded and quality improved.
What “Better” Actually Looks Like
When I evaluate whether LLMs are worth using, I don’t ask “did this save me time?” I ask “did this enable me to do work I otherwise wouldn’t have done, or wouldn’t have done as well?”
Take refactoring. I had a controller that had grown organically over two years—400 lines of tangled request handling, validation, and business logic. Classic legacy code that “works” but makes every change risky.
Without LLM assistance, I would have postponed that refactoring indefinitely. Not because I couldn’t do it, but because the investment of understanding all the edge cases, extracting patterns, and ensuring I didn’t break subtle behaviors felt like it would take days for questionable value.
With my architectural specialist, I spent about the same amount of time on the refactoring. But instead of postponing it, I actually did it. And I did it better—with clearer separation of concerns, better test coverage, and documentation of the patterns I extracted.
The time investment was similar, but the outcome was different: production code is now more maintainable, the next feature will be easier to add, and I understand that subsystem better than I did before.
That’s the real win: using the same hours to accomplish more valuable work.
Or take documentation. I used to write sparse documentation—enough to remember my own thinking, but not enough to really help someone else. Not because I didn’t value documentation, but because staring at a blank page after solving a hard problem felt like punishment.
Now I dump my problem, solution, and key decisions into an LLM and ask it to draft initial documentation. I still spend time on documentation—editing, adding context, ensuring accuracy. But I’m editing instead of creating from scratch, and the result is documentation that actually helps the next person understand not just what the code does, but why it’s structured that way.
The time investment is similar. The quality is dramatically better.
These are craftsmanship gains, not productivity gains. They’re harder to measure, but they’re what separates code that works from code that works well and can evolve with changing requirements. If your only metric is “time saved,” you’ll miss the entire point of what makes LLMs valuable for experienced engineers.
With that framework established, let’s look at where these tools actually break down.
Where It Breaks: Honest Limitations
Let’s start with the failures.
The Confidence Problem
LLMs present wrong answers with the same confidence as correct ones. This is the most dangerous limitation because it punishes uncritical trust—and here’s the embarrassing part: I know this, and I still fall for it.
I asked my SQL specialist about optimizing a query with a complex subquery. It suggested moving the subquery to a CTE (Common Table Expression) for better readability and “improved performance through materialization.”
I recognized CTEs as magic I had seen performed before, but like a magician’s assistant, I didn’t really understand what was happening behind the curtain. The explanation sounded technical and authoritative. And critically, I’d had a string of successes with this SQL specialist on previous queries.
So I made the change. Performance got worse.
Turns out, PostgreSQL doesn’t always materialize CTEs the way the LLM suggested, and in my case, the query optimizer was doing a better job with the inline subquery. The LLM’s explanation was technically accurate in general, but practically wrong for my specific database version and query pattern.
Here’s the trap I keep falling into: A series of successes builds confidence, and then boom—an embarrassing failure. The LLM was right about five things in a row, so I trusted the sixth thing without the same level of scrutiny. Especially when it was something outside my core expertise, where I didn’t have the knowledge to be appropriately skeptical.
This is a version of the Gell-Mann Amnesia effect. When you catch an LLM being wrong about something you know well, you recognize the error. But then you turn around and trust its confident explanations about domains where you’re less expert, forgetting that it generates plausible-sounding answers regardless of accuracy.
This is the fundamental challenge: LLMs optimize for plausibility, not truth. They generate answers that sound correct and follow reasonable patterns, but they can’t actually verify those answers against reality. And when you’re operating outside your expertise, you don’t know enough to challenge plausible-sounding advice.
That PostgreSQL mistake didn’t just cost me a bad commit—it cost me a false understanding of query optimization that could have led to worse decisions later. The solution isn’t to stop using LLMs—it’s to never stop verifying their suggestions, especially for critical decisions. And especially when you’re working in domains where you’re an advanced novice at best.
Context Window Limitations
Even with large context windows, complex conversations eventually hit a wall. I was working through a multi-file refactoring with my architectural specialist, and around message 40-50, the LLM started “forgetting” decisions we’d made earlier in the conversation.
“Wait, I thought we decided to extract that into a service class?”
“You’re right, let’s extract that into a service class...”
It wasn’t being contrary—it genuinely lost track of our earlier discussion. The conversation had exceeded its effective working memory, like opening so many browser tabs that your system starts thrashing.
The workaround: start fresh threads for complex work, with explicit summaries of previous decisions. Treat it like starting a new work session where you need to re-establish context. It’s not a bug—it’s an architectural limitation you have to design around.
Here’s the practical approach that works well: when you sense the conversation losing coherence, ask the LLM to “create a summary I can use to start a new thread” with all the context needed to make the next conversation as good as the current one. The LLM generates a complete continuation prompt capturing decisions, context, and current state. Copy that into a fresh thread, and you’re back to full effectiveness.
It’s like checkpointing progress in a long-running process. You lose nothing important, and the new thread starts with full context. And when the LLM loses track of earlier architectural choices, you’re forced to become the sole keeper of architectural coherence—which is actually a feature, not a bug. The limitation keeps you engaged in the system-level understanding that makes refactoring safe.
The Copy-Paste Trap
This is embarrassing to admit, but it needs to be said: when I first started using LLMs, I would copy drafted emails or Slack responses without actually reading them carefully.
The embarrassment came in two forms. First, there would be mistakes—subtle tone issues, or context the LLM didn’t have that made the response slightly off. But more importantly,
the content wasn’t really mine. When someone asked follow-up questions or wanted clarification, I didn’t know the material the way I should have. I’d sent words I hadn’t fully internalized.
It’s like having someone else write your presentation and then trying to present it without rehearsing. The words are fine, but they’re not yours, and it shows when you’re pressed on details.
The LLM generates plausible content, but you own what you send. If you can’t defend it, explain it, or build on it in conversation, you shouldn’t be putting your name on it. When you send words you haven’t internalized, you’re not just risking mistakes—you’re missing the opportunity to deepen your own understanding through the act of explaining.
The lesson: treat LLM-generated prose the same way you’d treat a draft from a junior colleague. It’s a valuable starting point that needs your review, refinement, and ownership before it goes out under your name. Read it. Understand it. Make it yours.
When Human Judgment Remains Essential
LLMs can’t make judgment calls that require organizational context, risk assessment, or stakeholder priorities.
“Should we refactor this controller now or wait until after the release?”
“Is the added complexity of Redis worth it for this feature?”
“Do stakeholders care more about feature completeness or performance optimization?”
These aren’t technical questions with clear answers—they’re judgment calls that require understanding your team’s capacity, your organization’s priorities, and your system’s operational constraints.
The LLM can help you think through technical tradeoffs, but that ability to weigh technical purity against pragmatic constraints? To know when “good enough” is actually good enough? That judgment remains yours, built from years of seeing which shortcuts become technical debt and which become pragmatic wins.
You remain the senior partner in this collaboration. The LLM is the research assistant with perfect recall and infinite patience, but you’re the engineer with the context, judgment, and responsibility for the decisions.
Integration Patterns That Stick
After months of daily use, certain patterns have proven sustainable while others faded.
The patterns that stuck are the ones that fit naturally into existing workflows and enable more capable engineering rather than just faster typing:
Error interpretation - When you hit a confusing error, pasting it into an LLM becomes as automatic as pasting it into Google used to be. But the value isn’t speed—it’s understanding. LLM explanations teach you about system internals, not just how to fix this specific error.
Code explanation - When inheriting complex code, asking for architectural analysis before diving in becomes standard practice. You still need to verify and understand, but you’re building comprehension faster and more thoroughly.
Documentation drafting - After solving a problem, generating initial documentation becomes part of the solution, not a separate task. The result isn’t just faster documentation—it’s better documentation that you actually create instead of postponing.
Decision framework generation - When evaluating options, having an LLM organize comparison dimensions becomes the starting point. You still make the judgment calls, but you’re thinking through more dimensions systematically.
These work because they’re insertions into existing workflows that make you more capable, not replacements that make you faster. You still debug errors—but with deeper understanding. You still own architecture—but with better analytical frameworks. You still write documentation—but starting from drafts instead of blank pages.
Learning What’s Worth Reformulating
Not every pattern I tried worked on the first attempt, but that’s different from saying certain patterns fundamentally don’t work with LLMs.
Take standup generation. My first attempt was asking the LLM to generate my daily status update. It felt hollow—the LLM didn’t know what I’d actually worked on or what challenges I’d faced. The problem wasn’t that LLMs can’t help with standups; the problem was how I’d framed the task.
The reformulation that works: I write bullet points of what I did, and ask the LLM to help me articulate blockers or identify patterns I might have missed. The act of writing those bullets is where I think through my priorities. The LLM helps me communicate them better.
Or code review. I do use LLMs to review every change—I even like using one model to write code and another to review it. But I had to learn that I’m still the coding expert who needs to evaluate whether the review makes sense. LLMs can go down rabbit holes in code review just like human reviewers can. The reformulation that works: treat LLM review comments as questions to investigate, not issues to fix. “Is this really a problem?” becomes part of the review process.
The pattern I’m learning: LLMs work best when they amplify thinking you’re already doing, not when they replace thinking you should be doing.
Some problems require reformulation that isn’t worth the effort—you’d spend more time restructuring the problem than solving it directly. But we’re still in the early days of understanding what’s possible. Most problems are worth giving an LLM a chance, because that’s how you learn what reformulations work.
The failures aren’t dead ends; they’re how you learn to ask better questions and frame problems more effectively. Every “this doesn’t work” has taught me something about how to make it work differently.
Tools and Setup
The practical reality of what tools I actually use daily:
ChatGPT Plus ($20/month) - Handles all non-work sensitive material: personal productivity, learning, research, and general technical discussions that don’t involve proprietary information.
M365 Copilot (work-provided) - For anything involving work-specific systems, internal APIs, or proprietary code. The security boundary matters.
GitHub Copilot (work-provided) - Integrated into VS Code for in-editor code suggestions. I treat it like advanced autocomplete, not a thinking partner.
Security rule: Never paste proprietary code, customer data, or internal system details into personal LLM accounts. For work-related technical discussions, I either use M365 Copilot or sanitize code by removing business logic and using generic examples. When in doubt: “Would I be comfortable posting this on Stack Overflow?” If not, it stays in work-approved tools.
The Journey From Skeptic to Daily User
Let’s bring this series full circle.
I started as a skeptic—20+ years of watching hype cycles taught me to be cautious about revolutionary claims. The $20 experiment and Co-Intelligence gave me permission to test LLMs properly instead of dismissing them. Safe experimentation with travel planning showed me they could handle complex coordination without breaking production
systems.
Then came the specific wins beyond autocomplete: error interpretation that taught me system internals instead of just providing fixes, legacy code analysis that compressed archaeological research, documentation generation that transformed blank-page dread into focused editing. The breakthrough wasn’t speed—it was quality and capability
expansion.
Scaling that usage taught me the hardest lesson: the monolithic approach fails. One conversation trying to handle everything creates the same problems we learned to solve in software architecture. Specialized assistants with focused context produce dramatically better results, just like microservices with clear boundaries.
Now you’ve seen both sides—the daily integration patterns that work and the honest limitations that keep you grounded. This isn’t hype; it’s sustainable integration built on clear-eyed assessment of both capabilities and constraints.
What Matters Going Forward
The technology will evolve. Models will get more capable, context windows will expand, specialized tools will emerge. But the principles that make this work won’t change:
You remain the senior partner. LLMs are research assistants and thinking partners, not replacements for your judgment, expertise, or responsibility. Maintain critical thinking. Verify suggestions. Own your decisions.
Enhancement, not replacement. The goal is “more you,” not “instead of you.” Use LLMs to become more capable at work you already value, not to avoid learning or building expertise.
Focus on “better,” not just “faster.” Time savings matter, but quality improvements and capability expansion matter more. Tackle problems you would have postponed. Build understanding you would have left shallow. Write documentation you would have left sparse.
Integrate gradually with skepticism intact. Don’t abandon the critical thinking that got you here. Channel skepticism into careful evaluation. Test thoroughly. Fail safely. Build confidence through experience, not hype.
Your Next Move
If you’re still reading, you’re ready to move past curiosity into experimentation.
Pick one workflow pain point—error debugging, legacy code analysis, documentation writing, technology research. Create one specialized assistant thread focused on that domain. Use it consistently for two weeks.
Notice what changes. Not just “am I faster?” but “am I better? Am I tackling problems I used to avoid? Am I understanding systems more deeply?”
After two weeks, you’ll know whether LLMs belong in your daily workflow. Not because I told you they’re revolutionary, but because you’ve tested them against your actual work and honest standards.
The tools will keep evolving. Your judgment about when and how to use them should evolve with them.
That’s what skepticism earned you: not dismissal, but informed adoption on your own terms.
This concludes the “AI for Software Engineers” series. The journey from skeptic to daily user isn’t about abandoning critical thinking—it’s about channeling it into thoughtful integration. Thanks for coming along.