By Gil Brodnitz, Co-founder and Managing Partner, Aileron Group
This is the first in an occasional series sharing practical experiences using GenAI to get work done — written for anyone who says “this stuff freaks me out!” I hope it helps you explore this technology with a little less anxiety.
At Aileron Group, we use GenAI tools every day. That means we also run into their limits every day. This is a story about one of those moments.
Today’s task: combine scanned receipts into a single PDF for an expense report. Five PDFs, five JPEGs. I can do this in Preview on a Mac in about two minutes. I thought ChatGPT’s natural language interface might make it faster.
It did not.
What Happened
I gave ChatGPT 4o clear instructions. It converted the JPEGs and combined them into a PDF but ignored the PDFs entirely. I corrected it. It tried again and came back with a file that contained references to the PDFs, or just left notes like “more of the same.” After five or six attempts, I finally got a usable file for my expense report.
If you’ve ever tried to get a four-year-old to clean up their toys, you’ll recognize the dynamic.
The Fix
After regaining my composure, I asked ChatGPT why this was so difficult. The answer was clear, if maddening: I needed to be utterly, explicitly specific in my prompt, leaving zero wiggle room.
ChatGPT suggested an improved prompt — laboriously specific — which worked on the first try.
But Why Does This Happen?
Two things worth understanding.
First, this may simply be the wrong tool for this kind of task. Until we have real GenAI agents, these tools are better at analysis than at performing discrete tasks. I tested Claude 3.7 with the same request. It simply said it can’t do document creation tasks like this. ChatGPT isn’t blocked, but it’s clearly not optimized for it either.
Second, when we ask GenAI to do something it isn’t good at, it responds with what looks like lazy misunderstanding. The model isn’t being difficult. It’s working at the edge of what it’s built for.
What I Took Away
When GenAI doesn’t perform as expected, you have two options: find a different tool, or reshape your prompt to be far more explicit than feels natural. Not all chatbots are the same, so it’s worth experimenting with your options.
And here’s the part that genuinely surprised me: asking GenAI for advice about why something isn’t working is a game changer compared to traditional software help functions. It actually told me what was wrong and how to fix it. That’s not something you get from a help menu.
Want to read more about this? Join the conversation on LinkedIn.
Gil Brodnitz is Co-founder and Managing Partner of Aileron Group, an AI-first enterprise consulting firm. Follow along for more practical GenAI field notes.
