Overview
Imagine if coding were like cooking.
In the old days, developers were like traditional chefs, doing everything from scratch — peeling the onions, grinding the spices, and stirring the pot themselves. Then, automation tools came along, like kitchen gadgets: food processors, blenders, and smart ovens. Developers got a hand with the tedious parts. Life got easier.
And now?
We’re entering a whole new phase — it’s like chefs having an intelligent sous-chef who can chop, season, and even suggest new recipes while they sip coffee and supervise. Welcome to the era of Generative AI coding assistants, or GenAI, where your computer doesn’t just obey — it helps.
🍳 So, What’s Cooking in Software Development?
For years, we’ve been improving the ways we build software. First, there was Continuous Integration (CI) — think of it like a conveyor belt in a kitchen where dishes get inspected and corrected fast before they leave the counter.
Now, Generative AI — specifically Large Language Models (LLMs) like those that power ChatGPT — is stepping onto the stage. Except, instead of writing essays or poems, these models help developers write, fix, and improve code.
It’s not just spell-check for programming — it’s a co-chef who sometimes knows surprising tricks… and sometimes, well, suggests putting ketchup in your tiramisu. (We’ll get to the “quirks” part later.)
🤔 But… Does It Actually Make Developers Faster and Better?
Good question. Liang Yu and his team decided to find out. They took a deep dive into two real-world industrial projects — one in FinTech and one in Telecommunications — both using GenAI coding assistants like Codeium and Amazon Q.
They surveyed and interviewed 18 developers, software architects, and DevOps engineers to figure out:
- When does GenAI help?
- When does it trip over its own virtual feet?
- And what really changes when humans and AI team up?
⚡ 5 Things That Shape GenAI Productivity
Let’s break this down with simple examples:
1. Type of Task Matters (a lot!)
GenAI shines when the job is boring and repetitive — like rewriting documentation (“Javadoc generation”) or cleaning up messy code (“refactoring”). Think peeling a mountain of potatoes — tedious but easy for a robot.
But when the task is complex — like designing a new module or fixing a sneaky bug across a huge codebase — GenAI stumbles. One developer said it’s like asking your helpful sous-chef to bake a soufflé without telling them you live at 5,000 feet altitude. (Context matters!)
2. Skill Level of the Developer
Ironically, the better you are at coding, the less impressed you might be with GenAI.
Why?
Experienced developers already know the shortcuts and best practices. They often found it faster to do the job themselves rather than wrangling AI suggestions that were “almost right, but not quite”. (It’s like a master chef watching a newbie chop carrots — sometimes it’s easier to just grab the knife.)
But! Even the pros admitted: Sometimes AI showed them new tricks they hadn’t thought of before — little flashes of unexpected genius.
3. Domain Knowledge: Experts See Through the AI’s Flaws
Imagine building an intricate Lego castle. You know exactly how each piece fits, but your AI assistant — bless its heart — has only seen castles on Instagram.
Developers steeped in their project’s history and quirks found that GenAI often missed the fine details. It made confident but sometimes laughably wrong suggestions. For new folks, though, it sometimes offered quick, helpful starting points.
4. How Well GenAI Fits into Your Tools (IDE Integration)
Good news! Integrating these GenAI tools into coding environments (IDEs) was usually super easy — most developers said it was like plugging in a microwave: click, done.
This mattered because switching between apps (like flipping between chopping veggies and checking recipes online) disrupts your “flow.” Staying inside the IDE, with GenAI whispering suggestions right there, helped coders stay focused.
5. Practice Makes Perfect (Tool Familiarity)
At first, using GenAI felt a bit awkward. But after a few months, developers got the hang of it — like learning a new dance partner’s moves.
Over time, they learned tricks like iterative prompting (breaking tasks into smaller instructions for better results) and clarifying requests to guide the AI more precisely.
Pro tip: Don’t bark a huge, complicated prompt at your AI buddy. Break it down into smaller, step-by-step instructions.
🎯 Big Lessons from the Field
Here’s the real meat of the study:
| 🍔 Lesson | ✨ What It Means |
|---|---|
| Different tasks = different GenAI success rates | Great for easy stuff, shaky on complex custom work |
| Staying inside your IDE is a win | No more distracting Google rabbit holes |
| GenAI doesn’t “get” deep project context | It knows lots, but not your secret sauce |
| Refine your prompts | The clearer your instructions, the better the AI behaves |
| No support for custom design rules (yet) | Organizations have to double-check AI work manually |
| You still need code quality checks | Don’t blindly trust — verify, always |
🧠 Definitions, Made Easy
Generative AI (GenAI): AI that can create things — text, code, images — instead of just analyzing or retrieving.
Large Language Models (LLMs): Giant AI systems trained on mountains of text (and sometimes code) to predict what comes next in a sentence (or command).
Productivity (in this study): Not just cranking out lines of code — it means satisfaction, learning new skills, quality of outputs, and saving time.
📉 But Wait — It’s Not All Sunshine and Rainbows
There are real challenges:
- GenAI can overpromise and underdeliver when context is complex.
- It doesn’t automatically obey your company’s unique coding rules.
- Verifying AI-written code is still critical — security and performance issues lurk in unverified outputs.
Think of GenAI as a powerful intern — brilliant but naive. It can help, but you better double-check its work.
🌟 Final Thought: Friend, Not Replacement
At the end of the day, GenAI isn’t here to steal developers’ jobs. It’s more like a trusty R2-D2 — a sidekick that helps you focus on the exciting parts while it handles the grunt work.
But — and it’s a big “but” — the human remains captain of the ship. Creativity, judgment, and critical thinking? That’s still 100% on us.
As the tech keeps evolving, one thing is clear: Coding with GenAI is less about writing code faster — and more about working smarter, learning quicker, and focusing on what really matters.
References:
Paradigm shift on Coding Productivity Using GenAI – Liang Yu
Assessment on the Effectiveness of GitHub Copilot as a Code Assistance Tool – Carlos Adriano Gonçalves and Célia Talma Gonçalves
Comparing the Effectiveness of Generative AI for Learning and Developing Flutter Application – Panha Heng, Karn Yongsiriwit, and Parkpoom Chaisiriprasert
