Is AI Better Than A Human Copywriter?
Is a robo-writer 🤖 better than a human copywriter? I decided to put it to the test. May the best ad win. TLDR… but you should.
So, Is AI Better Than A Human?
As the owner of the world’s premier copywriting service, Ad Zombies, I’ve played around with AI over the last few years. When you read the online comments and testimonials, it appears AI copywriting has come a long way since it was introduced to the world. I still remember testing AI copy when it was about a month old. It was terrible!
Since AI learns as it goes, it should deliver copy equal to or better than human copy, right? I decided to put my thoughts to the test and set up a few campaigns to test human copywriting against AI copywriting.
While there are a number of AI tools available, I had to pick one, so I chose one of the top 3.
At a lunch meeting with great clients, Janak and Urvi, a few weeks ago, Janak recommended I test more than one length of copy for better, more complete data. I thought that was a good idea, so I tested short (1-2 sentences), medium (around 75-100 words), and long copy (around 150 words).
Audience Selection
The audiences were identical (except for the individuals the algorithm delivered to on any given day) since we don’t actually know the exact human who was served the ads. The audiences were as follows:
Top 1% Value-Based LAL (This audience is created by Facebook and looks like the top 1% of our highest value clients to their algorithm)
Small Business Owners
Marketers
Business Page Admins
To best ensure pure data, we excluded our active subscription and one-off purchase clients from the test, assuming a fairly decent match rate.
Using ad set bid strategy, each audience was given a $25 per day budget across 4 human ad sets and $25 per day across 4 AI ad sets for a total of $200 per day.
The 3 ads (short, medium, and long) were placed in a single ad set for each audience. To clarify, the audiences each had their own ad sets, and each ad set contained the 3 ad length variants. So we had a single campaign for Human copy, with 4 ad sets and 3 ads in each, and a single campaign for AI copy, with 4 ad sets and 3 ads in each.
I decided to follow the Ron Popeil methodology of “set it and forget it,” starting at midnight GMT-7 and ending after 10 days at the same time.
The first few days were boring. The ads all sucked canal water, and I thought I’d have to start over if they didn’t start to gain traction, but eventually, the algorithm found its audience and started delivering consistently.
Creative
Since humans and AI had to do battle on a level playing field, I input the following creative brief information for both.
Objective: Convince readers to let Ad Zombies write your ads, emails, and web copy.
Style: Since I am a snarky wise-ass, I told the AI to be snarky wise-ass too.
Go, Human!
I wrote my copy first. To keep the comparison fair, I am sharing the medium-length copy.
Human Headline: More Viral Than Monkeypox
Human Body: We survived a pandemic and now we have to deal with Monkeypox? Can we get a break? Ummm, yes.
Look, there are some things we can’t avoid, like Monkeypox. 🐒 Writing crappy ads, emails, and web copy is TOTALLY avoidable. In fact, if you’ve read this far, we’re leading by example… and winning.
You’re already stretched 5x more than a pregnant woman with quintuplets. Today you might be pulled in 100 directions, and the one you suck at, no, dread, is copywriting.
Stop writing today. We have the cure for Monkeypox infected copy. That’s not actually true, the FDA cannot confirm that.
My copy totaled 102 words plus the headline.
Go, Robo-Writer!
Next, I let the AI robo-writer do its thing. I am also sharing the medium form AI copy, to give you an idea of what the AI spits out. BTW, we used the AI copy with ZERO intervention, meaning, a human didn’t edit it. The AI-generated it, and we simply copied and pasted it in.
AI Headline: Get more customers with our hilarious headlines
AI Body Copy: Hey, you. Yeah, you. The one with the drool on your keyboard from staring at cat pictures all day. You want to make more money, don't you? Of course you do! And Ad Zombies can help make that a reality.
We're the best ad writers in the business, and we know how to get people to take notice of your product or service. So why not let us write your ads for you? We promise they won't be boring or lame. In fact, they'll be so good that people will actually want to buy what you're selling!
So what are you waiting for? Click this ad and let us show you what we can do!
The AI copy spit out 115 words plus the headline.
The Results Are In
After the 10 days were complete, it was easy to see the winners.
For short-form copy, AI copywriting beat me with value-based lookalikes. The final ROAS was 3.1 for this ad set. AI failed to produce profitable results for the short-form copy with the 3 remaining audiences. My copy out-performed the AI copy pulling a 2.1, a 2.8, and a 3.4.
When looking at medium-length ads, (the ones I highlighted above) AI for the win! Again, a single audience, this time Business Page Admins, squeaked out a barely profitable 1.5 ROAS. For the 3 remaining audience segments, my copy delivered a 2.3, a 2.5, and a 3.1
The long-form copy was a loser for AI. Not one AI ad set was profitable and the one that came closest only hit a .8 ROAS. For the human copy, all 4 were profitable, one of them barley at 1.3 with the remaining ad sets pulling a 1.9 a 2.2, and a 2.9
Hits & Misses
Let’s talk about the 800-pound gorilla in the room. Do we really drool all over the keyboard from staring at cat pictures on the internet? Way to bring up an out-of-context cat reference from 2005 in a 2022 ad. Is the AI being programmed by out-of-work comedy writers from the Catskills? (that’s a place, not another cat reference, just Google it). Meow!
Another area AI didn’t impress me was the contextual understanding in the headline. In the medium-length copy above, it understood the snarky or wise-ass tone for the ad body, but the headline referenced humor without adding humor. It’s the equivalent of a standup comic saying I’m funny, but never delivering a joke. So it missed on the headline and did okay on the body copy. The line about drooling on your keyboard was good, but the remainder of the ad felt mechanical to me, dare I say, robotic? The line about not being boring or lame made me feel both.
Final Thoughts
AI has come a long way since V1, but it has a long way to go. What AI seems to miss is the human emotion, the heart that goes into the written word. AI isn’t going to tap into something pop-culture-ish and turn it into an ad that makes sense. Additionally, I think AI is good for a foundation, but not a finished product. In both of the above examples, I chose not to have a human editor massage the copy. I wanted it to be unedited, man versus machine. That means if I had typos in my copy that was on me. I felt an editor might find and clean up something, thus giving me a slight advantage over AI.
While AI has evolved since its inception, the reason I think we are years away from using it at scale is the lack of emotion and the ability to contextualize current events, humor, etc.
Will there be fully autonomous robo-writers 🤖 in the future? Probably. But until a robot feels like a human, I think copywriters will continue to hone their craft and deliver results for clients.