Duck vs Machine: We took on AI and this is what we found
Ever used AI tools to write for you? We asked our followers to choose between human- and machine-generated content, and this is what they preferred…
With the launch of OpenAI’s ChatGPT in November 2022, generative artificial intelligence (genAI) burst into the collective consciousness. While genAI has been transforming businesses for a while behind the scenes, companies’ and individuals’ adoption rates are now reported to be sky-high for all sorts of written workplace communications.
Caution has been urged from some quarters about the rush to adopt genAI, with warnings over ‘technical debt’, data security and brand reputation, but that hasn’t stopped 70 percent of respondents to a Grammarly-sponsored survey turning to the technology for most or all of their writing and editing (Maximizing Business Potential With Generative AI: The Path To Transformation, Forrester, July 2023). This might sound like an absurd proportion, but when you consider Google’s Smart Compose for Gmail has been around since 2018, it’s not farfetched at all.
Of course, we would be real (King) Cnuts to think we could turn back the tide on this technology. It’s been in the pipeline for over 60 years, after all. But at ChattyDuck, we like to believe that human content writers still provide a valuable and necessary service. So, to test our hypothesis, we made the (admittedly quite risky) decision to pit ourselves against various AI tools.
That’s right: it’s Duck vs Machine, and the stakes couldn’t be higher.
The contenders
We decided to test three different tools: ChatGPT because it has garnered so much publicity and, consequently, new users; SocialBee because it specialises in social media caption writing, which is one of our services; and Copy.ai because it seems to target the marketing function, as do we.
Our Director, Jim, took on ChatGPT. I faced off with Copy.ai, and Content Manager Dom was up against SocialBee. Both Duck and machine received the same fictional brief, and we asked our social media followers to vote on which they considered the most persuasive copy.
The verdict? Fortunately for us, #TeamDuck came out on top. A convincing 84 percent of respondents on our series of three social media posts preferred the human writer’s work to that of AI.
In praise of genAI
There’s no denying the many strengths of free genAI tools like ChatGPT, SocialBee and Copy.ai for content writers. They’re undoubtedly of value for those who wouldn’t describe themselves as writers but are nevertheless under pressure to produce a steady stream of fresh content for their organisation.
Each tool coherently answered the brief that was set, with SocialBee even including hashtags and emojis on request, and they did so in a matter of seconds rather than minutes. What’s more, the tone they used was broadly similar to that adopted by our human writers. With some light editing, their output would be passable.
Which is all to say that genAI tools are great for quick idea and outline generation, creating summaries, getting over writer’s block, and generally doing tedious, time-consuming tasks that make humans yawn. If you’re a startup or sole trader with next-to-no budget for content creation, judicious use of genAI could help get you out of the starting blocks.
In favour of Duckdom (aka humankind)
Limitations of genAI tools include the dearth of verifiable sources and their reliance on outdated or incorrect information. We found that some tended to take word counts with a pinch of salt or ignore them entirely, and it’s no surprise that indiscriminate use of AI-generated content has the potential to damage your SEO ranking rather than help it.
Where genAI really falls down, in my opinion, is in its lack of empathy, humour and – most damning of all – creativity. Take this sentence from a recent London Review of Books article: ‘Is it okay to club a fish over the head because lentils make a philosophy professor fart?’ Can you imagine an AI tool coming up with that line? In the context of a 7,785-word article where it not only makes sense but advances the argument?
Words can be connected in infinite ways to express infinite ideas. That’s the beauty of language. The beauty of the human brain is it can connect words and ideas that have never been connected before. AI tools in some senses do a great job, but they’re only collecting words and ideas that have already been connected.
A collaborative future?
For organisations looking to generate original written content for publication, AI-generated content appeals because it seems cheap and easy compared to human expertise. The technology has undeniably come a long way, and we can’t argue with its cost-effectiveness.
But as the internet is flooded with AI-written content, it will become ever more difficult to stand out against the competition. What AI produces is (IMO) all kind of boring and same-y. Which is understandable when you think it’s effectively averaging out everything that’s been written about your topic.
As the results of our experiment show, human opinions on the persuasiveness of written content will always be subjective – only one round out of three in Duck vs Machine was unanimously won by a Duck. And who can say whose writing future generations of humans and of AI will prefer?
There is no denying the usefulness of genAI, so we’re sure to see ongoing collaboration between humans and machines. But for now, at least, we believe there needs to be a skilled human at the heart of writing copy and content. And we’re not just saying that because we’re human writers. There may come a time when #TeamDuck is thrown out with the bathwater, but, for now, genAI is still a tool rather than a complete solution for content generation.
Full disclosure: this blog was written with ChatGPT.*
*Just messing. It’s 100 percent Duck. 💯🦆😉
Was this post useful? Follow ChattyDuck on LinkedIn for monthly insights into messaging, branding and content creation.