Musings from an MIT Study: Navigating the Intersection of Human Creativity and AI in Marketing

In May of 2023, two researchers at Massachusetts Institute of Technology’s Sloan School of Management put marketing content created by ChatGPT-4 against content created by professionals from a major consulting firm to see what people preferred. Did the roughly 1,200 participants favor the human content? Or was AI found to be the more competent marketer? Yes, and yes. 

MIT's study looked into whether people's perceptions of marketing content — specifically product descriptions for goods like an air fryer, a tumbler, and a projector and campaigns for “eat less junk food” and “wash your hands,” among other subjects — changed based on their awareness of its creation process. The research looked at three scenarios: when people had no idea about the content's origin, when they knew it was a human-AI collaboration but were uncertain about the specifics, and when they were fully informed about who created what. 

Here are two of the biggest findings from the report:

• People ranked content created entirely by ChatGPT-4, or curated by the AI following human input, higher than content created solely by human experts, or humans who had the final say on AI-generated content. 
• If participants knew content was human-made, its perceived quality got a boost due to “human favoritism:” people preferring content made by people. However, knowing AI had a hand in content generation didn’t change its perceived value. This contradicted predictions that people have “algorithm/AI aversion:” a bias against AI content resulting in lower satisfaction and willingness to spend advertising/marketing dollars on that content. 

In short, the study posits that consumers may slightly prefer content made by other humans while acknowledging that AI-generated content can be on par, or even better, than that human-generated content. Human favoritism and a lack of AI aversion can co-exist. 

What does this mean for marketing leaders and teams putting out content with AI? I spoke to Deanna Zammit, founder and head of content at B2B-focused creative content agency Industrious Content, about the implications of the study. 

She saw the study as proof that generative AI can free humans from simple creative tasks like writing product descriptions, allowing marketers to focus on more complex, engaging, and strategic work, like devising editorial strategies or uncovering novel insights. 

But no matter if people think AI–produced content is passable or superior to human-made content, “It's in the best interest of brands to make sure that the human fingerprint is on all of their content,” Zammit said. “And the reason is that marketing, at its core, is people trying to connect with people. Only humans can do that and can tell a human story.”

For Zammit, elevated human storytelling will prevail in the long-run. However, the MIT study opens the door for wider conversations about challenges marketing teams have to face as they use AI in the present, like being transparent about AI’s use, using AI for cost and speed efficiency while maintaining the human touch for high-quality content production, and companies forcing AI content creation unnecessarily. 

Hear more of the leader’s thoughts on the MIT study and the future of AI in marketing. 

‍In your opinion, what’s the most significant finding within the context of marketing and why is that finding important?

The MIT study gives us permission to free humans from low-level creative work. We don't necessarily need to be asking humans — especially if they're people with creative ambitions who want to do more — to work on things like product descriptions, catalog copy, or social media posts that then get massaged by somebody who really knows the brand. There should always be a human at the back end making sure that it's all good, but we don't necessarily need to be making the first pass at it when we have generative AI to help us.

I'm a professional writer, and so I didn't really want to hand the machine a win. I want to keep my sense of specialness in that only I can do certain things. But the fact is, if AI is writing product copy, I don't have to; I get to write something else that's much more interesting. It can also help broaden the capabilities of marketers where maybe their primary skill set isn't copy or image creation. 

This study gives us permission to elevate human creators so we can focus on telling bigger, more important, more interesting stories. We can spend time on editorial strategy. We can spend our time talking to subject matter experts and unearthing new insights. Generative AI isn't going to come up with new insights. It's derivative, it can only produce based on what's already been done. That's up to us. And this study shows us that we can have the human brains that are detailing the product benefits of an air fryer be applied to bigger challenges, where their time and expertise will be of better use.

‍What does human favoritism look like in marketing in your eyes? 

We can think of human favoritism in three ways. There's the initial skepticism. There are the ethical implications. And then there's how we assign value to things. 

With skepticism, there's an element of not believing a machine can do what I can do. At Industrious Content, we’ve been explaining AI — whether its generative AI, computer vision, or something else — for years. And the message is always the same: that it’ll elevate your job but not eliminate it. Most professionals need convincing but marketers, not so much. We surveyed 150 marketers on whether or not they've been using generative AI. Of the 70% who said they use it, 63% were excited about it. Only 8% of users were skeptical. So that points to the idea that marketers are impressed with the results of generative AI.

There's also an ethical component. We have to really understand how to properly label it. We have to understand whether inaccuracies are being introduced. We have to understand how it's going to affect jobs and job creation. That's something we have to think about when we consider why we prefer human-generated content. 

The third part is just generally how we assign value as human beings. We're already assigning additional value to things that are handmade or artisanal because human beings crave connection to one another. A handmade chair or small batch beer conveys expertise and specialness. People are willing to pay extra for that. Content really isn’t any different. As journalists, professional writers, and content creators, we bring a certain expertise, emotional intelligence, and intuition to the table and so that premium content is valued higher. 

The MIT study found that human favoritism exists at the same time as a lack of AI aversion. What implications does this lack of AI aversion have for marketing strategies? 

The study found that people prefer the AI content when they didn't know AI created it. And the AI content was actually better. The humans over-thought it. Creating a 100-word description or marketing copy for an air fryer, for example, isn’t the most interesting creative task. As human beings, we want to make our work interesting. So we try to be a little too clever. That’s what I saw with the human content. They worked into the information with too long of a runway.

By contrast, the AI copy was direct, informative, and exciting. The AI wrote something akin to what you might see on an Amazon listing. The assignment was to write persuasive copy that is going to get somebody to buy something. Frankly, the AI content served the purpose better. 

When it comes to general sentiment about generative AI, people feel kind of cautious about it. They know that it's the future but they're not sure if they should be excited. And a lot of them are still concerned about whether they’re going to know if something is written by AI. They might now know what’s real or if AI is going to steer them wrong in some way. 

These are very human fears that, over time, will be dispelled with exposure to AI-generated content. But it's going to take some time for consumers to get used to the idea. 

What does AI transparency in marketing look like for you?

It depends on the kind of content and how the AI was used. I don't know where you put an AI disclaimer in something like product or email copy, or if that's even necessary. When you're writing anything that's supposed to be informative, persuasive, or show expertise of a subject matter and AI had a hand in creating any of it, there should be a disclaimer. There should be something that explains how it was used or assures that it's been fact checked and edited by a human being. I think that is going to be absolutely necessary.

Ultimately, we're going to need to have some kind of standardized labeling that marketers agree on as a universally understood shorthand for AI disclosure to build trust.

What’s a mistake that some companies are making as they try to roll out generative AI tools to their (marketing) teams?

We're in this experimental period and most marketers and organizations are still trying to figure out where AI fits in their workflow. A mistake that some organizations are making is trying to get their creators to integrate AI in a way that doesn't make sense for those creators. 

So for example, writers may be able to produce something faster and at a higher level than if they used AI, especially if it's something that's longer form. They're professional content creators, so writing and editing prompts might create too much intellectual friction for them. They might prefer to do the first draft themselves.

‍What other potential challenges do you foresee for marketers and balancing the use of AI and retaining the human touch?

The challenge is that AI helps you create content cheaply and quickly. And it empowers folks with maybe lower level skill sets to create and approve content. So the question is, where does the desire for high quality content intersect with a desire for low cost content? And we've already seen this a little with content farms and SEO factories. Informational writing becomes commodified and the quality and the value of it sometimes drops.

For marketing teams that want to use AI to accelerate their content creation, the biggest challenge is going to be empowering storytellers and putting them in charge of the content. Not every marketer is a storyteller. And that’s okay. Some are strategists. Some are lead gen experts. But to create great branded content you need to have a storyteller on your team like a journalist, producer, creative director, designer — it doesn’t matter. And they may or may not use AI tools to help create the stories, but you need them there. 

In our Industrious Content study, we asked marketers which skill set was most uniquely human. The winner was editorial judgment: the ability to understand human stories and how to connect with another human being. That skill set transcends the mechanics of writing. It's the uniquely human ability to know: what's important, what’s intellectually or emotionally compelling, what's unique, what's cliche, what's off message, and what's off brand. 

[.blog-quote]“At the end of the day, it's human storytelling that's going to prevail. And I think that’s going to be the biggest challenge for marketers.”[.blog-quote]

As a marketer, you want to connect. You want to establish a relationship with audience members so they’re loyal to your brand and engage with your content. So you need human beings sitting on top of AI in order to give direction, check quality, and check facts. 

With more people creating more content, the content that is going to win attention is going to have to be that much better. Brands who put knowledgeable producers, storytellers, and creative directors in charge of their content are going to win because they're investing in quality versus simply churning stuff out.

At the end of the day, it's human storytelling that's going to prevail. And I think that’s going to be the biggest challenge for marketers, especially those in organizations that are very cost conscious. More content does not necessarily mean more quality. Quality content can do a lot more than reams and reams of mediocre content.