Artificial intelligence

Is ChatGPT your next email developer?

There are two ways to build a marketing email:

  • Hand-coding
  • WYSIWYG editors

We swear by the former, not only for quality but also for speed. But what if there’s an even better, quicker way?

Enter ChatGPT. Much hype has surrounded the AI platform’s ability to code. It can conjure up HTML and CSS in seconds. So too can it generate Javascript functions or back-end PHP or even truly hardcore programming such as C++. Whether or not it does it correctly is a different matter.

Let’s not worry about that just now. We’re here to put AI email development to the test, so let’s find out if ChatGPT can put together a responsive mailing.

The quirky world of email development

If you work in email marketing in any capacity, you likely already know that it requires some unusual coding techniques. There are lots of devices and email services out there, and they have widely different ideas about how HTML and CSS should be interpeted. In order to construct a mailing that looks presentable on all of them, the developer needs to be aware of these limitations and inconsistencies and the arsenal of tricks to work around them.

Has this niche set of knowledge made its way to ChatGPT? We’ll start with a bare bones request.

Prompt:

Code a responsive email template

Result:

Unusable!

ChatGPT has produced a very basic HTML document with some styling, but I wouldn’t call it an email template. It doesn’t include any means of stacking content on mobile, and the structure is based on HTML div elements rather than tables. While divs are the building blocks of a web page, tables remain the most reliable method for email.

On the plus side, it has picked an inbox-friendly width of 600 pixels. And it’s nice to see that accessibility has been implemented via an image description and a proper heading tag.

My request was extremely minimalistic. I need to do my part here too, and that means being more specific about what is needed.

A little lot more instruction

Take two. We don’t want divs, so let’s tell ChatGPT to use tables. There are some basic universal requirements in responsive email, so we’ll nudge it in the right direction regarding those.

Prompt:

Code a responsive email template, using HTML tables for structure. Set the width to 600 pixels on desktop, with a fluid width on mobile. Include CSS classes to enable stacking of content on mobile devices. Include all known email client fixes that are still relevant. Set the page background to a light grey colour, and the email content area to white.

Result:

Better… but still broken beyond repair.

This time it has used tables for structure, so that’s a major improvement. It has also set a breakpoint. That’s the backbone of responsive email code and the point at which mobile-specific styling is triggered. There’s some kind of attempt at stacking code, but I can see at a glance that it isn’t going to work. We’re also missing the usual pile of fixes that make an HTML email possible.

A rethink is needed.

A different approach

Here’s what we’re going to do: hand code a simple email, and then provide ChatGPT with detailed directions in order to recreate it. This is a reverse way to approach our project, but perhaps if ChatGPT has a more defined goal it will be able to produce a usable template.

Our email will have a main image, intro paragraph and a button. Under those will be a couple of secondary features laid out side-by-side on desktop, and stacking on mobile. For the sake of this test, let’s forget about any header and footer.

Image of our intended email layout

Now for our prompt. It’s going to be a long one. Let’s give ChatGPT a fighting chance and focus primarily on structure rather than styling.

Prompt:


Code a responsive email template, with the following requirements:
• 600 pixels wide on desktop
• Fluid width on mobile
• A page background colour of #f1f1f1
• Email content area background colour #ffffff
• A hero section with an image, heading, paragraph of text, and a button
• The hero image should be 600 pixels wide, to match the email content area
• Button should be pill-shaped, with a background colour of #a56e53 and white text
• Under the hero section should be two secondary features
• Each of these must also have an image, heading, paragraph and button
• Secondary feature images will be 290px wide on desktop, to match their containing column, and expanding to full width on mobile
• Hero text and button should be a bit larger than those of the secondary features
• These secondary features should take the form of adjacent columns on desktop, each at 290 pixels wide
• Place a 20 pixel gap between them
• The secondary features must stack into a single column on mobile
• All parts of the email should have 20 pixels of padding on each side on mobile, except for the hero image which can be full width and touching the edges of the viewport
• All body text should follow this font stack: HelveticaNeue-Light, Helvetica, Arial, sans-serif
• All body text should be colour #61524b
• All heading text should be colour #a56e53
• Use lorem ipsum placeholders for text
• Enter all hrefs as # placeholders
• Apply links only to buttons. Do not apply links to images
• Include all known, currently-relevant email client fixes
• Include CSS or HTML comments around each section to explain what it is or does
• Set a mobile breakpoint based on a max width of 639 pixels
• To ensure compatibility with Outlook and other email clients, use HTML tables for structure

Result:

Nice try… sort of.

In order to test this properly, I’ve saved a local copy and manually added my image references. Here’s how it looks in a browser:

Image of ChatGPT's email as seen in a web browser

At first glance, that isn’t too bad. The general layout, colouring and sizing are all correct. So too are the button shapes, and the secondary features switch to a single column on mobile as requested.

But there’s some strange overlapping going on. Our images are offset to the right, and sit partially over the grey background. This in turn causes some unwanted horizontal scrolling on mobile.

Behind the scenes, the true extent of the errors comes to light. It has reverted to a div-based structure, and uses some CSS code that won’t work universally in email.

Nonetheless, for the sake of completeness I’d like to test this as an actual email. It works, more or less, on iPhones and the Gmail app. Webmail is a mixed bag. Outlook however is where it all falls apart:

ChatGPT's email as seen in Outlook 2019

Outlook is the primary reason that email development requires such unorthodox coding methods. A lot of code that works just fine on a website, simply isn’t recognised by Outlook. Here we can see that the adjacent columns have failed and the pill-shaped buttons are reduced to tiny rectangles. To fix that would entail a complete recode.

No need to re-invent the wheel

So far, ChatGPT has failed to code a responsive email from scratch. To add some faux drama, let’s say our make-believe client is becoming impatient waiting for our make-believe email.

It’s time for a last ditch effort. At The Email Factory we already have a tried & tested template. We don’t need a new one. How about we give our base template to ChatGPT and then ask it to complete some content within that framework?

Result:

Now we’re getting somewhere.

But that doesn’t mean success. This time the template works reasonably well in a browser and even in Outlook, although the dodgy buttons are still present. The secondary features however don’t expand to full width on mobile:

ChatGPT's email using our template, as seen on an iPhone

That can however be easily fixed manually. In fact, it may be feasible to fix everything in this code rather than to start again. But I don’t want to do it myself, as that defeats the purpose of this experiment. Instead I’ll tell ChatGPT what needs to be corrected.

A few pointers

Final try. I’ve fed back some information to ChatGPT for it to make the necessary changes.

Result:

A huge step backwards.

Well, that was a big let-down. Instead of applying some finishing touches to the template, the layout has exploded. It no longer stacks on mobile. The code is now full of Microsoft conditional statements – a technique that should only be used sparingly and under specific circumstances. And the buttons? Still ugly in Outlook:

ChatGPT's corrected email as seen in Outlook 2019

Maybe with painstakingly detailed prompting and a lot of patience we could finally achieve a working email. But we’re already far beyond the point of convenience.

The current state of play

In my experience so far, ChatGPT has only done one thing consistently: fail. And I don’t only mean within the limited scope of this one project. I’ve had similar results when trying to generate marketing copy or website code. The output is usually along the right lines but ultimately too broken to actually use.

It’s clear that I set my expectations too high. The tales of ChatGPT’s near-miraculous capabilities were captivating, so perhaps the reality was always going to be disappointing. If there’s a perfect way to illustrate ChatGPT’s close-and-yet-so-far nature, it’s to ask it for an anagram.

Prompt:

Tell me an anagram of "The Email Factory"

Result:

The Fairy Comet Elf

I’ll save you the bother of checking that – it’s wrong. Trying to recreate that manually, letter by letter, results in this:

The Email Fctory e f

Re-evaluating our AI email development experiment

This project is arguably unfair from the outset. ChatGPT is a language model. Just because it can output code doesn’t mean it is a coder, or even knows what programming is.

Even so, it’s widely known that ChatGPT can generate code. So, despite all the mistakes and unusable templates, the fact that it can make a somewhat reasonable attempt is impressive.

Where do we go from here?

ChatGPT and AI in general are progressing at an incredible pace. It wouldn’t surprise me if everything I’ve written about AI email development above is laughably antiquated one year from now.

Perhaps when that time comes, I’ll prompt it to:

Write an article about how you surpass human email developers

Email Marketing

Is ChatGPT the messiah or a very naughty boy?

Is ChatGPT for email marketing subject lines the Messiah or a very naughty boy?

So with Intuit Mailchimp running nationwide radio advertising announcing its new subject line analytics tool, for the record we built something very similar back in 2018 and you can read about it here. Subject line optimisation and the role ChatGPT could have in it has become increasingly prevalent. Every second post on LinkedIn seems to be about ChatGPT and how it’s going to revolutionise content writing and that includes subject lines.

Having tested ChatGPT for subject line content generation exhaustively, here are my thoughts. I preface this by saying these are my thoughts, not that of the business but those of one cynic who works for the company, just in case Microsoft come after us. Needless to say my conclusion is… ChatGPT is a fraud! There I said it.

ChatGPT does not generate better subject lines for open rates. I can’t write email copy using it without having to edit it enormously. Actually taking more time on the edit than it would if I wrote my stream of consciousness!

For the purposes of this blog let’s concentrate on subject lines. I asked it for 5 subject lines for an electronics company selling TVs and it came up with the below.

The results were as follows

ChatGPT question on subject lines
Generate 5 email subject lines

Now I thought these a bit “Stateside” and as I am based in the UK and wanted to compare it to UK companies’ subject lines I changed the question to reflect that…

ChatGPT UK version of subject lines
Generate 5 email subject lines in the UK

ChatGPT thought it important to inform people that this was for UK TVs showing a scant understanding or “intelligence” of the actual requirement. Now some of you will say that perhaps I should have been smarter with my question but and here’s the rub – if I can be smart enough to ask the question in a way that ChatGPT spits out a killer set of subject lines, then I can probably write those subject lines myself way quicker. I’ve already spent more time than I’d like to have done asking the 2 questions and not getting answers I could use.

So then I took a real life situation.

I took 5 subject lines from a client which had been used in specific campaigns and asked which one it thought would be most successful. The first time I did it, it ignored the first subject line because I hadn’t put it on a separate line from the question. It then subsequently said the following…

ChatGPT subject line performance
Which subject line would perform better

So I asked my question again but this time put the first subject line on its own in the question. ChatGPT then contradicts itself from the first set of questions and decides that the subject line that was missed out was in fact the best but the one it previously said was the best is now not so good because it’s too generic. But it doesn’t move it down from 1st to 2nd but from 1st to 3rd!

ChatGPT subject line performance
Which of these subject lines would perform better

So I asked it to rank them in order of effectiveness and it said this

ChatGPT subject line rankings
Rank these subject lines by performance

What happens if you apply real intelligence instead of artificial?

Now I have real life data, I have a subject line tool which we built in house, which I can ask the above questions and get real answers. You can read about it here or Dela Quist’s SubjectLinePro or even Intuit Mailchimp’s version of our tool but ChatGPT, well I found it wasn’t useful at all. In fact it got things completely wrong.

So in order of their effectiveness in real life marketing solutions the rankings were…

1. Superb Sale savings… starting online today!

2. Our best 85” TV megadeal ever…

3. Samsung, LG TV and soundbar offers for an improved home cinema experience

4. Inflation-busting TV megadeals including a Samsung 70″ at £699…

5. Bring your TV to life 

And to remind you what ChatGPT said

  1. Samsung, LG TV and soundbar offers for an improved home cinema experience
  2. Inflation-busting TV megadeals including a Samsung 70″ at £699…
  3. Superb Sale savings… starting online today!
  4. Our best 85” TV megadeal ever…
  5. Bring your TV to life

At least we all agree that last one was the least effective. So my advice is, while AI is great and ChatGPT in particular is fun to play around with I will take real intelligence over artificial intelligence any day of the week! So in conclusion ChatGPT is a very naughty boy!

Email Marketing

The future is here. How will AI impact your emails?

Artificial intelligence is a hot topic. There are several AI-powered tools on the market, whether in a commercially-viable or prototype form. Perhaps most significant is ChatGPT by OpenAI, which was made public at the tail end of last year. What’s ChatGPT? Why don’t we ask it:

Me: what are you? ChatGPT: I am an AI language model developed by OpenAI, known as ChatGPT.

To describe itself as a “language model” may be correct but it’s a little modest. ChatGPT’s abilities are vast. It can answer questions, generate code, write articles, translate documents, tell you a joke, or engage with you in some good old-fashioned chit chat.

This of course is an impressive piece of technology and a fun tool to experiment with. But its scope and usefulness extend far beyond curiosity. It has practical, real world applications. Why spend time programming a website component when an AI can do it for you in a fraction of the time? Why struggle with writer’s block when a copywriting deadline is looming? Why be presented with adverts when searching for information online? Prompt ChatGPT and it’ll take on the task and output some code, content or answers within minutes if not seconds.

Amazing, right?

Let’s take a step back

The validity of the phrase artificial intelligence is often contested. As a species, we are yet to develop something that is actually aware, or truly understands what it is doing. Perhaps a more accurate description is fancy algorithms. Patterns, machine learning… and sometimes very wonky output.

On that topic – ChatGPT makes mistakes. That’s not a criticism. Just like a human, ChatGPT learns from mistakes… but it doesn’t feel embarrassed about it. When things go wrong, you can tell it so. It’ll then take steps to rectify the error. There’s some give and take when working with ChatGPT.

What artificial intelligence could mean for email

The combination of email and AI (or fancy algorithms) is nothing new. One-to-one product recommendations have been around for years, picking relevant items based on previous shopping behaviour. Spam filters automatically guard inboxes against emails of the shadiest kind. Customer journeys and automated emails are made possible through complex workflows with little post-development need for human intervention.

What is new however is the concept of emails that are predominately or even completely designed and coded by computers. Imagine high-quality, on-brand design and copy that is generated in seconds. Perhaps the same AI tool could then select the audience, send the email, read the report and optimise the next send. Is there a point at which human input becomes zero?

What artificial intelligence currently means for email

Don’t worry – we’re not at the human irrelevance stage yet. In my tests with ChatGPT, I’ve seen it output some erroneous facts and broken code. From what I’ve seen it’s neither ready to fly solo nor likely to achieve such independence any time soon.

That doesn’t mean it’s not a revolutionary and practical technology in its current state. I already use it almost daily for writing inspiration and to help with coding questions that would otherwise mean trawling through forums for answers.

ChatGPT and other pioneering AI technologies have already changed the way we work, and they will only continue to evolve. It’s incredible to think that what was only recently in the realm of science fiction is now becoming a reality. The future really is here.