Originally trained to become an actor, Michael took a circuitous route before he ended up in the world of email. He designed high-fashion women's footwear for a while and got into the digital world when he started a horse racing website:
"I eventually turned it into a SaaS platform and with that, I had to learn how to build a website, build an email list, and send emails about horse racing to people's inboxes. Everything around copywriting, deliverability, and all those other pieces became a part of my journey to email marketing automation."
These days, Michael works for DEPT, a full-service digital agency that provides its services to companies in various industries. His work at DEPT Ireland is currently focused on the country's Health Service Executive.
Why doing QA for your emails is important
When it comes to the importance of quality assurance, Michael says it depends on your organization.
"If it's your own business and you're just starting out, the QA level required is probably minimal. And then the further down the line you get, the larger the organization, and the more specific content needs to be."
It also depends on the industry you're in:
"For example, when you're in healthcare, the emails have to be right. You can't be sending those sorts of things out with incorrect information."
The bare minimum: read your emails aloud
For solopreneurs and startups, QA could be in the form of reading your emails out loud as the bare minimum:
"As a basic QA process, it does two things. First, you will hear mistakes. Second, when you say it out loud, you hear how it sounds inside someone's head and if it's conversational."
QA is still a manual process
While tools like Google Docs and email marketing platforms can do amazing things, Michael says that from a QA perspective, these tools are subpar:
"We can't do version control. Even if the dev platforms show you variants of staging or pre-prod or whatever for marketing automation before production, there's still no easy way of doing workflows and automation between those stages.
You have to manually export, but quite often, you can't be sending from the same IPs or send emails across their versions. There is no a really good way of doing QA."
Despite the lack of good tools, doing QA manually benefits startups and smaller companies because they can do it faster if they want to:
"There are workarounds for almost everything because you have to use workarounds anyway in a lot of situations with automation. You have the benefit of agility.
But of course, things get left behind when you do that and you will make mistakes. But in those situations it's normally fine."
QA in complex industries and big companies
In large organizations and complex industries, the ownership of the QA process depends on the organization:
"It's organization dependent. You might have a specific QA project manager but that's not always the case. Sometimes it's the project manager for the entire campaign or that piece of the communications, depending on how it's broken up within the organization.
If it works like that, then within a team, you would probably have a marketing automation lead and QA lead who are working side by side depending on where you are in the process like workflows, automations, creatives within emails, etc."
But with so many elements involved in emails — from the automation, the email itself, and the design template — what does the QA process look like?
Doing QA on the template design
Step 1. Creating the template design
The first QA piece is the template design:
"If you're working with an agency, the QA is going to be done by the client and the designers. If you're gonna be doing this internally for yourself, then obviously that'll be slightly different, but it's the same principle."
When it comes to choosing the designer for the task, it's important that they have firsthand experience in designing emails or have at least worked with someone with that experience:
"It's important that they understand the nuances of email design specifically. Otherwise, when you go past that QA process of design, you can't actually implement it in a way that's going to work properly and you'll need to redesign."
After the template design gets signed off, it becomes the QA team's source of truth.
As a personal preference, Michael likes to create a base template that contains all possible modules:
"When I say modules, these are the elements like blocks you can use in the emails. I like to have them containing all of those as pieces so that the base template can be signed off as an entire piece across all devices.
By doing a base template, you take into account the mobile responsiveness and you can reuse them for future emails:
"The mobile responsiveness of all the elements is done because we can break those elements and save them as blocks. We can reuse them in the future because we know all of these pieces are now functional."
Step 2. Testing the base template
To test the base template, DEPT uses tools like Litmus and Email on Acid:
"We would test that internally as an agency. We check the designs in every email client, both the desktop and mobile version. We might test it out in every single email client or just the important ones. But selecting email clients depends on who you're doing it for."
After the template is signed off on the agency side, it's then handed off to the client, and the process is repeated.
Michael says that by having multiple levels of sign-offs, you're actually reducing the amount of work in the long term:
"The purpose of these multiple sign-offs is that at the end of the process, you want to try and reduce the amount of work to be done on each individual email because there are things that will likely break or not quite work properly.
By putting in that time on the base template at that level, you take away 90% of the issues that you may face if you would have done it further down the line. Because if you do this further down the line, if something changes, you also have to change the fifty or a hundred emails that you've already finished."
What to consider when testing your workflows
Do it on a sandbox with staging
Depending on the size of your organization, you can test your automation by building a staging environment for the platform:
"It reduces the risk that you drop a bunch of people into it by mistake. Or if you apply a tag, or something else on a workflow that you didn't realize or forgot was live, causing a whole bunch of people to shift in there automatically. So my preference is, if you can, to build on a sandbox with staging."
It depends on the platform you're using
Since QA is a very manual process, it depends on whether your platform allows people through workflows or not:
"For example, if you have a workflow that goes for six months. Some platforms may be able to allow you to see when people are being held and then force them onto the next stage automatically.
Other platforms may not allow you to do that, so you have to create custom fields or custom tables. Sometimes, I have test tables where I put custom fields in so that I can change settings and force people through the workflow."
What to keep in mind when building the workflow
The key to doing QA for your workflows is to try to build your automation in such a way that you can also test them and have them work in production:
"What I will do when I'm building the automation is I'll be thinking, 'how am I going to be able to force QA testing accounts through this without needing to change the automation once we've done it, so we know it's working?'
So what I do is I might build extra coding that says, 'Okay, you flow through normally.' I can see it's working and now you come to the next bit. The simplest example would be to apply a tag that forces you to the next step."
The workflow will become more complex as you continue to build so it's important to plan it out from the beginning:
"The goal is to always make sure that I or the team plans from the beginning so we can test it without having to edit workflows after the QA process and potentially reintroduce bugs."
Two stages of doing QA on your workflows
Stage 1: Forcing people through the workflows
"I want to be able to force you through the workflow. In this way, I can see the big pieces are moving and everything is working."
This could be in the form of applying tags, conditions, doing extra coding, and more.
Stage 2: Using data feeds and manual interventions
Because tags and extra coding are essentially the introduction of artificial data, how do you make sure that the workflow would function when you deploy it live?
For example, you need an event to trigger an email campaign. Depending on the event and how it is fired, you could go into a table and edit a custom field to tweak the data (as a last resort).
You can also use data feeds to test your workflows:
"That feed can be happening in real time as it would. It could be fired by developers or by myself or whoever, depending on the team, which actually allows you to test in real time."
To test timeframes, there is no other way but to manually change the date or use tags. Michael shares that you can also use Postman for this purpose.
"You can use Postman to push an event call in and the event call can be real. You can hold it in Postman, and then just change the time frame within Postman so it's still coming into the platform as it would, but you've just changed it before hitting the platform.
So it would be real, but at some point you have to do some manual intervention."
As a shortcut, you can also tweak the time delays in your workflow but make sure to write down the original so you can change it back:
"Shorten the delays down to 1 minute, 5 minutes, 10 minutes, or whatever works for you.
If I do that, what I tend to do is I actually tend to write it on paper. I write each one down that I've changed and then I cross them out as I change them back. Because again, if you have lots of nodes or whatever on your screen, it's really easy just to skip over one by mistake."
Final advice
Do some form of QA for your emails.
"Even if it's just speaking the copy out loud, checking the email in the preview, or sending yourself the test email. Always do some for of QA"
Don't get stressed with QA.
"A lot of people get worked up by QA, especially when you've got an extended process that takes some time. The email iteratively goes through each stage at a time. And if it's an extended process, make sure the team around you understands that to actually have this level of confidence or security within the creatives and the content.
Because it's done manually, going through numerous people will take time and people will make mistakes. If it doesn't go through numerous people, there's likely to be an error in there."
Thanks for listening! If you found the episode useful, please spread the word on Twitter mentioning @userlist, or leave us a review on iTunes.
