New Emails, Old Tech
How HTML helped, then hindered, the evolution of email, or why all those fancy marketing emails you get in your inbox still rely on HTML tables in 2019.
Today in Tedium: What makes an email different from a web page? Depending on how it’s presented, not a lot—but they also might be miles apart. Things that might have taken a few minutes to lay out for a website can take significantly longer to do when targeting an email client, and with a lot of pain in the middle. With that in mind, I felt like it was good to talk a little bit about the process that goes into email, and where it’s really falling short. Today’s Tedium is an email … about email. Particularly where it needs a little modernizing. — Ernie @ Tedium
“P.S. I love you. Get your free email at HoTMaiL.”
— The marketing message that initially appeared on the bottom of messages sent through Hotmail, the pioneering webmail service later acquired by Microsoft. As journalist Adam L. Penenberg noted in his book Viral Loop, Hotmail creators Sabeer Bhatia and Jack Smith were initially unconvinced of the strategy when it was pitched to them, but after some modest changes (the “P.S. I love you” part was quickly ditched) it eventually helped Hotmail become a massive application with only a limited budget. All it needed was a link at the bottom of the message.
Netscape Communicator’s email client, the basis for the modern Thunderbird email client.
How HTML became the primary engine driving email
HTML and email weren’t destined to be together, but the browser wars might have ensured they wouldn’t spend much time apart.
Certainly, multimedia email wasn’t unheard of by the time HTML came into the picture in the mid-to-late 1990s. The MIME standard, which much HTML-based email is reliant on, dated to at least the early 1990s, and programs like Lotus Notes had multimedia elements well before consumer products did.
But the battle between Netscape and Microsoft over control of how people viewed webpages on the internet led to a lot of aggressive changes in how basic internet tools worked—and these changes came along, fast.
Microsoft was acquiring a lot of companies with internet footprints—like WebTV, FrontPage, and Hotmail—and responding to something it arrived not-so-fashionably late to.
Netscape, meanwhile, found itself trying to sell shrink-wrapped products for something that people largely downloaded and used for free. And to entice more people to pay (or, at least, pay more), it created versions of Navigator with extra features (first called Netscape Navigator Gold and later Netscape Communicator), with the killer feature being a lightweight WYSIWYG HTML editor.
Netscape, of course, decided it was probably a good idea to bring its HTML editor to email. And so, the HTML email was born.
Upon its release, Outlook Express was more capable of supporting HTML emails than its bigger brother, Outlook, was.
Microsoft, of course, quickly followed suit with its own take on this strategy. Oddly, Microsoft’s strategy for doing this was a little confusing at first: It decided to put HTML email support in Outlook Express, a free client distributed with Internet Explorer 4, but not in regular Outlook.
This was a problem for email administrators, because, despite the similar names, they were different programs based on different code bases—and if you sent an email in Outlook Express and read it in Outlook, it would show the raw HTML.
“It’s ugly, real ugly,” a Texas Instruments employee told Network World in 1997. “This is going to be a real support issue.”
(The HTML-supporting Outlook 98, when it was released, was initially distributed for free online, unique for an Office product at the time.)
More traditional competitors were also ready to give HTML a spin, too. Lotus Notes, a workplace communication platform built around email (and a key IBM acquisition), added a HTTP server called Domino and the ability to render HTML email in 1996. By late 1997, the cult favorite email client Eudora was also adding support for HTML mail.
And that’s to say nothing of the rise of webmail, which dates as far back as 1993 but really had its coming-out party with Hotmail (later Outlook.com) and RocketMail (later Yahoo! Mail) in 1996. (Gmail gave things a further kick when it was first launched in 2004—fashionably late, but with an approach that quickly won adherents.)
HTML mail, almost immediately, had its problems, with Outlook 98 in particular (and to a lesser extent Outlook Express and Netscape Communicator) falling victim to security flaws that were exposed by the software’s scripting capabilities and use of a malleable rich-text standard.
And connection speeds in 1998 generally weren’t what they are now, meaning that HTML messages taxed the server a whole lot more. (Hey, it could be worse—people could be sending executables to one another. Oh wait.)
But the fact that you could now, effectively, send webpages to inboxes at will helped to jumpstart the concept of email marketing, for better or for worse.
“So you spend all of that time creating a perfect layout for your message, hit Send, and assume that the recipient will see it in the same way. Bad assumption. There are millions of people using mail clients that are not HTML-compatible. No, you can’t just say ‘Well, they should upgrade then.’ It’s not as simple as that.”
— Scot Hacker, a programmer and former technology journalist, making the case against using HTML emails around 1998 or so. Hacker noted that a big part of the problem with HTML emails is that they did not account for clients that could not support HTML formatting, and that the use of HTML in email affected user preference. His comments drew a lot of response back in the day, but he notes that his views have largely changed over time, as clients have become more capable of supporting it. (There are other folks out there who likely feel this way for non-technical reasons, it should be said.)
Why HTML email eventually made everything more complicated
The way that the average person uses email-based software differs greatly from how a marketer or a publisher might use it.
Often, the average person isn’t messing with fonts, or layout, or background of an email. They might upload a picture, but 10 to one, they aren’t getting wild with the color scheme.
But with its roots in HTML, email would conceivably, with the right software, allow for more ambitious designs, right? Right?
Certainly, email marketing software—think MailChimp or Campaign Monitor or a dozen other similar tools I could mention in this space—is built for allowing this. But there are a few kinks with email that make HTML an imperfect vessel for sending marketing messages in 2019:
Emails have to be built for the lowest common denominator. In many ways, HTML-based email has generally not evolved from what could be done with it in 1998—which means that old relics of web design, like the use of tables and the inlining of cascading style sheets, are common in this medium. If your readers are looking at a message through an old version of Lotus Notes, you have to account for that in the messages you design. As clients such as Gmail have become more popular, this issue has faded to a degree, but it’s still prevalent.
Everyone renders things a little differently. Remember what it was like to code a website in about 2006 or so, when Internet Explorer 6 was still so widely used? The improved standards of browsers such as Safari, Chrome, and Firefox helped mitigate these problems on the open web—but when it comes to email, it’s a different story. The best way I can describe it: Imagine you’re coding for five separate browsers as finicky as Internet Explorer 6, if not more so. And you’re doing so in 2019.
HTML-embedded images inherently introduce privacy concerns. One of the problems with HTML email is that it allows for the use of external images, which allows for a form of tracking, just like on web pages. But since emails are often unsolicited, this creates natural privacy issues. This problem has dissipated to a degree thanks to moves by Google in particular to pre-load images in its email clients, thereby reducing the problem, but not eliminating it. If you’ve ever wondered why your email client asks if you want to turn on images, this is why.
The minified source code from a recent issue of Tedium.
The reliance on tables and inlined CSS makes messages far larger than they need to be. An issue I would frequently run into with Tedium early on was that messages would “jump,” or require users to click a button, to read the rest of the message. I eventually got around that using some coding tricks like minification, but it’s a lasting frustration for people who send longer emails. If emails were built with modern web standards, messages would be significantly smaller and would waste less code and bandwidth all around.
Email clients have other masters to consider. A little over a decade ago, Microsoft made a decision that made a lot of sense for the users that buy its software, but little sense for those that distribute emails that reach those users. Basically, the company moved away from using Internet Explorer’s rendering engine in Outlook to using Microsoft Word’s. This allowed for easy copying and pasting of stuff from other Office programs. But it came at the cost of a lot of rendering capabilities—to the point where an email that looks fine in Outlook 2003 can be totally janked up in the latest version of Outlook for Windows. (It broke a whole lot of emails!) Only within the last few years has Microsoft started to show interest in addressing the concerns of email marketers.
HTML email complicates webmail, too. You may think that the fact that people largely read messages in modern browser windows these days largely mitigates the issues I listed above, right? Well, not really. Basically, webmail clients have to heavily filter the email that gets sent through them, and often only support a subset of the HTML and CSS that the full browser does. (You don’t want an iframe or javascript in an email, for example.) The result is often not 100 percent the sender’s intention. Gmail has improved on this front in recent years, but competing clients like Outlook.com and Yahoo Mail have unique quirks that often add production time for those that build emails.
As someone who sends a lot of emails that use a somewhat fancy design, really the only email client that does a good job with this stuff across the board, without much in the way of headaches, is Apple Mail—basically because it’s rendering everything using the same web engine Safari uses.
Granted, lots of folks may simply hold the viewpoint that email isn’t the right place for HTML. That’s totally fair, and a point I understand—though it’s an argument that probably would have been better won two decades ago. Fact is, it’s 2019, and we’re stuck with HTML email that’s based on outdated technology, and because it’s outdated, it can’t take advantage of many of the efficiencies baked into modern web clients.
This is, in part, due to a situation where email has found wide use in a form that the makers of popular email programs appear not to have intended. This causes a lot of headaches. But people who send lots of emails are willing to deal with this because it’s an effective way of reaching a desired audience. Fixing busted tables with Litmus is still easier than sending 30,000 physical letters.
The question is, can it be made better?
Tom Hanks could be describing email’s relationship with HTML in this message. (via YouTube)
If email needs a reboot, who should do the rebooting? (Or, the debate around AMP for Email)
This post, I must admit, was inspired by a firm named Tutanota that produces a secure email client, one that is basically the opposite of everything Google has created with Gmail.
It called out a protocol called AMP for Email, which was announced last year with the goal of adding more interactivity to email. Google, which is still working out the kinks, based the work on AMP for Email off its existing Accelerated Mobile Pages effort.
AMP is controversial, partly because it was built by the already dominant Google, and partly because it’s seen as an unnecessary abstraction that isn’t necessary to speed up web pages. It’s frequently a topic of frustrated blog posts and petitions from developers who find Google’s use of AMP as a stick for the search-engine carrot distasteful.
Personally, I’m ambivalent about it: As a guy who follows internet issues closely, I understand the ethical problems, but as a publisher, I get the reality that AMP helps me better reach readers, and as control plays go, it’s less distasteful than most of what Facebook does. At least it’s still on the open web, right?
But the post from Tutanota, which drew a lot of attention earlier this week on Hacker News, did a disservice to the discussion about AMP for Email because it basically conflated most of the issues with the web version of AMP with the email version of AMP, which won’t work exactly the same way.
Here’s what AMP for Email is supposed to represent, in layman’s terms: I send a lot of emails, and I send those to users who generally want to receive them. I have, at times, spent weeks debugging issues that have prevented people from getting those emails. It’s to my benefit, and theirs, that the number of technical flaws in email messages be limited. AMP for Email creates a layer that both adds interactivity and modernizes the email format, while helping to ease technical problems, and without forgetting about people with old email clients.
It arguably gives Google more control, which is worth calling out. But fewer technical problems with the actual sending of emails should be something that a company that distributes emails, like Tutanota, wants as well, even if they don’t want it in AMP form.
But because it’s easier to attack an already unpopular Google initiative than it is to admit that there’s an ecosystem-based problem with email that needs solving, Tutanota missed an opportunity to move the conversation forward, instead creating a marketing opportunity for itself. That’s a shame.
We don’t need AMP for Email, to be clear, and others have made better arguments against it. Google should have called it something else, and it should have called in more stakeholders before announcing it to the public. Had Google and Microsoft announced that they would work together on improving email rendering in old clients, along the lines of Google Chrome Frame, that approach would have been welcomed. But by tying it to a program already treated as a point of concern for the developer community, it creates fear, uncertainty, and doubt.
But credit where credit is due. What folks who send email need is a way to lift up the overall technical quality of email marketing without leaving the least common denominator behind. By creating AMP for Email, Google was trying to do that.
For all intents and purposes, a giant, multibillion-dollar industry (marketing) is using 1999 technology to send emails in 2019, due to the lack of industry consensus on what an email should look like. We ran into this exact same problem before with the open web, and the tech industry solved it. Why can’t we do it again?
There has to be a way to modernize email to remove the technical flaws while still keeping the pro-privacy folks happy.
A few months ago, something really awful happened to my email account, and I have no recourse other than to complain and tell you about it.
Basically, someone decided to send a particularly sketchy piece of advertising to probably thousands of people by “spoofing” my email address. It was not from me—I checked—but I got the blowback.
I can’t tell you how many, but I can tell you that it was probably a lot based on the number of bounce-backs I got, of email addresses that no longer exist, that were on a Time Warner Cable domain that the cable company stopped distributing more than a decade ago. So this message was being distributed to a whole bunch of dead accounts, or to people who have probably been suffering from substandard email for decades.
I run a business that sends email. If you’re reading this in an email client, you’re accepting my rendered services. Obviously, I got really worried after this happened. Fortunately, even though it looked bad, there was no way this could hurt my sender reputation, because I use secure mechanisms such as DMARC and DKIM to protect and confirm the messages I send, and the recipients appeared to be confined to a single domain. The next issue of the newsletter sent without a hitch.
Still, though: For an hour, my inbox was basically helpless. I started getting angry replies from people who thought I had sent them something awful. I sent a reply to one, swearing it wasn’t me who sent it, but that I had been spoofed. I never heard back from the guy—which I get.
In the end, this was a momentary hiccup that didn’t affect anything, and my readers largely didn’t know about it unless they saw me complaining on Twitter. But it highlights the problems that come with email as a medium that has a history that long predates the rest of the internet. It’s old. It’s creaky. And nobody “owns” it, whether in a de facto way like Google owning the browser market or in an official way like Apple owning the App Store.
It gives it clear benefits—I can reach you without the need of an additional social network or an outside platform if I so choose—but it means we’re at the mercy of a fragmented infrastructure that only moves forward when those fragments decide to do something.
“Moving forward,” at least for now, looks something like AMP for Email. Even if it’s not the ultimate solution—and it might not be—we need something like it.
Because email isn’t going anywhere, and neither is its evolution.
--
Find this one an interesting read? Share it with a pal!
Is this discussion of email technology on your level? Want a career that leverages those technical skills? Try out this quiz from Triplebyte to see where your tech expertise lies. It might be the first step in finding your dream gig in the tech industry.