Mistakes Were Made

Technology vendors like SAP may rake in billions of dollars a year helping big companies build complex tech infrastructures, but they screw up—often, at scale.

By Andrew Egan

Hey all, Ernie here with a fresh piece from Andrew Egan, who last hit us with a couple of holiday-themed pieces. This time around, he ponders how enterprise technology vendors screw up in totally human ways.

Today in Tedium: The Baader-Meinhof phenomenon pops up in my life from time to time. Named after a German terrorist group who gained notoriety in the early 1990s, Baader-Meinhof is the idea that something you just recently learned about suddenly comes up everywhere. The first was when a coworker told me they were attempting to find a deck of playing cards made entirely of discarded ones. How often do you find discarded playing cards? Everywhere it turns out. The latest has come thanks to old-school panic, some random Tedium article, and the fear of Y2K. The New York Times had a slight contribution when they reported on the failure of parking meters to accept credit cards. This stupid little chain of events got me (or more importantly Ernie) thinking about software vendors. Today’s Tedium is talking about the mistakes software vendors make and how one company manages to dominate an industry while making high profile mistakes. — Andrew @ Tedium

⬇ Oh, and be sure to give our sponsor a look! ⬇

Pessimists Archive

Why did people fear coffee, novels, and teddy bears? In each episode of the podcast Pessimists Archive, we look at the moment that something new came along—something that’s commonplace now!—and try to understand why it freaked everyone out. Our goal: By understanding why people fear change, we can become better at embracing it.

Today’s Tedium is sponsored by Pessimists Archive. (See yourself here?)

78%

The percentage of the world’s food that is distributed using some form of SAP logistics software. Systems, Applications, and Products (SAP), was founded in 1971 and has become a global leader in entrepreneurial logistics. How exactly that happened is pretty funny to anyone that watched Pirates of Silicon Valley.

0114 xerox

An Xerox Copy-Flo machine, in operation during the late 1960s. (National Library of Medicine/Public Domain)

Yet another story about Xerox and IBM in the early days of computing

Before we take a deep dive into yet another story about how Xerox and IBM got screwed over in the early days of computing, it might help to discuss ERP (enterprise resource planning) software. I’ll make this quick.

Do you need to manage inventory between your warehouses and associated retail stores? That’s basically what ERP software does. It turns out that both IBM and Xerox made early investments in the field but bailed a little too quickly. Xerox’s infamous decision to abandon the hardware computing industry in 1971 had more consequences than just giving Steve Jobs an edge at Apple. It also gave IBM the confidence to spin off a division it thought wasn’t necessary.

Part of the Xerox spinoff was migrating their systems over to IBM. This included Xerox selling their ERP software to IBM for $80,000 credit on the contract. Basically, Xerox got a discount to give them their software for a new field. And this left a group of five German IBM engineers wondering what would come next for their project. In mildly ironic fashion, neither company ended up with the prize.

The engineers went their own route.

$28B

The 2018 annual revenue of SAP, a German-based software company that focuses on the important but rarely appreciated business of enterprise resource planning (ERP). Though ostensibly the method by which businesses get their products to market, ERP involves a lot of coordination, technology, and expertise to implement correctly. Helping companies perform this vital service can be incredibly lucrative to specialized companies. One firm based in Germany has long been an industry leader in this B2B field.

0114 sap

An early SAP headquarters building. (via the SAP website)

How IBM unwittingly created the basis for one of the world’s largest technology services companies

Rather than end up in a random department at IBM, the German engineers decided to create their own company. Starting with a German industrial chemical conglomerate as a pilot customer in 1972, the engineers created their own company, SAP, getting a product to market just nine months later.

(An American example of this kind of turnabout is the tale of Fairchild Semiconductor, whose employees left it to found two companies whose products you use daily—Intel and AMD.)

0114 sap2

A group of SAP’s early founders.

Jump ahead to today and SAP is basically on top of the world. Creating software to help business do business turned out to be pretty good business. By 2014, SAP had nearly $19 billion in annual revenue. Their product suite of ERP, CRM, and other acronym based software made the company one of the most influential in the world. While Microsoft and Amazon toy with valuations nearing a literal trillion dollars, SAP has been influential in the daily execution of the modern economy.

I’m not kidding. SAP is currently responsible for:

  • 77 percent of some portion of global payment transactions
  • 82 percent of global distribution of medical devices
  • 100,000 direct employees in more than 180 countries

With all this success, there are bound to be a few failures. And when you’re operating on the level of SAP, failure comes often—and often with a high profile.

“No one seems to believe it’s Hershey that’s having the problem.”

— Bruce Steinke, a candy buyer for the Alpena, Michigan-based distributor Great North Foods, talking to The Wall Street Journal in 1999 about a shortage of marquee Hershey’s candies. Around 100 of the supplier’s 700 stores at the time failed to get their full Halloween-season shipments of Hershey’s candy that September.

0114 kisses

(Bev Sykes/Flickr)

The Halloween without Hershey, and other notable vendor mistakes

Take the time to learn about ERP software, and it’s easy to realize small errors compound quickly. It might seem like we’re going to be dunking on SAP here, but as we previously noted during our recent dive into updates to NFL quarterback statistics, when you’re really, really good at something difficult, you’re allowed more errors than others. By any measure, SAP is a titan of logistics and widespread enough as to be vital to the world economy. So when they fail, they fail in ways that have some spectacular consequences.

Case in point: the Halloween without various Hershey’s candies.

This is admitted hyperbole, as there were still Hershey’s products on the shelves, but during an upgrade to SAP’s R/3 RFP platform in 1999, this proposition was fraught with peril.

A case study found that a lot of problems were based on the company’s advanced timeline, which shaved 18 months, from 48 months to 30, to account for Y2K. “This go-live scheduling coincided with Hershey’s busiest periods—the time during which it would receive the bulk of its Halloween and Christmas orders. To meet the aggressive scheduling demands, Hershey’s implementation team had to cut corners on critical systems testing phases,” the investment firm PEMECO found.

Rushing the implementation led to a shortage of Hershey’s Kisses and Jolly Ranchers, totaling approximately $100 million in losses. Rather than make it to retail stores, the product sat in storage.

Not every mishap involving SAP is the conglomerate’s fault, though. Often, companies hire third party contractors to manage and implement SAP legacy systems and software. MillerCoors did this with HCL Technologies, an India-based IT consulting firm, in 2013.

The nearly $64 million contract was seeking to bring seven SAP systems onto the same standard. Like with the Hershey’s incident, timelines became problematic and deadlines were missed.

The end result was a $100 million lawsuit filed by MillerCoors against HCL Technologies in 2017. Seeking a nine-figure result for damages, MillerCoors argued that HCL had “bungled SAP ERP implementation” before deciding to just settle the case. Terms of the lawsuit were not disclosed but did state that “each party will bear its own cost in the action.”

It turns out that ERP implementation is pretty difficult. Some 64 percent of ERP projects exceed budget—so these types of mishaps that cause mass inconvenience seem to happen all the time.

0114 lidl

A Lidl location in Europe. (klara_bella/Flickr)

And just to pick on SAP a little more, there’s the slight kerfuffle with German-based (with a growing US presence) grocery conglomerate, Lidl. When the grocer made the decision to update their ERP system, they said their legacy process “was hampered by process breaks, redundant master data storage, integration gaps and functional restrictions.”

However, when the SAP Hana system they were “upgrading” to took three years to get to operational use, Lidl dumped the project … after spending well more than half a billion dollars. The move was reported not through a lawsuit but a simple memo that explained “the strategic goals as originally defined by the project could not be achieved without the retailer having to spend more than it wanted.”

That’s the thing about ERP projects that tend to be fascinating. It’s poorly understood, inherently contextual, and extremely expensive. When companies screw up ERP implementation, there’s a chance Chicago ends up with too few diapers.

The aftermath of the Hershey’s candy fiasco is especially interesting for understanding ERP and its relevance to the modern world. The Wall Street Journal reported that 7-Eleven was ready to simply replace Hershey’s products with the next best selling item.

“If we ran out of Kit Kat or another Hershey item, we might expand facings of Snickers,” explained John Moser, the candy-category manager for 7-Eleven. Other candy buyers just purchased more product from Hershey’s competitors, like Mars. According to the Journal article, Hershey admitted to losing one-tenth of a percent of their market share over the incident—which may not sound like much, but that’s real money at Hershey’s scale.

It turns out the backend logistics of running a multi-billion dollar conglomerate are pretty complicated. The slightest screw up tends to costs millions initially and potentially much, much more in lost revenue.

These are the issues that create executive nightmares and cause a mass exodus of management. These are the silly things that make modern life possible and shine a little light on what well-heeled capitalists actually do for a living.

Turns out they’re just trying not to screw things up. And now I see SAP terminals and software everywhere.

--

Thanks again to Andrew for another great piece. Find this one an interesting read? Share it with a pal! And thanks to Pessimists Archive—a great podcast if you like the types of things that Tedium covers—for supporting us today. See ya soon!

Andrew Egan

Your time was just wasted by Andrew Egan

Andrew Egan is yet another writer living in New York City. He’s previously written for Forbes Magazine and ABC News. You can find his terrible website at CrimesInProgress.com.

Find me on: Website