Streaming suicide and other design decisions

A 12-year-old girl live-streamed her suicide. It took two weeks for Facebook to take the video down.”

So reads the headline of a January 15 news story in The Washington Post. Saying that she’d been sexually abused by a family member, 12-year-old Katelyn Nicole Davis, of Polk County, Georgia, hanged herself, broadcasting the suicide via a 40-minute live stream seen worldwide. While YouTube immediately removed the video, it “lingered on Facebook for nearly two weeks,” according to the Post’s reporting of Buzzfeed and other unspecified “media reports.”

First, it just hurts

I keep rereading the short article and its headline as if it will make sense or stop hurting on the next scan-through. As a human being, I can’t fully process the horror and sadness of this tragedy. I was initially going to write “as a human being and the father of a 12-year-old girl,” but that last part shouldn’t matter. You don’t need to be the parent of a child Katelyn’s age and sex to feel the feelings. Nor does there need to be someone in your life who was raped or molested—although, whether they’ve told you about it or not, there almost surely is. Statistically there are likely multiple someones in your life who have suffered unspeakably, too frequently at the hands of people whose main job in life was to protect them. I’m sorry to have to write these words, and I hope reading them doesn’t rip open wounds. But the point is, even if you and everyone in your circle has lived a magical life untouched by too-common crimes and horrors, it is still unbearable to contemplate too closely what Katelyn must have felt, and what she did about it, and what watching what she did must have done to those who watched the video—both the empathetic majority, and the hopefully small minority of viewers who, because of their own damage, may have gotten off on it, edging just that much closer to some future sociopathic acting-out.

A designer’s job

On a personal level, I’m good citing horror and sadness as a reaction to this ugly story. But as a web and product designer, I can’t help but see it as another instance of what Eric Meyer and Sara Wachter-Boettcher warn about in their book, Design For Real Life. Namely: For every wonderfully fresh use of the internet’s social power we conceive, we must always ask ourselves, how might this be used to make our world more hurtful, less loving, less kind? What unforeseen dangers might our well intentioned innovation unleash? I published Design For Real Life, but I don’t cite Katelyn’s story or repeat Sara and Eric’s moral here to sell copies. I do it to remind us all that what we make matters. Our design decisions matter. The little qualms that might float through our minds while working on a project need to be examined, not suppressed in the interest of continued employment. And the diversity of our workforce matters, because it takes many different minds to foresee potential abuses of our products.

Streaming suicide, monitoring content

Live.me isn’t the first live streaming app, and, as a category, live streaming likely does more good than harm. The existence of a live streaming app didn’t drive a girl to kill herself, although, in despair at not being listened to, she might have found solace and an appeal in the idea that her suicide, witnessed globally, could lead to an investigation and eventual justice. Similarly, when Facebook began allowing its customers to perform live streaming (or, in Nicole’s case) to post video streams from third parties, use cases like pre-teen suicide or the torture of a mentally disabled teenager most likely did not factor into those business decisions. But here we are. And, as much content as Facebook produces in a day, you can’t really fault them for not always being johnny-on-the-spot when some of that content violates their guidelines. But surely they can do better. 

Invention is a mother

There’s no closing Pandora’s box, nor would we wish to. But we who create websites and applications must remain mindful, honest, and vigilant. We must strive to work in diverse teams that are better than homogenous groups at glimpsing and preparing for the unforeseen. More than ever, we must develop design practices that anticipate the horrible and tragic—not to mention the illegal and authoritarian. And in life, as well as design, we must do a better job of asking ourselves what we are not seeing—what struggles the people we meet may be hiding from us, and how we can help them before it is too late.
Also published on Medium The post Streaming suicide and other design decisions appeared first on Zeldman on Web & Interaction Design.