What happens when tech firms end up at the center of racism scandals?

Nextdoors push to combat racial profiling is latest controversy when tech firms find themselves in scandals at intersection of race, social justice and free speech

The racist posts on Nextdoor.com became so frequent that they started to make Shikira Porter feel physically ill.

Billed as a private social network for your neighborhood, Nextdoor allows users to write public messages to neighbors who have joined the site, which now has more than 110,000 local groups across the US. After Porter, who is black, signed up in her Oakland, California, neighborhood in 2013, she quickly discovered that many of her white neighbors were posting crime and safety alerts about suspicious people of color.

A black man in a hoodie. A Latino man hiding near the bus stop. A black salesman who might be a burglar.

Its incredibly toxic, said Porter, 42, who said she has had migraines after arguing with racist users. I just feel like Im in a constant state of rage that I have to manage.

Porter channeled her anger into Neighbors for Racial Justice, an advocacy group that has for years pushed Nextdoor to combat racial profiling.

Following intense scrutiny and negative press, the San Francisco-based startup this month introduced a number of product changes, including an algorithm that directs users to offer more detailed descriptions of suspicious activity, beyond, for example, black man in hoodie.

The Nextdoor conflict is the latest controversy in what has become something of a predictable pattern when Silicon Valley tech companies find themselves at the center of scandals at the intersection of race, social justice and free speech. While some say Nextdoor has taken a bold and proactive step with its new features, the dispute sheds light on increasingly tense debates surrounding the ethical obligations of social media and sharing economy companies that are reinventing communication and reshaping industries.

To critics of corporations such as Facebook, Twitter and Airbnb, the cycle of tech scandals has become frustratingly familiar: driven by profits and public relations, the firms refuse to make substantive changes, often at the expense of their most vulnerable users.

A fundamental shift

Nextdoor claims its anti-profiling measures are radical changes that will dramatically reduce racism on the site. The concept behind the tweaks, however, is fairly simple and rooted in ideas that California activists suggested last fall.

Under the new system, before members can post a crime and safety message, the site warns them about profiling, saying: Ask yourself is what I saw actually suspicious, especially if I take race or ethnicity out of the equation?

Next, users are encouraged to focus on behaviors and specific descriptions, such as clothing, shoes and tattoos. If they want to mention race, they have to fill out a form and include at least two other physical descriptors before posting.

Its unique for a tech company to fundamentally change its product, Nextdoor CEO Nirav Tolia said in an interview.

Racial
Racial profiling product improvements on Nextdoor. Photograph: Courtesy of Nextdoor

After testing the tool, an internal study found that the algorithm reduced incidents of racial profiling by 75%, according to Tolia. Perhaps more significantly, many users are choosing not to post all, with a 50% increase in people deciding not to publish their crime and safety messages.

That drop in activity is a completely worthy cost relative to the benefit, Tolia said.

Activists say the changes are long overdue and dont go far enough. Porter, for example, shared a recent post that simply described a HMA suspect, referring to Hispanic male adult.

But some tech critics said Nextdoor deserves credit for taking a risky step that may reduce overall posts and engagement.

Theyre creating friction. Theyre making it a little harder to use their platform, said Jamila Jefferson-Jones, associate professor of law at the University Missouri, Kansas City, who has written about discrimination and technology. But in the end, theyre getting better information.

On the contrary, tech companies typically prioritize user-friendliness and convenience above all else even if that means shielding racist users and allowing offensive posts.

Racial
Racial profiling product improvements on Nextdoor. Photograph: Courtesy of Nextdoor


We know how to fix the problem

Airbnb, the popular home-sharing startup, has increasingly battled negative headlines about discrimination on its site. Independent research has found that guests with distinctively African American names are 16% less likely to be accepted than identical guests with white-sounding names.

The San Francisco tech company announced this summer that it has hired former US attorney general Eric Holder to help address racism on the site.

The scandal, amplified by the hashtag #AirbnbWhileBlack, has prompted an aggressive PR response, with CEO Brian Chesky claiming that the issue constitutes the greatest challenge we face as a company.

Researchers, however, say there are relatively simple fixes Airbnb could make but that the firm may be resistant to adopt out of fear that it would reduce usage.

They put out a lot of press releases relative to the amount of work they do and relative to the amount of changes they make, said Ben Edelman, associate professor at Harvard Business School, who has studied Airbnb and discrimination.

If Airbnb stopped having users display their names and faces and instead had them use anonymous screen names, it would go a long way toward reducing discrimination, Edelman said. We know exactly how to fix the problem.

Airbnb
Airbnb has increasingly battled negative headlines about discrimination on its site. Photograph: Martin Bureau/AFP/Getty Images

In blind auditions where musicians cant be seen, for example, orchestras hire more women and people of color, he noted.

Jefferson-Jones said Airbnb could also audit users and ban hosts who have faced repeated discrimination complaints or have a history of rejecting black users.

Fewer users, however, means reduced profit.

Nancy Leong, a University of Denver law professor, said that companies such as Airbnb are also concerned about legal liability, which makes them less likely to acknowledge major problems in the first place. Lawyers are trying to advise clients to say as little as possible and to issue statements that are fairly generic.

An Airbnb spokesman declined to comment on the suggestion of removing faces and names from the system, saying in a statement: We are committed to fighting discrimination and earlier this summer, we launched a comprehensive review of every part of our platform.

Why diversity matters

In the wake of a high-profile crisis, tech companies often focus on PR and liability as opposed to the experience of everyday users. As a result, regular people suffer, advocates said.

Facebook and Twitter have both experienced major scandals regarding harassment, bullying and abuse on their sites. Twitter in particular has been dogged by complaints that women and people of color on the site, especially outspoken activists and celebrities, are forced to reckon with violent hate speech and persistent threats.

After a wave of critical news stories, Twitter recently banned a rightwing writer who launched a coordinated attack against black comedian Leslie Jones, but many pointed out that the company has systematically failed non-celebrity women on the site facing similar abuse.

Critics have further noted that Twitter has been successful and eager to protect brands, aggressively removing posts that constitute copyright infringement.

Part of the problem is that some of the social media scandals delve into uncharted territory where there arent clear or easy solutions. After Facebook faced accusations that it was biased towards promoting liberal news sources, the company fired members of its so-called trending team and left an algorithm to highlight popular news stories.

The result was Facebook promoting a false story about Fox News host Megyn Kelly and links to an article about a man masturbating with a McDonalds sandwich.

While the trending topics PR debacle perhaps signals a lack of foresight in response to a scandal in the media, others argued that the poor responses to controversy stem from systemic problems.

Malkia
Malkia Cyril: The investors are wealthy. They are white. And they are mostly men. Photograph: Center for Media Justice

In particular, when tech companies lack diversity, especially in leadership positions, they can be unprepared to respond to serious and complex challenges that affect marginalized groups.

At Facebook, black employees account for only 3% of senior leadership in the US.

Activists said that disparity was hard to ignore in the context of Facebooks most recent controversy regarding racial inequities. Earlier this month, social justice advocates slammed Facebook after the company, reportedly at the request of police, shut down the live-stream of Korryn Gaines, a Baltimore woman in a standoff with officers. The police eventually killed her.

I really hope that Facebook goes through a serious re-evaluation process of where their values lie, said Taina Vargas-Edmond, state campaigner with the Ella Baker Center for Human Rights, which co-signed a letter to Facebook condemning censorship of Gaines.

Facebook declined to comment on Gainess account.

Malkia Cyril, executive director of the Center for Media Justice, noted that one possible reason why Nextdoor has been more aggressive in responding to concerns of racial justice is that Tolia, its CEO, is a person of color. The CEO has some empathetic understanding of what is happening here.

Cyril, who is active in Black Lives Matter and has spoken out about online harassment and diversity in tech, noted that many tech leaders prioritize the concerns of investors over users even in the face of scandal.

The investors are wealthy. They are white. And they are mostly men.

Read more: https://www.theguardian.com/technology/2016/aug/30/tech-companies-racial-discrimination-nextdoor-airbnb