Public Fool Systems are Spending More on Social Media Monitoring Technology Even Though It Disproportionately Targets Black & Latino Kids & There is No Proof It Keeps Students Safer

From [Brennan Center] School districts are spending more on social media monitoring technology, but there is little evidence it is keeping students safer.

While statistically rare, school shootings are traumatic for those directly affected, their loved ones, and society at large. A recent spate of campus shootings has led to renewed calls for gun control, often student-led. Legislatures and schools are facing mounting pressure to address the issue, but gun control appears out of reach in the current political climate. In the meantime, school officials and policymakers have turned to a broad range of surveillance technologies in an attempt to prevent attacks on schools.

Social media monitoring is one such form of student surveillance. A number of companies, many of which have sprung up in the last five years, are selling software that can allegedly identify signs of violence or other concerning behavior by trawling children’s social media posts and other online activity.

In an attempt to quantify expenditures on social media monitoring software by school districts, the Brennan Center examined contracts for such software using SmartProcure, a database of government purchase orders. Our review is based on self-reported procurement orders in the database, and thus likely depicts only a portion of school spending on these tools. According to these data, school spending on social media monitoring software has surged in recent years. As the graph below indicates, the database shows 63 school districts across the country purchasing social media monitoring software in 2018, up from just six in 2013 — more than a tenfold increase.

Aside from anecdotes promoted by the companies that sell this software, there is no proof that these surveillance tools work. But there are plenty of risks. In any context, social media is ripe for misinterpretation and misuse. But the possibility of misinterpretation is particularly high for middle school and high school students, who are more likely to use slang and quotes from pop culture, and who may be especially motivated to evade adults’ prying eyes. Difficulties in interpretation mean that social media monitoring of students is likely to lead to false positives. Moreover, monitoring programs are particularly bad at correctly understanding languages other than English and even non-standard English, which may be used by minority students.  And it is well known that school discipline falls more heavily on children of color. These factors suggest that social media monitoring tools are likely to disproportionately tag students of color as dangerous and that those students will be punished more severely than white students who are similarly identified. Overall, research shows that as school security measures proliferate, students often feel less safe. And overbroad and unnecessary surveillance is likely to have a detrimental impact on students’ privacy and chill their ability to express themselves.

Despite these considerable risks, dozens of companies new and old are stepping in to take advantage of schools’ and parents’ fears about the possibility of violence at school. Law enforcement agencies have used technology from some of these companies, like DigitalStakeout and MediaSonar, to monitor for “threats to public safety.” But in 2016, public records requests revealed that police were using these tools to surveil lawful protestors, leading major social media platforms to block data access by companies building surveillance tools for law enforcement. SmartProcure’s purchasing data suggests that in recent years, these companies’ share of the school surveillance market has been surpassed by companies that focus solely on schools, such as Geo Listening and Social Sentinel. These companies provide an extensive suite of services, including tracking social media, blogs, and forums. They also couch their services in the language of student wellness and mental health, claiming they can detect bullying and suicidal tendencies as well as acts or threats of violence.

Social media monitoring software is relatively affordable; according to our data, the median annual expenditure was $8,417 in 2018. But even a moderate expenditure can enable fairly extensive surveillance of students, with little guarantee that the money is making students and schools safer.

In addition, SmartProcure records show that the largest outlays are clustered within a small number of states (although this may be a result of the self-reported nature of the data). While districts in at least 20 states across the country have documented their purchases on SmartProcure, four states stand out among the purchases reported to the database: Texas ($654K), Illinois ($321K), Florida ($258K), and California ($197K). One school official has stated that “every school district across the state of Texas has been given some sort of direction to make these type[s] of moves.” In 2018, there was also a notable increase in purchases for Social Sentinel through a district services center or a regional organization rather than directly from the company. While this consortium pricing gives small districts purchasing power for educational technology that would otherwise be out of reach, it also magnifies the risk that local parents and other stakeholders will be in the dark about surveillance tools being implemented in their schools.

Social media and internet monitoring are only part of the burgeoning school surveillance system. Many school districts also have contracts with companies that integrate online monitoring of students with other “digital safety solutions.” Gaggle and Securly, for example, both monitor school-provided Gmail and affiliated applications (such as Google Drive, Google Docs, Google Calendar, and Contacts). These companies monitor content within the school’s network, including emails, texts, and networked documents. They are not included on the map because their focus is on school devices and networks rather than children’s social media use (though they can include social media posts originating from a school-provided email address). The contracts in SmartProcure suggest that purchases of these products are also increasing substantially: the total amount spent by school districts has more than doubled between 2013 ($3.9 million) and 2018 ($8.2 million). [MORE]