A new artificial intelligence-based student surveillance tool that has sparked controversy amid claims that it violates student privacy and exacerbates biased discipline practices was adopted on a trial basis Monday by the San Antonio Independent School District board of trustees.
The tool, called Gaggle, will monitor the online activity of students, including emails, Google Drive contents and chat messages, in an effort to identify potential risks such as cyberbullying, self-harm and violence, according to SAISD officials.
Once the tool identifies a risk, it sends the content to two levels of moderators who will review whether it is an actual threat. If it is deemed credible, a moderator alerts school officials who will then take appropriate action to intervene, according to Gaggle CEO and founder Jeff Patterson.
Mike Eaton, the district’s chief operations and building security officer, said the software will be tested for 90 days at CAST Tech High School, Sam Houston High School and Japhet Academy to see if the tool is right for the district.
The move comes amid growing concern about school violence, including shootings and mental health risks. Some school leaders say the tool is necessary for student safety.
Eaton, who used the software in his previous job overseeing security in Denver schools, said the three schools were selected for monitoring due to their high usage of devices and programs.
“We had Gaggle as one of our systems for monitoring student activities, student email, Google Drives, and it was just another piece of our security platform,” Eaton said. “So, when I joined SAISD and learned they didn’t have something like that, I recommended that during one of our district safety and security meetings.”
The district is paying $6,000 for the pilot, according to board documents.
Parents will not be able to have their children opt out of the program, according to Eaton. Students may be transferred to another school depending on availability.
“We reserve the right, obviously, to implement any safety measures that we believe are appropriate utilizing systems that are provided to the students that are district-owned,” he said.
Patterson told the San Antonio Report that the tool is meant as a safety measure, not a punitive one.
All content is anonymized before it is reviewed and revealed only in the event an administrator or law enforcement needs to make contact with the student, Patterson said.
According to a report compiled by the software company, which began as a secure student email service in 1999, there was a 26% increase in student safety incidents between the 2020-21 and 2021-22 school year. There was a 36% increase in incidents involving violence in elementary grades.
Eaton recalled several incidents in Denver, where students wrote about wanting to die, triggering late-night emergency responses that, in at least one case, stopped a child attempting to commit suicide in that moment.
Critics however, argue that Gaggle’s monitoring goes too far and could lead to unfair discipline. Some are concerned that the tool will have a chilling effect on student free speech and expression, as students may be hesitant to share their thoughts and feelings online knowing they are being monitored.
Unintended consequences
Chelsea Barabas, a Massachusetts Institute of Technology doctoral candidate who has researched the use of surveillance tools in Texas, said if Gaggle is used for disciplinary purposes, it could exacerbate longstanding disparities in school discipline.
“There’s a slippery slope, a slippage that can easily happen with surveillance tools like this because there aren’t strong checks and balances with regard to how they get used, what kinds of behavior … get monitored,” Barabas said. “So if you look in a state like Texas, where the largest reason that kids are expelled from school or put in some sort of alternative education program … as a discretionary decision by a teacher or an administrator are code of conduct violations, which don’t have anything to do with school safety or harming kids or violence or anything like that, but it’s a significant driver of pushing kids out of school.”
“So in an environment like that, if you adopt a tool like Gaggle … will this not slip into being used for much less serious code of conduct violations instead of to prevent school shootings and things like that?,” Barabas added.
A survey of students, teachers and parents about the technology by the Center for Democracy and Technology released last year found that concern wasn’t unfounded. It looked at student monitoring tools in general, including Gaggle.
Elizabeth Laird, the director of equity at the Center for Democracy and Technology, said the study was conducted to ensure the rapid expansion of devices being used by students during the COVID-19 pandemic did not result in loss of privacy.
According to the results, 70% of teachers whose schools use this technology say that they use it to determine if a student has violated disciplinary policy, while only 47% of teachers said it is used in the event of a mental health crisis. Teachers were able to selected multiple options for the question. Laird also expressed concern that such monitoring could lead to a chilling effect on student speech, citing the survey results:
- Approximately 5 in 10 students agree with the statement: “I do not share my true thoughts or ideas because I know what I do online may be monitored.”
- Approximately 8 in 10 students agree with the statement: “I am more careful about what I search online because I know what I do online may be monitored.”
Online surveys of nationally representative samples of 1,606 sixth- to 12th-grade parents and 1,008 sixth- to 10th-grade teachers, as well as two surveys of ninth- to 12th-grade students (with n-sizes of 400 and 460) were fielded May/June 2022.
The discipline resulting from the surveillance also falls along racial lines, according to Laird, who found that Black and Hispanic students are more likely to be disciplined because of the technology than their white peers.
The skewed results weren’t limited to race: 29% of students who identified as LGBTQ said they had been “outed” by monitoring software.
Until recently, Gaggle filtered for words including “gay” and “lesbian” to be forwarded for review. The practice drew scrutiny and has since been discontinued, but Patterson said it was intended to prevent bullying.
“We were monitoring for some LGBTQ terms, [but] not to out anybody. It’s because those terms are often used to harass students,” Patterson said. “We’ve kept some of the derogatory ones on because we really want to make sure we’re not missing any warning signs.”
The survey’s findings were not limited to Gaggle. Other school surveillance apps including Social Sentinel pose similar risks, advocates say.
The North East Independent School District has used that software, which uses AI to search public social media profiles, for six years.
NEISD Police Chief Wally McCampbell recently spoke about the success of using the social media-monitoring software to keep students safe, citing an uptick in emails about self-harm.
McCampbell emphasized that the software is not used to target specific groups of students and is solely focused on student safety.
“It’s purely 100% a safety system that we can prevent possible suicides or any type of homicide or, you know, obviously assaults and stuff, too,” McCampbell said.
Other area districts also use monitoring software for safety purposes, including Northside ISD, which uses Bark; Edgewood ISD, which uses Social Sentinel; and Judson ISD, which has a contract with Gaggle.
