CHARLOTTE, N.C. — The same body scanners you'll find inside Charlotte-Mecklenburg Schools middle and high schools, Bank of America Stadium and Blumenthal Performing Arts Center cannot create "weapons-free zones" as first advertised, according to a confidential test report. The report, which Evolv Technology wanted to keep private, shows the body scanners failed to detect some knives and even a handgun during a 2021 field test in Ohio.
It's unclear when the company disclosed those results to CMS. To date, the district has spent roughly $16.5 million with Evolv Technology.
The school system first bought the artificial intelligence-based touchless body scanners in March, following an increase in the number of guns found on campuses. In a video produced by CMS in April 2022, relieved students talked about the benefits of the technology.
"I know that I'm not going to be threatened with any firearms, any knives," one student said.
The truth is, neither the students nor CMS can say that with any certainty.
Overpromised?
“Evolv has made it very clear that they can't be trusted," IPVM Director of Government Research Conor Healy said. "There absolutely needs to be more transparency,"
IPVM, a research company, first obtained the test results through a public records request after the University of Southern Mississippi's National Center for Spectator Sports Safety and Security (NCS4) mistakenly released the private document.
The confidential report, put together in 2022 and secured by WCNC Charlotte’s content partner WBNS-TV in Columbus, shows while the scanners detected almost every gun, they missed one particular kind of gun on two walkthroughs, failed to detect about four out of every 10 knives and had mixed results for a certain type of pipe, which could be used in a bomb.
In multiple interviews in 2021 and 2022, CEO Peter George repeated a talking point that Evolv had written signatures for every weapon.
“The CEO has gone on the news and has said that they recognize all the weapons, all knives, and all bombs and it's patently false," Healy said. "We don't know if the schools that buy Evolv know that that's false."
Neither Evolv nor CMS would reveal specifically when the company made the district aware of the NCS4 findings, but by August 2022, four months after the school system first installed the technology in its high schools, CMS said it was “aware of strengths/shortcomings based on discussions with Evolv, people who already had the systems, and our own implementation and results at the high schools."
When pressed, CMS Chief Operating Officer Brian Schultz could not provide a timeline.
"I don’t recall that other than the fact that Evolv has always been open about their systems and the upgrades they are implementing to improve upon the successful results we are experiencing related to the number of weapons discovered in schools," he said through a district spokesperson.
Columbus City Schools in Ohio, meanwhile, bought body scanners last year, but confirmed today to 10TV News Chief Investigative Reporter Bennett Haeberle the district was not provided a copy of the NCS4 report prior to purchase.
Near the end of 2022, the Utica City School District in New York removed its Evolv scanners after a knife fight.
"It’s not a reliable system for detecting knives," the superintendent said during a November 2022 school board meeting. "The Evolv weapons system was designed for use at large arenas or stadiums to prevent incidents of mass casualties and is not adequate or practical for school use."
Changing the language
Evolv changed its marketing language from "weapons-free zones " to "safer zones" around the same time, but the company said the change was not in response to any criticism.
Chief Growth Officer and Co-founder Anil Chitkara said Evolv has remained transparent with its customers, including CMS, about the technology’s limitations.
"Our goal is to fully inform our customers of the system's capabilities, so they can make the best and most informed decision," Chitkara said.
"Do you all feel like you've misled your customers?” WCNC Charlotte asked Chitkara.
“I hope we've not misled our customers," he said. "I hope they are fully informed of what the system does and doesn't do. The worst case would be if they expected it to do something and it doesn't do it. That doesn't do anybody any good."
Chitkara said Evolv provides access to the results of the 2021 field test and other reports if customers ask to view the specific results. However, Evolv said the information is not shared with the general public, because that would be reckless.
"We want to be as transparent as possible," he said. "We don't want it to be in the public's hands."
"Do you regret the messaging early on?” WCNC Charlotte asked Chitkara.
“I think we've talked about the system differently over time as you have different conversations," he responded. "I think what's important is what does it actually do? What it does is, it's very accurate at detecting weapons and ignoring personal items and the technology is very good at doing that. It's very good. It's not perfect, because nothing's perfect. What we try to do is to be as clear as possible in terms of what it does and what it doesn't do and that there are limitations and we can talk about those in a very detailed way with our customers."
"When you look at the original messaging, do you all regret the way things were worded?" WCNC Charlotte asked.
"We're focused on what the system can do and we're trying to be as clear, transparent to our customers and whatever language we can use, whatever explanation we can use, reports that we can use to help inform our customers. That's what we're trying to do," Chitkara said.
"Should you all have used different language earlier on?" WCNC Charlotte pressed.
"I don't know. We're trying to work with our customers to fully inform them and help them understand in the clearest way possible and that's what our goal is," he said.
"Words are impactful and we want to make sure we're using the right words to explain the system and so, we're doing that and we're working to be better at that every day," Evolv Director of Education Jill Lemond added.
Lemond, a former assistant superintendent, worked in a Michigan district when a mass shooter killed four high school students
"I can say with integrity the Evolv system was the only way in which I felt comfortable bringing my staff and students back into that physical structure," she said.
Lemond and Chitkara told WCNC Charlotte the company has improved its technology since the 2021 NCS4 test and said they are not aware of any additional reports of guns or knives getting through.
"Any absolute sounding claims, while in the heat of a conversation, can be troublesome and that's not what we're trying to do," Lemond said. "We're committed to being open and honest."
Evolv reported its body scanners successfully detected 170,000 concealed guns and knives in all of 2022.
"I will say that nothing is infallible, and detection depends on the specific settings in use, circumstances, and security operations of a venue," Chitkara clarified by email after his interview with WCNC Charlotte. "It is important for the public to know that no technology is 100% perfect which is why a layered approach that includes people and processes is essential to mitigate risk."
Guns found on campus
CMS declined an on-camera interview for this story, but offered a statement instead.
"In the 2021-22 return to full-time face-to-face instruction, the district experienced an alarming increase in the number of firearms being brought to and reported on CMS campuses. Evolv was the solution selected to directly address and solve for the problem of firearms being brought onto our campuses from the community," CMS said in an email. "In our work with Evolv, they have communicated all aspects of the Evolv Express system – including limitations and capabilities – with CMS staff. We are confident in the fact that Evolv is proactively taking steps in improving their detection of weapons through their software upgrades since the time of the NCS4 report."
The district said Evolv “has always been forthcoming about their systems,” including connecting the school system with others who already use the body scanners.
Within their first month of use, the scanners found a loaded gun at Berry Academy. The data suggest the mere presence of body scanners has helped deter weapons too.
The district has reported a huge drop in guns and other weapons found on campuses after the school system moved from random searches to body scanners. CMS documented 31 guns on campus during the 2021-22 school year, but only three so far this school year. However, what we don’t know is how many, if any, guns or knives have come into schools undetected.
"There is no foolproof system to ensuring the safety of staff and students in every single possible situation," the district added. "With that being said, CMS has been a public school leader in proactive measures to reduce the frequency of incidents. The security approaches in CMS include multiple layers that address different types of threats and provide redundancy (surveillance cameras, active survival training, campus security associates, anonymous reporting system, entry buzz-in systems, Lobby Guard, SROs, Evolv weapon detection system, etc.) Most recently, CMS received a $2.8M grant to increase the number of cameras at elementary schools, install a pilot badge access/camera integration system, and equip classrooms with active survival response tools. Each layer typically incorporates people, processes and technology. This approach recognizes the reality that no single layer or single technology is 100% effective. As a result of what we have discovered thus far is a significant reduction in the number of firearms and non-firearm weapons."
Emails show Evolv shared a list of "Talking Points" with CMS in March after WCNC Charlotte first reached out with questions about the NCS4 report.
In response, CMS's COO praised Evolv.
"I really appreciate how transparent and proactive the whole team at Evolv has been with strengths, limitations, process improvement, software upgrades," emails show he said in reply.
CMS announced in December the body scanners also detected a gun at West Mecklenburg High School.
Even with this technology, a gun was still able to get on South Mecklenburg High School's campus earlier this year. CMS confirmed to WCNC Charlotte the student in question did not go through a body scanner. Instead, law enforcement found the gun in the student's backpack during a fight. CMS has declined to comment on how the gun made its way onto campus and wouldn't reveal why the student did not go through a body scanner. It's important to note that South Meck, like some other schools, is an open campus.
NCS4 published a shorter report last year that gave Evolv's system a score of 2.84 out of 3, but did not reveal specifics about the technology's limitations. The BBC, with help from IPVM, was the first to report the technology's shortcomings.
Blumenthal Performing Arts Center CEO Tom Gabbard said successful use of these body scanners is based on a balancing act.
"The sensitivity of these scanners is adjustable by the operator," Gabbard said in a statement to WCNC Charlotte. "The art in their use is finding a balance between identifying objects of concern, and false alarms that unreasonably delay public entrance."
WCNC Charlotte reached out Bank of America Stadium as well. A stadium spokesperson said, “Consistent with stadium policy we don’t discuss details related to stadium security measures and procedures.”
Editor's note: Since Evolv's scanners are so widely used in Charlotte and have cost more than $16 million, WCNC Charlotte felt taxpayers deserve to know what the technology can and cannot do, especially since other outlets have previously reported on the field test in question.
Contact Nate Morabito at nmorabito@wcnc.com and follow him on Facebook, Twitter and Instagram.
WCNC Charlotte is always asking "where's the money?" If you need help, reach out to WCNC Charlotte by emailing money@wcnc.com.