Fighting Back Against AI in the Classroom

New York City’s nearly 1 million public school students returned to class on Thursday morning for the beginning of a new school year. They were greeted by a new statewide ban on using cell phones during the school day. The new law sailed through Albany in the spring with broad, bipartisan support. The case is clear: rampant cell phone usage has been shredding students’ attention spans and making it harder for them to make friends and just be normal kids.

In 2023, NYC public school librarian Ben Mankoff wrote about his experience with cell phone addiction, the paradox of literacy, how we became so hooked on our digital devices and the sublime pleasures we rediscover when we set our phones aside.

Don’t Read This Article on Your Smartphone

By Ben Mankoff

Two years later, Mankoff argues in our August print edition that our schools face another, even bigger threat from Big Tech – the growing role of artificial intelligence in the classroom and how it de-skills students and teachers alike.

“Students are already surrounded by screens,” Mankoff writes, “their attention shredded, their critical thinking and empathy skills diminished by AI-generated slop. As librarians, we should nurture the skills that will guard students against the flood of misinformation: specifically, deep reading and organizing their thoughts through writing.”

Yes, I want to support The Indypendent today and help keep thoughtful, grassroots journalism flourishing in NYC!

Educators Should Be AI Antagonists

By Ben Mankoff

A high school teacher in New York City welcomes a new student from Haiti into their Global History class in the middle of a school year. This student reads only very basic English, and no one in the school speaks their native language of Haitian Creole. The teacher has to rapidly adapt course materials for their new student, and AI promises to be the most efficient tool for doing so. The teacher takes a short reading on the Industrial Revolution that discusses its effects on the Dominican Republic (where a majority of her students have recent ancestry) and asks ChatGPT to adapt it into simplified English, citing examples from Haitian history to make it more relevant to the new student. The AI program does this in a flash, and the Haitian student understands the lesson and is able to complete its accompanying short-response assignment at the same time as their peers. The course can move on, flow with no disruption.

The scenario above aggregates classroom situations in which New York City Public School teachers could imagine an appropriate use of generative artificial intelligence, or AI. Single instances such as this, in which GenAI seems to solve a classroom problem, are often put forth by the most well-meaning educators to defend adopting the technology’s adoption in schools, while “AI can be a tool for equity and accessibility” is the alluring promise made by ed-tech companies to progressive educators, recruiting them to welcome the technology into their classrooms.

One must credit a teacher who recognizes a student facing a challenge in their classroom. Better yet, this teacher deals with the challenge in ways that allow them to meet the bureaucratic and social demands of their workplace. Nonetheless, we should ask ourselves the following: is AI the necessary solution to this challenge? And what other problems does its use bring with it?

Accepting ed-tech companies’ promises of equity and accessibility presents at least four problems, as I will argue below. These arguments are aimed at the clarifying the problems associated with the use of AI by educators and school systems, not by students themselves.

First, there is little evidence that the best education is one that is precisely tailored, Netflix-like, to students’ existing preferences, beliefs, tastes and biases, especially when attention to the lived experience of the student is delivered by an algorithm rather than a caring teacher. This is an argument against the use of GenAI, even if it works exactly as advertised.

Second, there is a mountain of evidence that AI systems reproduce the prejudices, biases and prevailing ideologies of the social systems from which their data has been extracted. The opacity of these systems renders them unacceptable for use in schools.

Third, while those critical of AI use by students recognize its detrimental effects on building skills like critical thinking, they are less likely to recognize that these effects can be corrosive to the skills that adults have already built. “I’ve been teaching for a decade without AI, so I know good lessons when I see them and can effectively use ChatGPT to brainstorm lesson plans,” goes the argument. This perspective misses the very real de-skilling that occurs in adults with the introduction of automated tools.

Finally, there is persuasive evidence that AI is primarily a system of control for the owners of the technology far more than it is a tool for the liberation of its users.

Netflix, but for schools!

A number of convincing arguments hold that Netflix exercises a negative influence over both the aesthetics of cinema and the business of film production. The promise of efficient delivery of content through algorithmically determined profiles of the viewer (justified to funders as cost-effective and to viewers as increasing “representation”) has led to the devaluing of communal experiences like theatrical films or watercooler television — as well as to the devaluing of the labor that produces the content in whatever form it is delivered. Netflix was able to establish its dominance of entertainment by using the zero-interest funding that flowed into Silicon Valley in the wake of the 2008 financial crisis to prioritize growth and accumulation of users over profitability. It is this same financial situation that positioned tech firms to make further incursions into every realm of life, from transportation to education, carrying the imperative of infinite growth along with them.

Building monopolies by offering free services has been the practice of technology companies for ages. Capitalism’s tendency towards monopoly is supercharged by software; just ask anyone who has attempted to leave Instagram or Facebook. All your friends are there. We can observe that the same effects operate in ed-tech, such as in Microsoft and Google’s duopoly in New York City schools. How can we resist Gemini, Google’s AI model, when all of our students and teachers already rely on Google Classroom?

But let’s say you are not concerned about concentrated power or the flow of capital from public institutions to private corporations as long as the tools offered by this situation are useful in the classroom. It still remains to be proven that course material differentiated to the greatest possible degree is beneficial to any learner. For instance, one of the premises of personalized learning is that it is always better to teach to students’ “learning styles.” But according to cognitive scientist Daniel T. Willingham, there is insufficient evidence to support the idea that tailoring instruction to individuals’ learning styles is actually effective.

Personalized learning resulting in better education has been the promise of the computer industry since its inception, but that promise remains unrealized. For instance, the Organisation for Economic Co-operation and Development found in 2015 that the introduction of desktop computers into classrooms had little benefits for learning.

Everything is a trade-off; automation always has a cost, and there is reason to think that AI-adapted materials push that cost onto the student. As education researcher M.G. Prezioso recently wrote, “AI-adapted materials may not be the best way to improve reading outcomes. Students need access to complex texts with varied sentence structures and sophisticated language to support reading comprehension development. AI-generated writing, by contrast, can be syntactically repetitive and stylistically flat.”

There is evidence that personalized learning can produce gains in fact acquisition, under certain circumstances. But there is no evidence to suggest that personalized learning does anything special to support critical thinking, inquiry, or cooperative knowledge production — the core democratic promises of public education. Personalized learning employs techniques such as “chunking” texts to make the delivery of the information in those texts more manageable. This technique fails to produce the kind of engagement and skill building that deep reading does, skills that make up the foundation of critical thinking, metacognition and reflection, which support education’s democratic mission. Individual skill-building, on the other hand, supports education’s parallel mission of producing workers that the current day’s economy demands.

Treating personalization as the end goal of educational encounters tends to lead educators away from communal learning experiences. Rather than efficiently working our hypothetical new Haitian student into the existing course plan, another route would be to recognize the opportunity for important discussions to be had among students about the intertwined histories of Haiti and the Dominican Republic and the role industrialization has played in those histories.

While it has some narrow benefits, personalized learning brings with it dangerous goals of efficiency. AI-mediated personalized learning, taken to the extreme as its boosters would like, would serve to extinguish the benefits of group learning. At best, AI ed-tech providers envision the teacher’s role as an intermediary step in which the work of personalization is assisted by an AI that will eventually, through increased surveillance and data capture, be smart enough to do the work without a teacher at all.

The AI world is flat

Human beings, teachers included, come with their own biases, no doubt, but it is the variation, the diversity, of those biases within a classroom and school building that helps correct for that bias. An encounter between a teacher with a bias and a student who challenges that bias ultimately deepens the learning experience for both. AI flattens difference by design and privileges the prejudices and biases of the statistical average of the dataset.

It has been demonstrated in study after study that Large Language Models (LLMs), the bases for most systems we mean when we refer to AI, reproduce the biases of their datasets. This is not a bug but a feature of the LLM. Tech companies have methods for counteracting this bias, but those methods are informed by the interests of the companies themselves. Bias is inescapable, yet AI is presented as objective and omniscient.

You will often hear the refrain, “AI is like any other tool; what matters is how you use it.” Do not be fooled. In truth, AI is a collection of tools such as neural networks, transformers, silicon, and circuit boards, generative LLM chatbots, data centers, and many more. When we talk about AI in education, we are talking about a collection of tools, organized in very particular ways with very particular purposes. Even if we assume that the development of the technologies that comprise AI are inevitable (a claim worth interrogating), there is nothing inevitable about how those technologies are assembled or to what ends. The technologies that make up AI, and in fact most digital tools, are usually opaque to the individual user. They are the results of vast resource and production networks, and they are intricately, atomically designed and built by equally intricate machines. The “just a tool” line calls to mind a simple machine, such as a lever, that can be improvised from nearly any material and used by nearly anyone. Nothing could be further from the assemblage of technologies we call AI, which are almost universally wielded by corporations beholden to a legal imperative to maximize profits.

Considered in this light, this assemblage of tools presents the troubling scenario in which authoritarian governments in partnership with technology companies conduct massive re-education through the manipulation of the AI systems we are inviting into schools. Donald Trump is currently repurposing the Department of Education, issuing executive orders about what content is to be allowed in public schools alongside other orders aimed at increasing AI’s presence in K-12 education. OpenAI’s recent partnership with Instructure, “the world’s largest ed-tech ecosystem,” all but ensures that the same company contracting with the U.S. military, and therefore certain to adhere to the Trump administration’s new dictates on acceptable content, will be providing AI education services to millions of American students.

The adults in the room

More and more workers are relying on AI to help them rewrite emails to sound more professional. One administrator I spoke to told me they received 17 emails over spring break from an angry parent wanting to know if their child was failing a class. This information had been given to the parent already, and the administrator was understandably feeling frustrated. They drafted an email that didn’t hold back their feelings, then had ChatGPT revise it into something more polite, something that was less likely to exacerbate the situation. This administrator, who is critical of AI use in schools, chalked up this case as a win for AI. But was this not a lost opportunity for the practice of empathy? (If emotional losses are hard to quantify, there is research that shows that the use of generative AI like ChatGPT has a significant effect on critical thinking, through a process of cognitive offloading.)

It is hard to blame an overworked administrator who is accountable for the efficiency of their workplace for not wanting to spend her mental energy dealing with a stubborn parent. But the existence of this problem has social roots, not technical ones, and technical solutions have the overall effect of draining empathy from the educational experience while leaving the social causes unaddressed by the governments who contract with ed-tech companies.

There is a long history of automated tools serving to de-skill a workforce, and there is recent research on automation in South Korea that suggests that, far from simply augmenting human labor, as many tools do, software in particular primarily serves to replace it. The replacement of labor with an AI god is the express goal of the companies building AI, and in the case of education, this would represent a massive shift of public funding from labor to capital. Instead of paying more teachers more money to improve student outcomes, governments are paying private corporations to replace teachers with poor simulations of teachers.

There is nothing paranoid or conspiratorial about this. It’s happening in the open, and people like OpenAI CEO Sam Altman are explicit that their technologies are designed to replace workers on a massive scale. Even if the technology fails to live up to its promises, that isn’t stopping the replacement from happening. Just like the mechanical looms that replaced weavers in the earliest days of industrialization, the work-product is inferior but as long as capital is successfully redistributed up to the owners of capital, the replacement will continue.

Data colonialism

Alongside the reproduction of social inequalities through algorithms, the data-fication of education represents an extension and evolution of colonial domination. As Ulises Ali Mejias and Nick Couldry argue in their 2024 book Data Grab, data extraction by corporations with the support of governments is a direct descendant of colonial systems of oppression.

As Timnit Gebru, the prominent AI researcher who was fired from Google for exposing the biases of its algorithms has argued, AI itself has roots in eugenic movements.

Ideologically and financially influential people like billionaire venture capitalist Peter Thiel have expressed the belief that humanity’s biological and mental frailties must be overcome through technological means. Eugenics traditionally understood is the selective breeding of humanity. In the hands of people like Thiel, it becomes like technologically-driven human extinctionism, as revealed in his interview with Ross Douthat in which he cannot straight-forwardly say that he thinks the human species should continue to exist.

In his 2024 book, Access is Capture, Roderic Crooks argues that ed-tech programs generate great profits for the owners of the technology while doing little to change the outcomes of the students to whom they are supposedly bringing equality of opportunity. In fact, Crooks argues, the effect of these programs is to use public funds to turn minoritized students into sites of data extraction for the benefit of the technology industry. Ed-tech companies often require the schools that use their tools to enter into agreements that share students’ data.

‘AI is the future; we must embrace it.’

There is no guarantee that time or research will inevitably resolve any of the issues presented in this article. A typical refrain heard from AI boosters is that the technology “is the worst it will ever be,” suggesting that its problems are only one or two innovations away from being solved. Yet OpenAI’s latest release, its o3 “reasoning” model, is incorrect at a higher rate than its most recent predecessors.
The builders of these tools make compelling comments about how technological and social progress are one and the same. This idea serves to bring ever greater realms of society under their control, entrenching inequalities.

It is irresponsible to say, “AI is the future; we must embrace it.” This is such a broad statement that it hardly means anything, and yet it is often where the conversations in schools stop, opening the door far too wide for tech companies to enter through.

Students are already surrounded by screens, their attention shredded, their critical thinking and empathy skills diminished by AI-generated slop. As librarians, we should nurture the skills that will guard students against the flood of misinformation: specifically, deep reading and organizing their thoughts through writing.

We don’t have to shame students or teachers for using AI, but we should push back against the notion that the best way to deal with the disastrous information ecosystem created by AI is to immerse ourselves in it. It is better to practice the literate life, to respond to our students ourselves, with all our flaws, even if using ChatGPT could make course materials more “responsive.” It is more important that students are responded to by a human being than it is that the response be as efficient as possible. Educators should reinforce this approach, and not give in to pressures about preparing students for a world of work defined by AI.

If our care for students is unmediated by algorithms, if we nurture their skills of critical thinking and reflection, then they will do just fine navigating whatever world awaits.

Ben Mankoff is a New York City public school librarian.

The Indypendent is reader-funded. We publish a free monthly newspaper, a website, a weekly radio show on WBAI-99.5 FM and more. To make a one-time or recurring monthly donation, click here.