To Test the A.I. Learning Hype, I Visited Classrooms

Times Insider explains who we are and what we do and delivers behind-the-scenes insights into how our journalism comes together.

In January, Marisa Shuman, a computer science teacher at the Young Women’s Leadership School of the Bronx, invited me to spend a few days embedded in her classroom.

Her school, a public middle and high school for girls, specializes in math, science and technology. And she thought I might be interested in a lesson she had just prepared on ChatGPT, an artificial intelligence-powered chatbot that can manufacture book reports and social studies essays.

As a reporter who has spent years chronicling how tech companies and their tools are reshaping public schools, I jumped at the chance.

At the time, ChatGPT was beginning to blow up in schools and on college campuses. Tech executives had started promoting familiarity with A.I. tools as a crucial skill for students.

Meanwhile, New York City Public Schools, the nation’s largest school system, had just blocked access to ChatGPT on school devices and networks over concerns of cheating and inaccuracy.

Ms. Shuman, however, saw it as a teachable moment.

She used ChatGPT at home to generate a lesson on fitness trackers and other wearable technology. Then she tried the material with her 11th and 12th graders.

She told her students that she didn’t care if they learned nothing about wearable tech. But she did want them to examine the accuracy and effectiveness of the lesson that the chatbot had generated.

In other words, Ms. Shuman was using the A.I. tool as an exercise for her students to practice critical tech thinking.

And her students were freely critical. They found that the chatbot-generated lesson contained errors, used advertising come-ons and asked over-simplistic questions.

“It reminded me of fourth grade,” one student said.

It was a reminder to me that there is no substitute for journalists visiting institutions to observe what is happening firsthand and interviewing participants face-to-face. It was also the spark for a reporting project that would take me across the country: If we wanted to offer readers an on-the-ground view of the new A.I. education boom, I needed to visit a lot more classrooms.

I was already aware that some school districts were feeling pressure to quickly introduce generative A.I. technologies — that is, tools like ChatGPT, trained on vast databases of digital texts or images, that can produce texts or visuals in seconds — for student use.

That was partly because some prominent tech companies, executives and billionaires were hailing A.I. chatbots as education game-changers. The tools, they promised, were sure to revolutionize, and automatically personalize, student learning.

There was also widespread FOMO: Some tech leaders warned that students would be unable to compete for jobs if they didn’t know how to use A.I.

I set out to learn how these tools were affecting teaching and learning in schools — and whether the classroom reality lived up to a form of ed-tech hype I’d covered before.

Over the years, Silicon Valley companies, billionaires and industry-financed nonprofits have promoted a series of tech products as revolutionary education innovations. But so far, there’s not much rigorous evidence showing that video-based tutorials or personalized learning apps have significantly improved students’ educational outcomes.

So I wondered: Would generative A.I. be different?

I was fascinated by the promise of A.I. tutoring bots. So I started off by spending a morning at Khan Lab School, a nonprofit private school in Palo Alto, Calif., where a sixth-grade math class was trying out a new A.I. tutor called Khanmigo.

There, teachers encouraged students to tinker with the bot, which was developed specifically for school use by Khan Academy, a related — but separately run — nonprofit education organization.

Some students playfully asked Khanmigo to answer math questions in Gen Z slang or in the form of a rap song. One student who caught Khanmigo making an addition error promptly corrected the bot.

Across the country, I found reviews of the tutoring bot more mixed.

At First Avenue Elementary School in Newark, a third-grade teacher leading a class on fractions posted specific math questions on a white board that she wanted her students to ask Khanmigo. The bot responded by giving the students step-by-step instructions to solve the problems.

School officials who observed the class told me that they found the A.I. tool overly helpful. They said they wanted students to be able to think through the problem-solving steps themselves.

I’ve seen a lot of enthusiasm, and innovative uses, of A.I. in schools as well. On a recent visit to Walla Walla, Wash., about a four-hour drive from Seattle, I met teachers who were using ChatGPT to create imaginative literary games and storytelling assignments for their students.

But the lesson I learned from visiting schools this year was not so much about technology skills.

From the Bronx to Walla Walla, school officials and teachers told me that they felt it was as important for students to learn to ask critical questions about artificial intelligence as it was to learn how to use the technology. In fact, for some of them, it was even more important.

I also learned that there were more stories to report, as many schools and teachers are only beginning to discuss what they think A.I. education should look like.

So I’m planning to visit more schools soon. If you’re an educator who would like to host me at your school or share your experience using A.I. tools, please fill out this form.