The other day I was building a data visualization tool for the teachers at SLA, and for the first time in over a decade, I had to write a regular expression. The wikipedia definition of regular expression is: “A regular expression is a sequence of characters that specifies a match pattern in text.” Regular Expressions (RegEx as it’s often called) are a royal pain to write, so much so that programming languages and spreadsheet function scripting have developed a bunch of functions for common RegEx uses so that you don’t have to write them.

But what I was looking to do wasn’t common, so I had to write a RegEx. I spent about a half-hour trying to remember how to do it, and then I realized that I could probably plug what I wanted to do into one of the GenAI sites and get it to work. Interestingly, I couldn’t make the code Gemini generated work, but I was able to get the code Claude created to work.

But here’s the thing — I didn’t learn how to write Regular Expressions by doing what I did. I figured out how to make Regular Expressions work for me, but that’s not the same thing. Fiddling around to make the code work gave me a better general working knowledge of RegEx, but if you asked me make some code using RegEx work without GenAI, I couldn’t do it. Now, for me as a working principal trying to just make a tool that works for my school, that’s all I needed. I wasn’t trying to learn, I was trying to build. And in that case, GenAI was a good tool for me to use… as long as I wasn’t fooling myself that I was learning RegEx. So if my goal was to learn how to build smart data visualization tools, mission accomplished. If this was a CSE class where I was supposed to be learning how to create regular expressions, I didn’t accomplish that goal.

This presents educators with a pretty big problem when it comes to thinking about when, how or why to incorporate Generative AI into the classroom. In the world of progressive education, the concept of “Learning by Doing” is pretty core to how we think about how kids learn. Fundamentally, it’s the idea that process matters as much as product. And when we assess product, we are using that product – in some part – as an manifestation of the learning process.

We can’t assume that anymore. And that creates a problem.

It’s a conundrum, because I think we want to expose kids to GenAI in school. It seems to fall into that category of “It’s a tool they’re going to use, and we probably want to make sure they have a working understanding of when and how to use it.” (And if you noticed the overwhelming use of hedge words in the beginning of this paragraph, they’re on purpose. The only think I am certain of about AI use in schools is that I am not certain of much.)

This is going to require all of us who love project-based learning and student writing and such to think deeply about when process matters and what that process has to look like. There will be clear uses of AI — I think every student who isn’t interested in being an artist but had to illustrate a story will find use for image generation AI. If a science teacher is more concerned with the analysis of data than the actual writing in the lab report, a thoughtful student might use GenAI to craft the writing. I know there are AI tools in video and audio creation that make it easer to make professional

And then there’s the use cases when GenAI can do interesting analysis that require students to ask better questions… Here’s an example. I asked Claude.ai the following prompt:

Do an analysis of the winners of the national league batting title for the last twenty years and draw conclusions about whether or not hitting has gotten harder.

I did that knowing that my analysis was pretty thin, but I was curious what Claude would say. Sure enough, Claude knew that the conclusions it drew weren’t that strong:

However, it’s important to note that this analysis is limited to the top performer each year. A more comprehensive study including league-wide averages and other offensive metrics would provide a more definitive answer about the overall difficulty of hitting in the National League over this period.

Would you like me to elaborate on any specific aspect of this analysis or explore any additional factors?

So sure… I asked it to include league-wide averages, and I got a better level of analysis… and because the analysis comes so quickly, I thought of more data to include. And I know from watching baseball that hitters aren’t worried about striking out like they used to be, so I thought that might be interesting to analyze. And there’s a stat OPS (on-base percentage plus slugging percentage) that would be interesting to look at too.

What I ended up with was some really interesting analysis that, if I wasn’t using AI, would have taken me hours and hours to do — more time than I think we’d want students to do. (Go check out the analysis. I would have pasted it here in its entirety, but it’d be longer than the blog entry.) Now, I think the chart is actually the most interesting part of the analysis, with the bullet points summarizing the chart also really important. I actually think that the conclusions Claude draws are the worst part of the artifact. As a baseball fan, there are things I think Claude misses, the addition of the designated hitter in 2022 and the banning of the defensive shift in 2023, for example. But the data collection is amazing, and the idea that we can give kids the ability to ask questions and gather data at speed does seem to matter. And it’s not hard to think of an assignment that asks kids to do data collection and draw their own conclusions.

So yes, I think we can create some really thoughtful ways students will use AI in schools, and it’s important we show students ways we can GenAI in ways to create and do and learn in ways we could not before, and yes, we probably should consider the promise this brings as much as we think about the concern we very much need to have about how GenAI can allow students to create product without going through a meaningful learning process.

But I’m not sure of any of this. I’m really not. I think about my own use of GenAI, and almost none of my uses of it could be considered learning. Mostly, I use it when I want a shortcut (writing a letter for a teacher who needs a jury duty postponement during state testing) or I want to do something I currently can’t do (the RegEx use case), but I don’t end up learning much from my use of it. And it’s not a leap to imagine that if I use it in those ways, so will students.

And that worries me a lot when I think about school. Because I think we have to think a lot about what it means to truly learn something. For a long time, doing a thing meant you learned a lot about the thing, so we could look at the product and understand that there was a learning process at play in most cases. That’s harder now.

For now, I think we’re really going to have to be even more intentional than we have been about considering the question of what it means to really learn something, and how we, as educators create the conditions for kids to learn by doing and how we document both product and process so everyone – students, teachers and other stakeholders – can reasonably understand what kids have really learned.

And I worry that this is going to get harder.