This is spot on. I'm involved in AI work in one of the hyper scalers and have been discussing this with a college professor friend for years. It's a huge problem and you hit on all the contours.
First, the incentives all point to scores over everything. You cannot expect people to make decisions counter to their incentives. So the incentives will have to get realigned which is super hard.
Second, you will never win an innovation battle against school kids. They outnumber you, are highly motivated and have lots of free time.
I think we will have to shift back to in person, non electronic methods. For instance, let them write the essay however but ask them to articulate their arguments and defend them in the class, in person, and grade on that. This will be costly.
Excuse my language here but this was a fucking brilliant read. I'm so fucking bored of reading AI commentary in education that is some variation on the theme of 'kids these days'. If there's one thing, just *one* thing that educators of any sort who work with kids should know and understand it's that the kids of these days are much the same as kids from those days and, indeed, you as a kid. It's not to underestimate the issues her but if you're going to get to grips with them then you have to understand them beyond the most banal, surface level of 'cheating is bad, m'kay?'
I think this also highlights a few shibboleths of, in particular, 'evidence based' or 'traditional education', namely:
Standardised tests are unquestionably good, and actually measure what they (claim to) set out to measure.
Grading improves performance and is motivating.
Homework is good and improves performance.
Ironically, the evidence for all of these is sketchy, at best, and often tautologically used 'x methods improves scores in standardised tests therefore X method must work because standardised tests prove it works'.
Exclusively teacher led education cannot cut it anymore because the kids have so many other ways of accessing knowledge, so many ways around adult gatekeeping, and kids know and can't stand it when adults are obviously and consistently hypocritical.
The AI lesson planning point illustrates that perfectly - teachers consistently expect from students things that they, as adults, neither do nor would accept for themselves. Students using AI to write? Awful! A disgrace! The end of education! Teachers using AI to write lesson plans and summarise information they could look up for themselves in *minutes*? Inject it into my veins, baby!
We're cooking the planet so that students can use AI to answer AI generated questions to get better at (soon to be if not already) AI generated tests marked by AIs in order to access qualifications for jobs that AI is going to render obsolete within 5 years anyway.
We use google classroom at my high school, and the students are given chromebooks. This actually makes giving exams pretty easy, since google forms (what we use to administer tests) “locks” a students Chromebook (they cannot access any other tab or program while taking the test). If a student exits the exam, I get a prompt email and will have a conversation with the student.
Most of my work is done on paper, because chromebooks are cheap pieces of shit. But this is one instance in which I’m grateful to google.
Stray thought - maybe AI could start answering "Yeah, it looks like you're a school student studying for a test, so I'm not going to give you the answer. Try working it out for yourself. Maybe I could give you some pointers if you're stuck?"
There's got to be a million reasons this isn't going to happen.
When you ask someone to build a chair, reap a field or climb a ladder, they cannot cheat. They might use tools to help them, but the point is the completion of the work. Why is it that a student can cheat in their work? Why is it that an LLM can replace the labour of a person?
The purpose of the GPA is as a validation of the students’ mental labour. The examination is the point at which this exchange of mental labour for a grade takes place. Not focusing on the GPA is equivalent to not trying to get paid for your work. Cheating, here, is like skimming the cash register or fudging your own wages. The test is the exchange, not the process of learning.
If the content that students studied were self-directed, they would no longer be working for someone else. If they weren’t working for someone else, their goals in studying wouldn’t be orientated for the purpose of a test. In other words, the student would not be producing knowledge for the purposes of exchange, but for the purposes of the knowledge. An LLM couldn’t replace them, because they would be working on knowledge whose existence must be created, both in themselves and in their output.
LLMs are only able to make a mockery of education because education is a mockery of learning — study that only exists because it is necessary for the maintenance of our present society. Without this society, the ruling class would cease to exist. The only rational solution is the abolition of the exam in schools, and in society, the abolition of wage-labour and private property.
What’s crazy is that AP exams, which college-level and challenging for the strongest students, only have hand-written essays. I think this means schools are takin the easy road and giving up on really teaching the students below AP level.
I also wonder if AP scores will start dropping as students cheat with AI more and more. I don’t see how they will be able to pass their AP exams.
This all seems to reflect a larger identity crisis in society. I’m not sure we even have a working definition of knowledge anymore. Artificial “intelligence” only exposes the heart of that crisis; and in many ways, it’s a bastardization of the term. We’ve confused producing the correct output with actually knowing something. I think this confusion extends from early education all the way through the workplace. So, to echo your sentiment, students are just trying to meet the standards and play the game the adults have set for them.
If you switch to bluebooks and in-class on-paper tests and your students score noticeably lower than others in the same courses but taught by different faculty, is everyone else going to be held at fault or you?
The underlying problem is not the new technology, but the demand for high scores for all students all the time. Seems to me, anyway.
I conduct most of my tests on paper, including multiple-choice questions and short-answer questions. And this year, I'll be giving entirely paper-based tests. You're right, anything on Canvas is ridiculous at this point, even written assignments. At this point, I'd rather they receive an answer from ChatGPT and then have to transcribe it by hand and submit it than simply cut and paste.
I had been moving away from online assignments (I'll be talking about my Explainers soon...) and was pretty satisfied with where I was, but as I said, the final was at the end of the year, exhaustion had set in, and judgment went AWOL. I think a large part of this was the lie we were sold about how wonderful Canvas would be for online learning, how overworked we get, and how doing things online streamlined a lot of our work for us.
My concern is that trying to convince a school full of teachers this coming year that they need to get off Canvas will not endear me to any of them. And I love your comment about "too European!" - we should do a side-by-side comparison of systems, assignments, etc. It would be like the videos of Europeans eating American foods... :)
I get your whole point absolutely, but i have a question (trying to not come as aggressive or anything, it's just curiosity): why do you not have paper tests?
In France, we don't take online tests, like at all. Some teachers have slides to present their classes, but most just use the projector for documents and then teach. All exams, tests, ... that count for your grades are done on paper, without phone or computer access. That's true for all ages, and even at university. Maybe the only difference is that at university you might get assigned group projects, but the grade isn't on the project itself, it's on the oral presentation of it, with deep questions from the jury to all members individually (and at that point if you cheated with AI but are able to answer all the questions thoughrouly, i'm not sure it's a bad thing).
You might get some at-home projects, exercises or essays, but they typically don't count in your grade, they're just training that the teacher corrects, and maybe gives an indicational grade to tell you how well you did.
It's not impossible to cheat, some people always manage to, but it reduces it by a lot!
To me, having anything on Canva is a complete non-sense! What is the point of it? What does it allow you to do that you can't do on paper? Again, genuine question. Maybe i'm just too european, but the first thing that comes to my mind is data privacy for minors, and then costs for the school.
I think moving to paper would solve a lot of these problems. I taught college students for years and never did exams or quizzes online except for a couple of terms when Covid made that the only option.
Most schools really push educational software on you, and most faculty claim to like using it, which baffles me. It's invariably clunky and poorly designed. It serves as a surveillance tool for administrators. It makes it easy for students to cheat. I don't get the appeal for teachers outside of laziness and not wanted to seem old-fashioned.
What I would have to figure out if I still taught would be how to do out-of-class papers, which made up the bulk of what I graded people on. I just fed ChatGPT an old assignment and asked it to produce a paper as a response, and what it spat out would have gotten a D, an F, or a required meeting with me that would have resulted in getting a zero on the grounds of cheating. I gave the bot more specific instructions, which produced something much better but still banal--a low passing grade despite being free of writing errors, which means that the argument was junk. I think my more clever students could have produced something pretty good with less than an hour's work, though, and would have learned little doing so, and that would have been a problem.
I'm glad it's not a problem that I'm having to deal with since leaving academia. I'm sure, though, that managing other tech-related problems will be an ongoing source of frustration.
Now i have a new question: why do your essays have to be "out of class"?
I'm going to parallel that again with the French system: the out of class ones are practice, that are fake-graded to help up improve. But the real ones are done in class, as "exams" if you want. There's generally 2 per semester for a class, and they last anywhere from 3 hours to 6. Some teachers allow open books, others nothing. Of course, there are accommodations for disabled students who need a computer (without internet) or a note-taker, but everyone else writes it down by hand.
The other common thing is oral presentations, that people prepare at home, often in groups. The teacher usually chooses the topics, or at least approves them, and every student in the class listens in too. Then there's questions, which remove the "i used ai" aspect. We generally get 1 per class per semester.
This idea of having to write essays and turn them in printed from your computer is so foreign to me! I've always seen it in study vlogs on youtube, and i've been quite jealous because it looked so much easier than what we have to do 😅
They didn't have to be, but the idea is to simulate the kind of writing you do when you're writing academic papers and other types of writing for publication. Sitting with a pile of books around you, resource articles printed out and marked up, full-text articles up on multiple tabs on your laptop, word-processing documents open that you use to outline and keep track of your writing and resource materials, and the document open that you're actually writing in (which will probably undergo revision as you refine your argument, cut things, move things, polish sentences to improve verb choice and improve sentence-to-sentence cohesion, respond to objections you didn't think of in the first place, and many other possible changes). It's not the kind of thing that you can do in six hours, or a dozen, especially if the paper is over ten pages long, and it's a writing process that's substantially different than what would usually take place in an examination room. My focus in a literature class would be to have the students write a final research paper that would be suitable to send somewhere. It would prepare them for scholarly work--or to write a well-researched piece on a place like Substack. I encouraged them to write for humans more than academics even when doing scholarly writing--to produce something that reads like a literary essay.
It's a very useful model that I learned a lot from as an undergraduate and in grad school. I also learned how to write well on 3-hour closed-book essay exams that I sat for in class, but I think there are things that I wouldn't have learned as well about research and writing if I hadn't done so much out-of-class writing that counted a lot toward my grade in the course.
We might have to give up doing this as much because of AI. And because cheaters should be kicked out of school. I think that is unfortunate, though.
Something that might help is assigning out-of-class papers that also have to be part of an oral presentation with a high-stakes (and challenging) question-and-answer sessions. This worked well with students in our senior seminars. It would be difficult to implement in every class, though, because of time and labor considerations.
Matt - all of this resonates. I wrote my most recent post covering much of the same ground from a slightly different angle. Are you familiar with LockDown Browser? We use a different LMS, but it’s a third party tool that literally does what it says / while the students are taking an online assessment through the LMS, they cannot do ANYTHiNG else on the device. Ask your IT to look into it. It’s a nightmare for most teachers right now and I don’t know how it gets better in the short term.
I’ll check it out, thanks! Agreed - this coming year will be tough from the standpoint of convincing the faculty this is actually an issue, unifying them towards action (as implied, some don’t even check logs, and the kids know…), and coming up with solutions. I’m planning on going with pencil and paper for all tests and quizzes, and a buddy of mine was telling me he wants to go “technology free” in his room. I think there are plenty of solutions for folks to use…as long as we can get them all convinced it’s an issue.
This is spot on. I'm involved in AI work in one of the hyper scalers and have been discussing this with a college professor friend for years. It's a huge problem and you hit on all the contours.
First, the incentives all point to scores over everything. You cannot expect people to make decisions counter to their incentives. So the incentives will have to get realigned which is super hard.
Second, you will never win an innovation battle against school kids. They outnumber you, are highly motivated and have lots of free time.
I think we will have to shift back to in person, non electronic methods. For instance, let them write the essay however but ask them to articulate their arguments and defend them in the class, in person, and grade on that. This will be costly.
Excuse my language here but this was a fucking brilliant read. I'm so fucking bored of reading AI commentary in education that is some variation on the theme of 'kids these days'. If there's one thing, just *one* thing that educators of any sort who work with kids should know and understand it's that the kids of these days are much the same as kids from those days and, indeed, you as a kid. It's not to underestimate the issues her but if you're going to get to grips with them then you have to understand them beyond the most banal, surface level of 'cheating is bad, m'kay?'
I think this also highlights a few shibboleths of, in particular, 'evidence based' or 'traditional education', namely:
Standardised tests are unquestionably good, and actually measure what they (claim to) set out to measure.
Grading improves performance and is motivating.
Homework is good and improves performance.
Ironically, the evidence for all of these is sketchy, at best, and often tautologically used 'x methods improves scores in standardised tests therefore X method must work because standardised tests prove it works'.
Exclusively teacher led education cannot cut it anymore because the kids have so many other ways of accessing knowledge, so many ways around adult gatekeeping, and kids know and can't stand it when adults are obviously and consistently hypocritical.
The AI lesson planning point illustrates that perfectly - teachers consistently expect from students things that they, as adults, neither do nor would accept for themselves. Students using AI to write? Awful! A disgrace! The end of education! Teachers using AI to write lesson plans and summarise information they could look up for themselves in *minutes*? Inject it into my veins, baby!
We're cooking the planet so that students can use AI to answer AI generated questions to get better at (soon to be if not already) AI generated tests marked by AIs in order to access qualifications for jobs that AI is going to render obsolete within 5 years anyway.
AI forces us to realize what was already true.
The best teachers aren't the ones who catch students cheating. They are the ones who can enchant them with the process of education itself.
Why can't you have a high res camera in the back of the classroom that records all the students screens at the same time.
We use google classroom at my high school, and the students are given chromebooks. This actually makes giving exams pretty easy, since google forms (what we use to administer tests) “locks” a students Chromebook (they cannot access any other tab or program while taking the test). If a student exits the exam, I get a prompt email and will have a conversation with the student.
Most of my work is done on paper, because chromebooks are cheap pieces of shit. But this is one instance in which I’m grateful to google.
It's a very good article thank you.
Let's be creative: do you think school is possible without homework?
Stray thought - maybe AI could start answering "Yeah, it looks like you're a school student studying for a test, so I'm not going to give you the answer. Try working it out for yourself. Maybe I could give you some pointers if you're stuck?"
There's got to be a million reasons this isn't going to happen.
When you ask someone to build a chair, reap a field or climb a ladder, they cannot cheat. They might use tools to help them, but the point is the completion of the work. Why is it that a student can cheat in their work? Why is it that an LLM can replace the labour of a person?
The purpose of the GPA is as a validation of the students’ mental labour. The examination is the point at which this exchange of mental labour for a grade takes place. Not focusing on the GPA is equivalent to not trying to get paid for your work. Cheating, here, is like skimming the cash register or fudging your own wages. The test is the exchange, not the process of learning.
If the content that students studied were self-directed, they would no longer be working for someone else. If they weren’t working for someone else, their goals in studying wouldn’t be orientated for the purpose of a test. In other words, the student would not be producing knowledge for the purposes of exchange, but for the purposes of the knowledge. An LLM couldn’t replace them, because they would be working on knowledge whose existence must be created, both in themselves and in their output.
LLMs are only able to make a mockery of education because education is a mockery of learning — study that only exists because it is necessary for the maintenance of our present society. Without this society, the ruling class would cease to exist. The only rational solution is the abolition of the exam in schools, and in society, the abolition of wage-labour and private property.
What’s crazy is that AP exams, which college-level and challenging for the strongest students, only have hand-written essays. I think this means schools are takin the easy road and giving up on really teaching the students below AP level.
I also wonder if AP scores will start dropping as students cheat with AI more and more. I don’t see how they will be able to pass their AP exams.
This all seems to reflect a larger identity crisis in society. I’m not sure we even have a working definition of knowledge anymore. Artificial “intelligence” only exposes the heart of that crisis; and in many ways, it’s a bastardization of the term. We’ve confused producing the correct output with actually knowing something. I think this confusion extends from early education all the way through the workplace. So, to echo your sentiment, students are just trying to meet the standards and play the game the adults have set for them.
If you switch to bluebooks and in-class on-paper tests and your students score noticeably lower than others in the same courses but taught by different faculty, is everyone else going to be held at fault or you?
The underlying problem is not the new technology, but the demand for high scores for all students all the time. Seems to me, anyway.
Is that with lockdown browser on Canvas, or is it just as easy to get around that?
My district doesn’t believe lockdown browsers exist. The video was made with Edge, which is an acceptable browser to use with Canvas.
Ah OK, so lockdown could be an improvement, but I hear there are ways to get around it. And if it's unsupervised, forget about it.
You had me at the image from Idiocracy.
Hi Rose!
I conduct most of my tests on paper, including multiple-choice questions and short-answer questions. And this year, I'll be giving entirely paper-based tests. You're right, anything on Canvas is ridiculous at this point, even written assignments. At this point, I'd rather they receive an answer from ChatGPT and then have to transcribe it by hand and submit it than simply cut and paste.
I had been moving away from online assignments (I'll be talking about my Explainers soon...) and was pretty satisfied with where I was, but as I said, the final was at the end of the year, exhaustion had set in, and judgment went AWOL. I think a large part of this was the lie we were sold about how wonderful Canvas would be for online learning, how overworked we get, and how doing things online streamlined a lot of our work for us.
My concern is that trying to convince a school full of teachers this coming year that they need to get off Canvas will not endear me to any of them. And I love your comment about "too European!" - we should do a side-by-side comparison of systems, assignments, etc. It would be like the videos of Europeans eating American foods... :)
I get your whole point absolutely, but i have a question (trying to not come as aggressive or anything, it's just curiosity): why do you not have paper tests?
In France, we don't take online tests, like at all. Some teachers have slides to present their classes, but most just use the projector for documents and then teach. All exams, tests, ... that count for your grades are done on paper, without phone or computer access. That's true for all ages, and even at university. Maybe the only difference is that at university you might get assigned group projects, but the grade isn't on the project itself, it's on the oral presentation of it, with deep questions from the jury to all members individually (and at that point if you cheated with AI but are able to answer all the questions thoughrouly, i'm not sure it's a bad thing).
You might get some at-home projects, exercises or essays, but they typically don't count in your grade, they're just training that the teacher corrects, and maybe gives an indicational grade to tell you how well you did.
It's not impossible to cheat, some people always manage to, but it reduces it by a lot!
To me, having anything on Canva is a complete non-sense! What is the point of it? What does it allow you to do that you can't do on paper? Again, genuine question. Maybe i'm just too european, but the first thing that comes to my mind is data privacy for minors, and then costs for the school.
I think moving to paper would solve a lot of these problems. I taught college students for years and never did exams or quizzes online except for a couple of terms when Covid made that the only option.
Most schools really push educational software on you, and most faculty claim to like using it, which baffles me. It's invariably clunky and poorly designed. It serves as a surveillance tool for administrators. It makes it easy for students to cheat. I don't get the appeal for teachers outside of laziness and not wanted to seem old-fashioned.
What I would have to figure out if I still taught would be how to do out-of-class papers, which made up the bulk of what I graded people on. I just fed ChatGPT an old assignment and asked it to produce a paper as a response, and what it spat out would have gotten a D, an F, or a required meeting with me that would have resulted in getting a zero on the grounds of cheating. I gave the bot more specific instructions, which produced something much better but still banal--a low passing grade despite being free of writing errors, which means that the argument was junk. I think my more clever students could have produced something pretty good with less than an hour's work, though, and would have learned little doing so, and that would have been a problem.
I'm glad it's not a problem that I'm having to deal with since leaving academia. I'm sure, though, that managing other tech-related problems will be an ongoing source of frustration.
That's super interesting to hear!
Now i have a new question: why do your essays have to be "out of class"?
I'm going to parallel that again with the French system: the out of class ones are practice, that are fake-graded to help up improve. But the real ones are done in class, as "exams" if you want. There's generally 2 per semester for a class, and they last anywhere from 3 hours to 6. Some teachers allow open books, others nothing. Of course, there are accommodations for disabled students who need a computer (without internet) or a note-taker, but everyone else writes it down by hand.
The other common thing is oral presentations, that people prepare at home, often in groups. The teacher usually chooses the topics, or at least approves them, and every student in the class listens in too. Then there's questions, which remove the "i used ai" aspect. We generally get 1 per class per semester.
This idea of having to write essays and turn them in printed from your computer is so foreign to me! I've always seen it in study vlogs on youtube, and i've been quite jealous because it looked so much easier than what we have to do 😅
They didn't have to be, but the idea is to simulate the kind of writing you do when you're writing academic papers and other types of writing for publication. Sitting with a pile of books around you, resource articles printed out and marked up, full-text articles up on multiple tabs on your laptop, word-processing documents open that you use to outline and keep track of your writing and resource materials, and the document open that you're actually writing in (which will probably undergo revision as you refine your argument, cut things, move things, polish sentences to improve verb choice and improve sentence-to-sentence cohesion, respond to objections you didn't think of in the first place, and many other possible changes). It's not the kind of thing that you can do in six hours, or a dozen, especially if the paper is over ten pages long, and it's a writing process that's substantially different than what would usually take place in an examination room. My focus in a literature class would be to have the students write a final research paper that would be suitable to send somewhere. It would prepare them for scholarly work--or to write a well-researched piece on a place like Substack. I encouraged them to write for humans more than academics even when doing scholarly writing--to produce something that reads like a literary essay.
It's a very useful model that I learned a lot from as an undergraduate and in grad school. I also learned how to write well on 3-hour closed-book essay exams that I sat for in class, but I think there are things that I wouldn't have learned as well about research and writing if I hadn't done so much out-of-class writing that counted a lot toward my grade in the course.
We might have to give up doing this as much because of AI. And because cheaters should be kicked out of school. I think that is unfortunate, though.
Something that might help is assigning out-of-class papers that also have to be part of an oral presentation with a high-stakes (and challenging) question-and-answer sessions. This worked well with students in our senior seminars. It would be difficult to implement in every class, though, because of time and labor considerations.
Matt - all of this resonates. I wrote my most recent post covering much of the same ground from a slightly different angle. Are you familiar with LockDown Browser? We use a different LMS, but it’s a third party tool that literally does what it says / while the students are taking an online assessment through the LMS, they cannot do ANYTHiNG else on the device. Ask your IT to look into it. It’s a nightmare for most teachers right now and I don’t know how it gets better in the short term.
I’ll check it out, thanks! Agreed - this coming year will be tough from the standpoint of convincing the faculty this is actually an issue, unifying them towards action (as implied, some don’t even check logs, and the kids know…), and coming up with solutions. I’m planning on going with pencil and paper for all tests and quizzes, and a buddy of mine was telling me he wants to go “technology free” in his room. I think there are plenty of solutions for folks to use…as long as we can get them all convinced it’s an issue.