Robin Pemantle

Over the past few years, I’ve been experimenting with giving students oral exams. I’m aware that there are lots of reasons that folks hesitate to use oral exams—don’t they take too much time? Don’t students get nervous? Will students share questions with each other? What can you really tell in a short exchange? I understand these questions but what I’ve found is that giving students an oral exam is a great opportunity to connect with students and see what they know. 

I give an oral exam in an advanced calculus class that has about 50 students (it might be too time-consuming in a larger class but could possibly work there, too). Each student signs up for a 15-minute time slot. In that time, I give them one easy problem and one hard problem. I have ten different problems of each type and I use a random generator to assign them to students. This way, absent a rather large consipiracy, most students have no specific information about the problems they will actually get. The student works each problem on the board in front of me, talking through their solution on the board. I give them time in the beginning to collect their thoughts while I do email or some other task that lets them know I’m not watching, hovering, or hurrying them. They’re always more eager to get started than I am, so I’ve literally never had to prompt someone to begin speaking. Sometimes I offer suggestions or support. As they work, I take notes on how they are doing. I assign students a score using a four point rubric: a four means they “nailed it,” a three means they did it with some help, a two means they struggled but got there, with more help than it should have taken, a one means in the end they still could only get bits and pieces, and a zero means they couldn’t answer the question at all. Keeping the grading rubric simple means that I don’t have to spend a lot of time worrying about what grade to give and the grade gets assigned quickly. 

In terms of the time it takes to conduct oral exams, it’s nearly a wash when there are 40 students, and is feasible even at 60 students. If there would have been two written midterms, the one oral midterm (a second is not necessary) takes way less time. Creating the exam is quick because I don’t have to vet the questions as thoroughly, there is no makeup exam, and I don’t write solutions. Some years, by popular demand for a study aid later in the term, the TAs write solutions. A written exam is usually followed by a grading party, taking up about 15 person-hours for a class this size. Add another 4.5 person-hours of proctoring and it’s clear: a traditional midterm eats up more time. True, in the oral exam the person-hours are all on me, not shared with the TAs. So for me personally, it’s still more work, not less. But the TAs make up for this later with review sessions, creating solutions to homework and exams, and a ton of communication back to me, twice weekly. 

Concerning the accuracy of the assessment, the small number of problems is more than made up for by the flexibility to respond to students if they misunderstand the question or start on an unproductive path. I also get important feedback about the questions I’m asking: I can see a bad question and fix it before it throws off the whole exam. Whether the student receives a question he or she knows, versus one that is a bit of a reach, I get to see the thinking process and hear how well each student has learned to organize, articulate, and reason. This important part of the meta-curriculum would go largely untested in a rushed 90-minute written exam with 8-10 questions. On those exams, one can’t grade on presentation much, if at all.

The 15-minute oral exam also allows a great fallback in cases where a student has a grade complaint. My rule on this is, “No problem, let’s schedule a retake where you can show me you really know this material a lot better than I gave you credit for.” Retakes are easy on oral exams:15 minutes. Despite my willingness to do this for every student who wants, no matter the reason, the uptake is under 10%. Last semester, for example, it was 3 out of 40. What if a student wants a retake because he or she didn’t really know the material the first time, and manages to learn it much better by the second time? Great! I got a student to learn a buch of stuff that he or she didn’t learn before. Do I care whether the student knew this in week 7 versus week 6? No. Does this induce students to procrastinate and get behind? No. No one wants to look bad in front of the teacher the first time. Most bad showings are due to the fact that the student had no idea how to self-assess. The oral exam helps students learn to gauge what they know, more than does a traditional written exam. On the written exam a student’s takeaway is “I got a 58”. On the oral exam, I tell them what I saw and why, e.g., “You know a lot of techniques, but they seem jumbled in your mind,” or, “You actually understand this problem, you just got thrown off by a messy calculation.” The benefit of individual feedback, targeted to each student’s needs, is huge. 

When students want do-overs because they are in denial about how much they don’t understand, that’s fine with me. For one thing, I don’t always know. For another, even if I did know, they benefit from seeing that the second time didn’t go so well either. After the second try, they are ready for my advice about how to learn better, and I’m in a better position to give advice because I have a better idea of the origin of their struggles. I can talk with them on the spot. I can better use tools like Course Action Notices because I’ve seen in person how they have come up short. I can also see class-wide struggles more clearly and address them. For example, one year I saw that lots of my students had trouble with linear algebra. Because I saw it in action, I could provide a resource that helped everyone. 

Overall, making a better connection with my students is useful to me and useful to them in taking exams. Having another person there with them gives them perspective. They see their process through my eyes, and in case they don’t, I articulate it for them. This helps my students better value their own problem solving process and take their own learning more seriously. Giving these exams is somewhat time consuming, and not for everyone, but I’ve found it much more useful, interesting, and fun than any other mode of assessment.

Robin Pemantle is a professor of mathematics and the Christopher H. Browne Distinguished Endowed Professor in the School of Arts & Sciences.

--

This essay continues the series that began in the fall of 1994 as the joint creation of the College of Arts and Sciences, the Center for Teaching and Learning and the Lindback Society for Distinguished Teaching. 

See https://almanac.upenn.edu/talk-about-teaching-and-learning-archive for previous essays.