In a bold initiative to adapt to the AI era in education, an NYU professor has incorporated AI into oral exams. Panos Ipeirotis, from NYU’s Stern School of Business, recognized a troubling trend in student assignments that appeared well-crafted yet reflected a lack of true understanding. This prompted him to implement AI-powered oral exams as a countermeasure.
Revolutionizing Student Assessments
Ipeirotis observed that when students were called upon to defend their coursework, they frequently struggled. This raised concerns about whether written assignments were accurately measuring comprehension. To enhance assessments, he stated, “We need to reward understanding, decision-making, and real-time reasoning.”
AI-Powered Oral Exams
- Ipeirotis revived oral exams using AI technology to make them scalable.
- The AI examiner was developed using ElevenLabs’ conversational speech technology, a process he described as quick and straightforward.
- This assessment involved two parts: questioning students about their capstone projects and prompting them to analyze a case discussed in class.
Over nine days, the AI system evaluated 36 students, with each session lasting about 25 minutes. The entire operation incurred roughly $15 in compute costs, contrasting sharply with the hundreds of dollars typically spent on human-administered oral exams.
AI Grading Process
In addition to administering the exams, Ipeirotis employed AI to grade the students’ performances. Three different AI models—Claude, Gemini, and ChatGPT—independently graded each oral exam transcript. They cross-reviewed their evaluations to arrive at a final score, with Claude acting as the decision synthesizer.
Mixed Reactions from Students
While the AI grading was found to be more consistent and fair than human evaluations, student feedback was mixed. A minority expressed a preference for AI oral exams over traditional written assessments. However, many students found the AI format more stressful, despite acknowledging it offered a better measure of understanding.
AI in Academia: Challenges and Opportunities
The implementation of AI in education has raised important discussions. As universities confront the challenges of AI in assessments, a recent paper highlighted how instructors feel overwhelmed by the changing landscape. Many are unsure about crafting AI-proof evaluations, while some regard AI as undermining academic integrity.
As the educational community navigates these challenges, Ipeirotis’s experience offers a glimpse into possible solutions. He believes in harnessing AI not as a substitute for critical thinking but as a means to enhance students’ understanding and capabilities.
As discussions about the future of assessments continue, Ipeirotis’s innovative approach highlights the evolving relationship between education, technology, and the importance of genuine learning.
