Introduction: Survey studies yield mixed results on the influence of the COVID-19 pandemic on academic performance, with limited direct evidence available. Methodology: Using the academic platform from the Italian university system, a large-scale archival study involving 30,731 students and 829 examiners encompassing a total of 246,416 exams (oral tests only) to scrutinize the influence of the COVID-19 pandemic on the likelihood of passing exams was conducted. Examination data were collected both in face-to-face and online formats during the pandemic. In the pre-pandemic period, only face-to-face data were accessible. Results: In face-to-face examination, we observed a lower probability of passing exams during the pandemic as opposed to pre-pandemic periods. Notably, during the pandemic we found an increased chance of passing exams conducted through online platforms compared to face-to-face assessments. Discussion and conclusions: These findings provide the first direct evidence of an adverse impact of the COVID-19 pandemic on academic performance. Furthermore, the results align with prior survey studies underscoring that using telematics platforms to evaluate students' performance increases the probability of exam success. This research significantly contributes to ongoing efforts aimed to comprehend how lockdowns and the widespread use of online platforms impact academic assessment processes.