As AI technology continues to evolve in the digital era, developing AI literacy among college students has become a crucial educational priority. This study aims to establish a scientific AI literacy evaluation system and to empirically assess the AI literacy levels of undergraduate students at Wuhan University, with the findings providing data support and theoretical reference for future AI education policy-making and curriculum design in higher education institutions. In response to the demands of AI education and university talent cultivation objectives, this study develops an AI literacy evaluation system for college students, based on the KSAVE (knowledge, skill, attitude, value, and ethics) model and the UNESCO AI competency framework. The system includes 4 level-1 indicators (AI attitude, AI knowledge, AI capability, and AI ethics), 10 level-2 indicators, and 25 level-3 indicators. The Delphi method was used to determine indicator content, while the analytic hierarchy process was employed to calculate the weights for each level of indicators. Through large-scale questionnaire surveys and statistical analysis, the study empirically measured the AI literacy levels of 1,651 undergraduate students at Wuhan University and analyzed variations in AI literacy across factors including gender, academic year, academic discipline, and technical background. The results demonstrate that the constructed AI literacy evaluation system is scientifically sound and highly applicable, providing a comprehensive and objective measure of students’ AI literacy levels. Furthermore, notable differences were observed in AI literacy levels across different dimensions among Wuhan University undergraduates, with variables such as academic discipline, technical background, and participation in digital intelligence education programs significantly influencing students’ AI literacy, particularly in knowledge and capability dimensions.