Human–AI Feedback Synergy Assessing the Reliability and Contextual Depth of Generative Evaluation Systems in Enterprise-Scale Education
DOI:
https://doi.org/10.63282/3050-9416.IJAIBDCMS-V6I4P102Keywords:
Generative AI, Large Language Models (LLMs), Human–AI Collaboration, Feedback Automation, Educational Assessment, Reliability, Contextual Depth, Enterprise Learning Systems, Human-in-the-Loop Evaluation, EdTech IntegrationAbstract
The rapid evolution of Generative Artificial Intelligence (GenAI) and Large Language Models (LLMs) has redefined automation capabilities in enterprise-scale education, particularly in the domains of assessment, personalized feedback, and learner analytics. As organizations increasingly deploy AI-driven evaluation tools to enhance scalability and reduce instructor workload, questions remain regarding the reliability, contextual sensitivity, and pedagogical authenticity of AI-generated feedback when compared with human evaluators. This study investigates the qualitative dimensions of human–AI feedback synergy within creative learning contexts, focusing on design-based education where subjective interpretation and contextual judgment are central to evaluation quality. Utilizing OpenAI’s GPT-4 and its custom-configured evaluation models, the research compares AI-generated feedback with that of experienced human assessors across 25 student typography projects from the Visual Media program at King Abdulaziz University. A mixed-methods framework is adopted, combining rubric alignment analysis, thematic coding of qualitative feedback, and perception surveys from both instructors and learners. The findings reveal that while AI systems demonstrate high consistency and linguistic precision, they exhibit limitations in contextual depth, aesthetic reasoning, and value articulation, leading to perceptual divergence in learner reception. The study concludes with a discussion on best-practice design principles for integrating GenAI evaluation models within institutional workflows, proposing a Human-in-the-Loop feedback architecture that balances efficiency with academic authenticity. The results contribute to the growing body of knowledge on AI-augmented assessment ecosystems, offering insights relevant to EdTech developers, enterprise learning platforms, and higher education administrators aiming to operationalize trustworthy, scalable, and context-aware AI feedback systems
References
[1] C. Luckin, W. Holmes, M. Griffiths, and R. Forcier, Intelligence Unleashed: An Argument for AI in Education. London, U.K.: Pearson Education, 2019.
[2] N. A. Johnson, P. Shum, and R. Ferguson, “AI for Learning: Scaling Intelligent Feedback in Education,” IEEE Trans. Learn. Technol., vol. 16, no. 4, pp. 522–537, 2023.
[3] W. Holmes, M. Bialik, and C. Fadel, Artificial Intelligence in Education: Promises and Implications for Teaching and Learning. Boston, MA, USA: Center for Curriculum Redesign, 2022.
[4] R. Ferguson, S. Buckingham Shum, and R. Clow, “AI and Analytics in Education: Opportunities and Challenges,” Brit. J. Educ. Technol., vol. 54, pp. 1541–1561, 2023.
[5] D. Boud and E. Molloy, Feedback in Higher and Professional Education: Understanding It and Doing It Well. London, U.K.: Routledge, 2019.
[6] L. Floridi, “AI and Education: Towards Responsible Deployment,” AI & Society, vol. 39, pp. 1123–1137, 2024.
[7] J. W. Creswell and J. D. Creswell, Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 5th ed. Thousand Oaks, CA, USA: Sage Publ., 2023.
[8] K. Siau and W. Wang, “Building Trust in Artificial Intelligence, Machine Learning, and Robotics,” Cutter Business Technol. J., vol. 33, no. 2, pp. 47–53, 2020.
[9] M. Zawacki-Richter, T. Marín, M. Bond, and F. G. González, “Systematic Review of Research on Artificial Intelligence Applications in Higher Education,” Int. J. Educ. Technol. Higher Educ., vol. 16, no. 1, 2019.
[10] A. Holzinger, G. Langs, H. Denk, K. Zatloukal, and H. Müller, “Interactive Machine Learning for Health Informatics: When Do We Need the Human-in-the-Loop?,” IEEE Access, vol. 8, pp. 101996–102009, 2020.
[11] G. Chen, R. Ferguson, and S. Buckingham Shum, “Human–AI Collaboration in Education: Emerging Trends and Research Opportunities,” IEEE Trans. Learn. Technol., vol. 18, no. 1, 2025.
[12] D. Carless and D. Boud, “The Development of Student Feedback Literacy: Enabling Uptake of Feedback,” Assessment & Evaluation in Higher Educ., vol. 44, no. 7, pp. 1060–1070, 2019.
[13] D. Nicol, “The Power of Feedback Revisited: A New Model for Feedback Practice,” Assessment & Evaluation in Higher Educ., vol. 47, no. 6, pp. 817–831, 2022.
[14] J. Piaget, The Principles of Genetic Epistemology. London, U.K.: Routledge, 1972.
[15] A. N. Kluger and A. DeNisi, “The Effects of Feedback Interventions on Performance: A Historical Review, a Meta-Analysis, and a Preliminary Feedback Intervention Theory,” Psychol. Bull., vol. 119, no. 2, pp. 254–284, 1996.
[16] L. W. Anderson and D. R. Krathwohl, A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. New York, NY, USA: Longman, 2019.
[17] R. K. Yin, Case Study Research and Applications: Design and Methods, 6th ed. Thousand Oaks, CA, USA: Sage Publ., 2021.
[18] T. Brown, B. Mann, N. Ryder, et al., “Language Models Are Few-Shot Learners,” in Proc. NeurIPS, 2020.
[19] J. Cohen, “A Coefficient of Agreement for Nominal Scales,” Educ. and Psychol. Measurement, vol. 20, no. 1, pp. 37–46, 1960.
[20] V. Braun and V. Clarke, Thematic Analysis: A Practical Guide. London, U.K.: Sage Publ., 2022.
[21] M. Q. Patton, Qualitative Research and Evaluation Methods, 5th ed. Thousand Oaks, CA, USA: Sage Publ., 2022.
[22] L. Van der Spoel, E. Hennissen, and L. Volman, “AI-Enhanced Learning Environments,” IEEE Access, vol. 11, pp. 45602–45618, 2023.
[23] B. Mittelstadt, P. Allo, M. Taddeo, S. Wachter, and L. Floridi, “The Ethics of Algorithms: Mapping the Debate,” Big Data & Society, vol. 6, no. 2, 2019.
[24] S. Buckingham Shum and R. Ferguson, “Social Learning Analytics,” Educ. Technol. & Soc., vol. 22, no. 1, pp. 3–17, 2019.
[25] M. Anyoha, “The Rise of Artificial Intelligence in Learning Systems,” IEEE Computer, vol. 57, no. 2, pp. 68–77, 2024.
