IMPLEMENTASI CHATBOT AKADEMIK BERBASIS LARGE LANGUAGE MODEL UNTUK MENINGKATKAN LAYANAN INFORMASI MAHASISWA
DOI:
https://doi.org/10.35969/187m6e69Keywords:
Academic Chatbot, Artificial Intelligence, Higher Education, Large Language Model, Digital TransformationAbstract
The rapid advancement of Artificial Intelligence (AI) has significantly transformed digital services in higher education institutions. Academic information services often face challenges such as limited service hours, repetitive inquiries, and delayed responses from administrative staff. This study aims to design and implement an academic chatbot based on Large Language Models (LLMs) to improve the effectiveness and accessibility of academic information services for university students. The research adopts the Design Science Research Methodology (DSRM), consisting of problem identification, objective definition, design and development, demonstration, evaluation, and communication stages. The proposed system integrates an LLM with an academic knowledge base containing information regarding academic regulations, course schedules, tuition payments, thesis procedures, graduation requirements, and student services. Evaluation was conducted using Black Box Testing, response accuracy assessment, and the System Usability Scale (SUS). The results show that the chatbot achieved a response accuracy of 91.5% from 200 academic questions and obtained an SUS score of 84.2, categorized as Excellent. Furthermore, the system reduced the average response time from approximately 15 minutes to less than 10 seconds. The findings indicate that LLM-based chatbots can significantly improve academic service quality while supporting digital transformation initiatives in higher education institutions.
Downloads
References
[1] E. Kasneci, K. Sessler, S. Küchemann, M. Bannert, D. Dementieva, F. Fischer, et al., “ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education,” Learning and Individual Differences, vol. 103, pp. 102274, 2023.
[2] C. K. Lo, “What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature,” Interactive Learning Environments, vol. 31, no. 4, pp. 1–15, 2023.
[3] E. Adamopoulou and L. Moussiades, “An Overview of Chatbot Technology,” Artificial Intelligence Applications and Innovations, vol. 584, pp. 373–383, 2020.
[4] L. Labadze, M. Grigolia, and L. Machaidze, “Role of AI Chatbots in Education: Systematic Literature Review,” International Journal of Educational Technology in Higher Education, vol. 20, no. 56, 2023.
[5] M. Sallam, “ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review,” Healthcare, vol. 11, no. 6, pp. 887, 2023.
[6] D. Yigci and M. Yildiz, “Large Language Model-Based Chatbots in Higher Education,” Advanced Intelligent Systems, vol. 7, no. 2, 2025.
[7] W. Gan, Z. Qi, J. Wu, and J. C.-W. Lin, “Large Language Models in Education: Vision and Opportunities,” arXiv preprint arXiv:2311.13160, 2023.
[8] Y. Dan, Z. Lei, Y. Gu, Y. Li, J. Yin, J. Lin, et al., “EduChat: A Large-Scale Language Model-Based Chatbot System for Intelligent Education,” arXiv preprint arXiv:2308.02773, 2023.
[9] M. Bernabei, G. Marrocco, and A. Di Carlo, “Students’ Use of Large Language Models in Engineering Education,” Education Sciences, vol. 13, no. 11, 2023.
[10] M. Abedi, I. Alshybani, M. R. B. Shahadat, and M. S. Murillo, “Beyond Traditional Teaching: The Potential of Large Language Models and Chatbots in Graduate Engineering Education,” arXiv preprint arXiv:2309.13059, 2023.
[11] K. Labadze and N. Bokhua, “From Intent-Based Chatbots to LLM-Powered Teaching Assistants,” Results in Engineering, vol. 22, 2024.
[12] Y. Kurniawan, M. T. Santoso, and A. Wijaya, “Development of Chatbot Application to Support Academic Services in Higher Education,” IEOM Proceedings, pp. 4334–4343, 2022.
[13] L. Fan, L. Li, Z. Ma, S. Lee, H. Yu, and L. Hemphill, “A Bibliometric Review of Large Language Models Research from 2017 to 2023,” arXiv preprint arXiv:2304.02020, 2023.
[14] M. Peláez-Sánchez, A. García-Holgado, and F. J. García-Peñalvo, “The Impact of Large Language Models on Higher Education: Exploring the Connection Between AI and Education 4.0,” Frontiers in Education, vol. 9, 2024.
[15] Y. Raini, A. Nugraha, dan R. Prasetyo, “Chatbot Edukasi AI Berbasis Large Language Model untuk Pembelajaran Digital,” Educate Journal, vol. 8, no. 1, 2026.
[16] I. Sommerville, Software Engineering, 10th ed. Boston: Pearson, 2021.
[17] R. S. Pressman and B. Maxim, Software Engineering: A Practitioner’s Approach, 9th ed. New York: McGraw-Hill, 2020.
[18] D. Jurafsky and J. H. Martin, Speech and Language Processing, 3rd ed. Stanford University, 2024.
[19] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, et al., “Attention Is All You Need,” dalam Advances in Neural Information Processing Systems, 2017.
[20] K. Peffers, T. Tuunanen, M. Rothenberger, and S. Chatterjee, Design Science Research Methodology for Information Systems Research. New York: Springer, 2007.
[21] T. B. Brown et al., “Language Models Are Few-Shot Learners,” NeurIPS, vol. 33, pp. 1877–1901, 2020.
[22] W. X. Zhao et al., “A Survey of Large Language Models,” arXiv preprint arXiv:2303.18223, 2023.
[23] J. Brooke, “SUS: A Retrospective,” Journal of Usability Studies, vol. 8, no. 2, pp. 29–40, 2013.
[24] G. Booch, J. Rumbaugh, and I. Jacobson, The Unified Modeling Language User Guide, 2nd ed. Boston: Addison-Wesley, 2005.
[25] M. Fowler, UML Distilled: A Brief Guide to the Standard Object Modeling Language, 3rd ed. Boston: Addison-Wesley, 2004.
