Article Sidebar
Main Article Content
Abstract
This paper investigates and compares the use of three Large Language Models (LLMs), i.e., ChatGPT4, Google Gemini, and Claude 3.5, as decision support systems to plan the syllabus of a university course. The experiment was conducted in the context of the bachelor’s degree in Music Informatics of the University of Milan. The course under the exam focuses on the MIDI protocol, a very technical subject in the field of sound and music computing. The responses provided by LLMs have been evaluated by the author, a domain expert who has been teaching the subject for more than 10 years. From here emerges the provocative question in the title: can an AI-based chatbot prove to be more effective than an experienced teacher in defining educational objectives, materials, and the lesson plan for a university course? The results of the experimentation show that all three LLMs enable the formulation of a fairly comprehensive syllabus, allowing for the structuring of a university course. Their responses present interesting points of convergence in terms of both structure and content, while also highlighting some specificities. At present, biases and limitations still exist that make chatbots excellent co-pilots but do not replace the role of the teacher. Finally, a generalization is proposed to outline potential benefits and risks in the applicability of LLMs to the planning of educational activities.
Keywords
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
The author declares that the submitted to Journal of e-Learning and Knowledge Society (Je-LKS) is original and that is has neither been published previously nor is currently being considered for publication elsewhere.
The author agrees that SIe-L (Italian Society of e-Learning) has the right to publish the material sent for inclusion in the journal Je-LKS.
The author agree that articles may be published in digital format (on the Internet or on any digital support and media) and in printed format, including future re-editions, in any language and in any license including proprietary licenses, creative commons license or open access license. SIe-L may also use parts of the work to advertise and promote the publication.
The author declares s/he has all the necessary rights to authorize the editor and SIe-L to publish the work.
The author assures that the publication of the work in no way infringes the rights of third parties, nor violates any penal norms and absolves SIe-L from all damages and costs which may result from publication.
The author declares further s/he has received written permission without limits of time, territory, or language from the rights holders for the free use of all images and parts of works still covered by copyright, without any cost or expenses to SIe-L.
For all the information please check the Ethical Code of Je-LKS, available at http://www.je-lks.org/index.php/ethical-code
References
- Brown, P. F., Della Pietra, V. J., Desouza, P. V., Lai, J. C., & Mercer, R. L. (1992). Class-based n-gram models of natural language. Computational linguistics, 18(4), 467-480.
- Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877–1901.
- Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., Ye, W., Zhang, Y., Chang, Y., Yu, P. S., Yang, Q., & Xie, X. (2024). A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 15(3), 1-45.
- Christiano, P. F., Leike, J., Brown, T., Martic, M., Legg, S., & Amodei, D. (2017). Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30.
- Perković, G., Drobnjak, A., & Botički, I. (2024). Hallucinations in LLMs: Understanding and addressing challenges. 2024 47th MIPRO ICT and Electronics Convention (MIPRO), 2084-2088. IEEE.
- Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł, Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30.
- Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., & Le, Q. V. (2022). Finetuned language models are zero-shot learners. International Conference on Learning Representations (ICLR 2022). https://iclr.cc/virtual/2022/oral/6255