在ChatGPT推出後，其即時回應功能顯示了人工智慧在教育領域的潛力，特別是在解答疑難問題、提供學術建議和協助資料查找方面。然而，它在教育應用上仍面臨諸多挑戰，例如被動回覆、生成內容通用話，不夠具體、無法融入教師們的教學經驗等等。因此我們創建了名為 Educational Agent Crafting Tool 的教育代理人創建系統，讓教師們將自己的教學經驗融入自己的教育代理人。本研究著眼於改善與擴展EduACT系統，特別是針對聊天機器人的創建流程與教學任務補足。

首先，本研究設計了一個新的代理人創建方法（Agent Builder），此方法通過及時的聊天與互動引導，使教師能夠在引導之下一步一步的創建他的對話代理人。此外系統還支援了自動化的對話測試，大大提高了創建過程的效率和幫助創建者發現設計上的問題。其次，根據實驗發現系統中的代理人有11.3%的對話情境沒有適合的任務，因此我們設計了動態任務將沒有任務可選的對話情境減少至6.7%，動態任務的提示詞會根據每個代理人的任務目標而有所不同，從而生成更為精確和個性化的回覆給使用者。

為了評估選擇動態任務後的回覆是否更好，我們評估了不適合的任務所生成的回覆與兩種設計方法的動態任務所生成的回覆的差異。結果表明，人工設計的動態任務其回覆不足以處理不同代理人的對話情境，而生成的動態任務其回覆品質有64%比起不適合的任務更好或相等，甚至在16%的對話資料中，回覆品質能遠遠超過前述兩種任務，證明了動態任務在補足對話上的靈活性和有效性。

總之，本研究的貢獻在於為教育聊天機器人的設計與應用提供了一種新的方法論框架，這一框架強調了用戶友好性和系統靈活性的重要性，為未來在相關領域的研究和實踐提供了寶貴的經驗和參考。

Since the launch of ChatGPT, its real-time response capabilities have showcased the potential of artificial intelligence in the education sector, especially in answering questions, providing academic advice, and assisting with data retrieval. However, its application in education still faces many challenges, such as passive responses, generic generated content, lack of specificity, and inability to integrate into teachers' teaching experiences. Therefore, we created an educational agent crafting system called the Educational Agent Crafting Tool (EduACT) to allow teachers to incorporate their teaching experiences into their own educational agents. This study focuses on improving and expanding the EduACT system, particularly the process of creating chatbots and supplementing teaching tasks.

Firstly, this study designed a new agent creation method (Agent Builder) that guides teachers through real-time chat and interaction, enabling them to create their conversational agents step-by-step. Additionally, the system supports automated dialogue testing, significantly improving the creation process's efficiency and helping creators identify design issues. Secondly, according to experiments, 11.3% of dialogue scenarios in the system lacked suitable tasks, so we designed dynamic tasks to reduce the scenarios with no available tasks to 6.7%. The prompts for dynamic tasks vary based on each agent's task goals, generating more precise and personalized responses for users.

To evaluate whether the responses after selecting dynamic tasks were better, we assessed the differences between responses generated by unsuitable tasks and those generated by dynamically designed tasks using two methods. The results indicated that manually designed dynamic tasks were insufficient to handle different agent dialogue scenarios. In contrast, dynamically generated tasks produced responses that were superior in 64% of cases compared to unsuitable tasks. In 16% of dialogue data, the response quality far exceeded the aforementioned tasks, demonstrating the flexibility and effectiveness of dynamic tasks in supplementing dialogues.

In conclusion, this study contributes a new methodological framework for designing and applying educational chatbots. This framework emphasizes the importance of user-friendliness and system flexibility, providing valuable experiences and references for future research and practice in related fields.

[1] Benjamin S Bloom. The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational researcher, 13(6):4–16, 1984.
[2] Lasha Labadze, Maya Grigolia, and Lela Machaidze. Role of ai chatbots in education: systematic literature review. International Journal of Educational Technology in Higher Education, 20(1):56, 2023.
[3] David Baidoo-Anu and Leticia Ansah. Education in the era of generative artificial intelligence (ai): Understanding the potential benefits of chatgpt in promoting teaching and learning. Journal of AI, 7, 03 2023.
[4] Duong Mai, Can Da, and Nguyen Hanh. The use of chatgpt in teaching and learning: a systematic review through swot analysis approach. Frontiers in Education, 9:1328769, 02 2024.
[5] Yang Deng, Lizi Liao, Liang Chen, Hongru Wang, Wenqiang Lei, and Tat-Seng Chua. Prompting and evaluating large language models for proactive dialogues: Clarification, target-guided, and non-collaboration. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 10602–10621, Singapore, December 2023. Association for Computational Linguistics.
[6] 林子平. 基於大型語言模型應用指示詞打造無程式碼對話系統平台- 以聊
故事機器人為例. 碩士論文, 國立中央大學, 2023.
[7] Mohammad Amin Kuhail, Nazik Alturki, Salwa Alramlawi, and Kholood Alhejori. Interacting with educational chatbots: A systematic review. Education and Information Technologies, 28(1):973–1018, 2023.
[8] Liesbeth Kester, Paul A Kirschner, and Jeroen JG Van Merriënboer. The management of cognitive load during complex cognitive skill acquisition by means of computer-simulated problem solving. British journal of educational psychology, 75(1):71–85, 2005.
[9] Yuhao Dan, Zhikai Lei, Yiyang Gu, Yong Li, Jianghao Yin, Jiaju Lin, Linhao Ye, Zhiyan Tie, Yougen Zhou, Yilei Wang, Aimin Zhou, Ze Zhou, Qin Chen, Jie Zhou, Liang He, and Xipeng Qiu. Educhat: A large-scale language model-based chatbot system for intelligent education, 2023.
[10] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022.
[11] Alexandria Katarina Vail and Kristy Elizabeth Boyer. Identifying effective moves in tutoring: On the refinement of dialogue act annotation schemes. In Stefan Trausan-Matu, Kristy Elizabeth Boyer, Martha Crosby, and Kitty Panourgia, editors, Intelligent Tutoring Systems, pages 199–209, Cham,
2014. Springer International Publishing.
[12] Jionghao Lin, Shaveen Singh, Lele Sha, Wei Tan, David Lang, Dragan Gašević, and Guanliang Chen. Is it a good move? mining effective tutoring strategies from human–human tutorial dialogues. Future Generation Computer Systems, 127:194–207, 2022.
[13] Dor Bernsohn, Gil Semo, Yaron Vazana, Gila Hayat, Ben Hagag, Joel Niklaus, Rohit Saha, and Kyryl Truskovskyi. LegalLens: Leveraging LLMs for legal violation identification in unstructured text. In Yvette Graham and Matthew Purver, editors, Proceedings of the 18th Conference of the European
Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2129–2145, St. Julian’s, Malta, March 2024. Association for Computational Linguistics.
[14] Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. Large language models are not fair evaluators. arXiv preprint arXiv:2305.17926, 2023.