傳統的學習方法大多依賴於紙本教材，並由老師口頭教導，形成了老師主導教學，學生在教室內被動接受知識。然而在技能密集型的學科如體育運動、餐旅服務等實作學科中，學習者需要透過自身的多種感官及肢體行為進行知識的學習與實際應用，並於學習後對其多模態的行為進行反思。
為了能使學習者能在真實的情境中，透過自身多種具身行為進行學習及反思，本研究設計了一款多模態的具身情境學習及反思回顧系統，並將其應用於數位同步情境中，透過立體視覺相機及麥克風將學習者各種不同的感官表現擷取並儲存下來，使學習者能透過自身多模態的具身行為作為互動基礎，控制虛擬化身進入虛擬情境中進行學習。學習過程中，系統能透過多模態資料對學習者的具身行為進行即時評量並給予相對應的多模態學習鷹架及系統回饋，且學習者能透過觀看多模態的學習紀錄進行反思學習，使學習者在不提升認知負荷的情況下，提升其學習成效、學習動機、反身性思考。
為了驗證此系統及教學方法是否能提升學習相關面向，本研究進一步將系統應用於實際課程中。研究結果顯示，本研究之系統在不提升認知負荷的情況下，可以提升學習者的學習成效，且學習動機及反身性思考方面也有正向的影響。

Traditional learning methods largely rely on printed materials, with teachers delivering knowledge orally, leading to teacher-dominated instruction and passive knowledge reception by students in the classroom. However, in skill-intensive disciplines such as physical education and hospitality services, learners need to engage multiple senses and bodily actions to learn and apply knowledge practically. Furthermore, learners are required to reflect on their multimodal behaviors after learning.
To enable learners to engage in learning and reflection through their own embodied behaviors in scenarios, this study has designed a multimodal embodied situated learning and reflection system. Applied in a digital synchronous setting, the system captures and stores the learners' various sensory performances using stereo cameras and microphones. This allows learners to interact based on their multimodal embodied behaviors and control a avatar to enter virtual scenarios for learning. During the learning process, multimodal data analysis allows for real-time assessment of learners' embodied behaviors, providing corresponding multimodal learning scaffolds and system feedback. Learners can engage in reflective learning by reviewing their multimodal learning records. With this system, learners can enhance their learning outcomes, motivation, and reflective thinking without increasing cognitive load.
To validate whether this system can enhance related learning aspects, the system was further applied in actual courses. The research results indicate that this system can enhance learners' learning outcomes without increasing cognitive load, and also positively affects their learning motivation and reflective thinking.

黃招憲、郭德信、王淑麗 (2005)。餐旅日語(上)。致良出版社。
黃招憲、許惠端、王靖絜 (2010)。餐旅日語(下)。致良出版社。
吳建宇(2023)。以學生具身沉浸與具身辨識達成具身認知學習以提升學習成效。國立中央大學資訊工程學系學位論文。
唐浩(2024)。使用者自定義之數位情境學習系統。國立中央大學資訊工程學系學位論文。
Akers, J., Zimmermann, J., Trutoiu, L., Schowengerdt, B., & Kemelmacher-Shlizerman, I. (2020, October). Mixed reality spatial computing in a remote learning classroom. In Proceedings of the 2020 ACM Symposium on Spatial User Interaction (pp. 1-3).
Ambady, N., & Rosenthal, R. (1993). Half a minute: Predicting teacher evaluations from thin slices of nonverbal behavior and physical attractiveness. Journal of personality and social psychology, 64(3), 431.
Anderson, J. R., Reder, L. M., & Simon, H. A. (1996). Situated learning and education. Educational researcher, 25(4), 5-11.
Baltrušaitis, T., Ahuja, C., & Morency, L. P. (2018). Multimodal machine learning: A survey and taxonomy. IEEE transactions on pattern analysis and machine intelligence, 41(2), 423-443.
Bandura, A. (1986). Social foundations of thought and action. Englewood Cliffs, NJ, 1986(23-28), 2.
Barsalou, L. W. (1999). Perceptual symbol systems. Behavioral and brain sciences, 22(4), 577-660.
Barsalou, L. W., Santos, A., Simmons, W. K., & Wilson, C. D. (2008). Language and simulation in conceptual processing. Symbols, embodiment, and meaning, 245-283.
Belk, R. W. (2013). Extended self in a digital world. Journal of consumer research, 40(3), 477-500.
Blikstein, P., & Worsley, M. (2016). Multimodal learning analytics and education data mining: Using computational technologies to measure complex learning tasks. Journal of Learning Analytics, 3(2), 220-238.
Boud, D., Keogh, R., & Walker, D. (2013). Reflection: Turning experience into learning. Routledge.
Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. 1989, 18(1), 32-42.
Chang, E., Kim, H. T., & Yoo, B. (2020). Virtual reality sickness: a review of causes and measurements. International Journal of Human–Computer Interaction, 36(17), 1658-1682
Chen, J. C. (2016). The crossroads of English language learners, task-based instruction, and 3D multi-user virtual learning in Second Life. Computers & Education, 102, 152-171.
Chen, L., Feng, G., Leong, C. W., Joe, J., Kitchen, C., & Lee, C. M. (2016). Designing an automated assessment of public speaking skills using multimodal cues. Journal of Learning Analytics, 3(2), 261-281.
Chew, S. W., Jhu, J. Y., & Chen, N. S. (2018, July). The effect of learning English idioms using scaffolding strategy through situated learning supported by augmented reality. In 2018 IEEE 18th international conference on advanced learning technologies (ICALT) (pp. 390-394). IEEE.
Chiang, P. Y., Lee, C. H., Lin, S. H., Fan, C. Y., Chen, Y. H., & Chen, G. D. (2018, June). A Situational Comic Play Drama Learning System to Enhance Students’ Learning Motivation. In EdMedia+ Innovate Learning (pp. 1269-1274). Association for the Advancement of Computing in Education (AACE).
Cohen, J. (1992). Statistical power analysis. Current directions in psychological science, 1(3), 98-101.
Dalsgaard, C. (2020). Reflective Mediation: Toward a Sociocultural Conception of Situated Reflection. Frontline Learning Research, 8(1), 1-13.
Danish, J. A., Enyedy, N., Saleh, A., Lee, C., & Andrade, A. (2015). Science through technology enhanced play: Designing to support reflection through play and embodiment. International Society of the Learning Sciences, Inc.[ISLS]..
Davis, B., Tu, X., Georgen, C., Danish, J. A., & Enyedy, N. (2019). The impact of different play activity designs on students’ embodied learning. Information and Learning Sciences, 120(9/10), 611-639.
Dewey, J. (1933). How we think, revised edition. Boston: D.C. Heath.
Dickey, M. D. (2003). Teaching in 3D: Pedagogical affordances and constraints of 3D virtual worlds for synchronous distance learning. Distance education, 24(1), 105-121.
Dourish, P. (2001). Where the action is: the foundations of embodied interaction. MIT press.
Edelson, D. C., Gordin, D. N., & Pea, R. D. (1999). Addressing the challenges of inquiry-based learning through technology and curriculum design. Journal of the learning sciences, 8(3-4), 391-450.
Eraut, M. (1994). Developing professional knowledge and competence. Psychology Press.
Farhadloo, M., Sharma, A., Shekhar, S., & Markovic, S. N. (2023). Spatial computing opportunities in biomedical decision support: The atlas-ehr vision. arXiv preprint arXiv:2305.09675.
Farrah, M. (2012). Reflective journal writing as an effective technique in the writing process. An-Najah University Journal for Research-B (Humanities), 26(4), 997-1025.
Fleming, M., Merrell, C., & Tymms, P. (2004). The impact of drama on pupils' language, mathematics, and attitude in two primary schools. Research in drama education, 9(2), 177-197.
Fugate, J. M., Macrine, S. L., & Cipriano, C. (2019). The role of embodied cognition for transforming learning. International Journal of School & Educational Psychology, 7(4), 274-288.
Fuste, A., Reynolds, B., Hobin, J., & Heun, V. (2020, April). Kinetic ar: A framework for robotic motion systems in spatial computing. In Extended abstracts of the 2020 CHI conference on human factors in computing systems (pp. 1-8).
Genome, S. (2020). The Global Startup Ecosystem Report 2020 (GSER2020). Relatório anual. Disponível em:< https://startupgenome. com/reports/gser2020>. Acesso em, 7.
George, D. (2011). SPSS for windows step by step: A simple study guide and reference, 17.0 update, 10/e. Pearson Education India.
George-Palilonis, J., & Filak, V. (2009). Blended Learning in the Visual Communications Classroom: Student Reflections on a Multimedia Course. Electronic journal of e-learning, 7(3), 247-256.
Gerrish, K. (1993). An evaluation of a portfolio as an assessment tool for teaching practice placements. Nurse Education Today, 13(3), 172-179.
Gibson, J. (1985). The ecological approach to visual perception. 生態学的視覚論.
Greenwold, S. (2003). Spatial computing. Massachusetts Institute of Technology, Master.
Halliday, M. A. K. (1978). Language as social semiotic: The social interpretation of language and meaning. (No Title).
Herrington, J., & Oliver, R. (2000). An instructional design framework for authentic learning environments. Educational technology research and development, 48(3), 23-48.
Hodge, R. I. V., & Kress, G. R. (1988). Social semiotics. Cornell University Press.
Hsia, L. H., Hwang, G. J., & Lin, C. J. (2022). A WSQ-based flipped learning approach to improving students’ dance performance through reflection and effort promotion. Interactive Learning Environments, 30(2), 229-244.
Hussain, M. S., Monkaresi, H., & Calvo, R. A. (2012). Categorical vs. dimensional representations in multimodal affect detection during learning. In Intelligent Tutoring Systems: 11th International Conference, ITS 2012, Chania, Crete, Greece, June 14-18, 2012. Proceedings 11 (pp. 78-83). Springer Berlin Heidelberg.
Hwang, W. Y., Shih, T. K., Ma, Z. H., Shadiev, R., & Chen, S. Y. (2016). Evaluating listening and speaking skills in a mobile game-based learning environment with situational contexts. Computer Assisted Language Learning, 29(4), 639-657.
Inkpen, K. M., & Sedlins, M. (2011, March). Me and my avatar: exploring users' comfort with avatars for workplace communication. In Proceedings of the ACM 2011 conference on Computer supported cooperative work (pp. 383-386).
Jewitt, C. (Ed.). (2009). The Routledge handbook of multimodal analysis (Vol. 1). London: Routledge.
Johnson-Glenberg, M. C., Birchfield, D. A., Tolentino, L., & Koziupa, T. (2014). Collaborative embodied learning in mixed reality motion-capture environments: Two science studies. Journal of educational psychology, 106(1), 86.
Karpov, A. A., & Yusupov, R. M. (2018). Multimodal interfaces of human–computer interaction. Herald of the Russian Academy of Sciences, 88, 67-74.
Kayes, D. C. (2017). The limits and consequences of experience absent reflection: Implications for learning and organizing. In Organizing reflection (pp. 79-94). Routledge.
Kember, D., Leung, D. Y., Jones, A., Loke, A. Y., McKay, J., Sinclair, K., ... & Yeung, E. (2000). Development of a questionnaire to measure the level of reflective thinking. Assessment & evaluation in higher education, 25(4), 381-395.
Kiefer, M., & Trumpp, N. M. (2012). Embodiment theory and education: The foundations of cognition in perception and action. Trends in Neuroscience and Education, 1(1), 15-20.
Kolb, A. Y., & Kolb, D. A. (2005). Learning styles and learning spaces: Enhancing experiential learning in higher education. Academy of management learning & education, 4(2), 193-212.
Kress, G. (2003). Literacy in the new media age. routledge.
Kress, G., & Van Leeuwen, T. (2020). Reading images: The grammar of visual design. Routledge.
Leppink, J., Paas, F., Van der Vleuten, C. P., Van Gog, T., & Van Merriënboer, J. J. (2013). Development of an instrument for measuring different types of cognitive load. Behavior research methods, 45, 1058-1072.
Mayer, R. E. (Ed.). (2005). The Cambridge handbook of multimedia learning. Cambridge university press.
Mohamed, K. S. (2023). Deep learning for spatial computing: augmented reality and metaverse “the Digital Universe”. In Deep Learning-Powered Technologies: Autonomous Driving, Artificial Intelligence of Things (AIoT), Augmented Reality, 5G Communications and Beyond (pp. 131-150). Cham: Springer Nature Switzerland.
Niedermeyer, F. C., & Oliver, L. (1972). The development of young children's dramatic and public speaking skills. The Elementary School Journal, 73(2), 95-100.
Ochoa, X., & Worsley, M. (2016). Augmenting learning analytics with multimodal sensory data. Journal of Learning Analytics, 3(2), 213-219.
O'Toole, J., Stinson, M., & Moore, T. (2009). Drama and curriculum: A giant at the door (Vol. 6). Springer Science & Business Media.
Oviatt, S. (1999). Ten myths of multimodal interaction. Communications of the ACM, 42(11), 74-81.
Quek, F., McNeill, D., Bryll, R., Duncan, S., Ma, X. F., Kirbas, C., ... & Ansari, R. (2002). Multimodal human discourse: gesture and speech. ACM Transactions on Computer-Human Interaction (TOCHI), 9(3), 171-193.
Palincsar, A. S. (1998). Keeping the metaphor of scaffolding fresh—-a response to C. Addison Stone's “The metaphor of scaffolding: Its utility for the field of learning disabilities”. Journal of learning disabilities, 31(4), 370-373.
Pijeira-Díaz, H. J., Drachsler, H., Järvelä, S., & Kirschner, P. A. (2016, April). Investigating collaborative learning success with physiological coupling indices based on electrodermal activity. In Proceedings of the sixth international conference on learning analytics & knowledge (pp. 64-73).
Pintrich, P., Smith, D., Garcia, T., & McKeachie, W. (1991). A manual for the use of the motivated strategies for learning Questionnaire (MSLQ)
Plopski, A., Ienaga, N., & Sugimoto, M. (2023). Tracking systems: Calibration, hardware, and peripherals. In Springer Handbook of Augmented Reality (pp. 211-238). Cham: Springer International Publishing.
Puntambekar, S., & Kolodner, J. L. (2003). Distributed scaffolding: helping students learn science from design. Cogn Inst.
Schön, D. (1983). The reflective practitioner. London, UK: Temple Smith.
Sediyani, T., & Hadi, E. (2017). Integration of Audio Visual Multimedia for Special Education Pre-Service Teachers' Self Reflections in Developing Teaching Competencies. Journal of Education and Practice, 8(6), 106-112.
Spurgeon, S., & Bowen, J. L. (2002). Digital Video/Multimedia Portfolios as a Tool To Develop Reflective Teacher Candidates.
Spurgeon, S., & Bowen, J. L. (2002, December). Promoting reflective thinking in teacher candidates: are digital video/multimedia portfolios a useful tool?. In International Conference on Computers in Education, 2002. Proceedings. (pp. 1427-1428). IEEE.
Sharma, K., & Giannakos, M. (2020). Multimodal data capabilities for learning: What can multimodal data tell us about learning?. British Journal of Educational Technology, 51(5), 1450-1484.
Sharma, R., Pavlovic, V. I., & Huang, T. S. (1998). Toward multimodal human-computer interface. Proceedings of the IEEE, 86(5), 853-869.
Shin, D. (2018). Empathy and embodied experience in virtual environment: To what extent can virtual reality stimulate empathy and embodied experience?. Computers in human behavior, 78, 64-73.
Steed, A., Pan, Y., Zisch, F., & Steptoe, W. (2016, March). The impact of a self-avatar on cognitive load in immersive virtual reality. In 2016 IEEE virtual reality (VR) (pp. 67-76). IEEE.
Stolz, S. A. (2015). Embodied learning. Educational philosophy and theory, 47(5), 474-487.
Sung, H. Y., & Hwang, G. J. (2013). A collaborative game-based learning approach to improving students' learning performance in science courses. Computers & education, 63, 43-51.
Turk, M., & Robertson, G. (2000). Perceptual user interfaces (introduction). Communications of the ACM, 43(3), 32-34.
Waisberg, E., Ong, J., Masalkhi, M., Zaman, N., Sarker, P., Lee, A. G., & Tavakkoli, A. (2024). Apple Vision Pro and why extended reality will revolutionize the future of medicine. Irish Journal of Medical Science (1971-), 193(1), 531-532.
Walker, R. (2002). Case study, case records and multimedia. Cambridge journal of Education, 32(1), 109-127.
Wald, H. S., Davis, S. W., Reis, S. P., Monroe, A. D., & Borkan, J. M. (2009). Reflecting on reflections: enhancement of medical education curriculum with structured field notes and guided feedback. Academic Medicine, 84(7), 830-837.
Walqui, A. (2006). Scaffolding instruction for English language learners: A conceptual framework. International journal of bilingual education and bilingualism, 9(2), 159-180.
Wilson, M. (2002). Six views of embodied cognition. Psychonomic bulletin & review, 9, 625-636.
Wood, D., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem solving. Journal of child psychology and psychiatry, 17(2), 89-100.
Worsley, M. (2018, March). (Dis) engagement matters: Identifying efficacious learning practices with multimodal learning analytics. In Proceedings of the 8th international conference on learning analytics and knowledge (pp. 365-369).
Wu, W. Y., Luo, Y. F., Huang, D. Y., Huang, C. W., Peng, Y. I., & Chen, G. D. (2015). A Self-Observable Learning Cinema in the Classroom. In The 23rd International Conference on Computers in Education (pp. 257-262).
Yang, F. Y., Chang, C. Y., Chien, W. R., Chien, Y. T., & Tseng, Y. H. (2013). Tracking learners' visual attention during a multimedia presentation in a real classroom. Computers & Education, 62, 208-220.
Yee, N., & Bailenson, J. (2007). The Proteus effect: The effect of transformed self-representation on behavior. Human communication research, 33(3), 271-290.
Yee, N., Bailenson, J. N., & Ducheneaut, N. (2009). The Proteus effect: Implications of transformed digital self-representation on online and offline behavior. Communication Research, 36(2), 285-312.
Yildiz, E., Møller, C., & Bilberg, A. (2020). Virtual factory: digital twin based integrated factory simulations. Procedia CIRP, 93, 216-221.
Zhao, S. (2005). The digital self: Through the looking glass of telecopresent others. Symbolic interaction, 28(3), 387-405.