問答系統在自然語言處理中越來越流行，尤其是在智慧工廠的環境中。在這項研究中，我們讓使用者經歷一個多步驟的機器人組裝任務來模擬工廠環境。我們設計了一個互動式對話系統，通過對使用者在組裝過程中的意圖進行分類來處理使用者的問題。為了解决使用者在組裝過程中，相同的問題在不同的步驟中應給出不同的回答，我們的系統將使用者的問題與使用者當前影像相結合，以更好地掌握使用者的問題並分析他們的意圖。此外，我們引入(a) 對話行為分類和(b) 判別當前問句使否與當前步驟相關兩項相任務作為輔助任務來支持我們的使用者意圖分類。透過多模態和多任務學習，提高了使用者意圖分
類的準確率。

Question answering systems are becoming increasingly popular in Natural Language Processing, especially when applied in smart factory settings. In this study, we simulated the manufacturing setting by having user go through a multiple stage robot assembling task. We designed an interactive dialogue system to deal with user question by classifying their intent during the assembly process. To tackle the issue that users required different answers for questions sharing the same intent since they are present in different stages of the work process, our system incorporates user utterances with real time video feed to better situate user questions and analyze their intent. Also, we introduce dialogue act classification and step-independent
classification as auxiliary tasks to support our user intent classification. With the combination of multimodality and multitask learning, our proposed model enhances the performance of intent classification.

[1] F. Meziane, S. Vadera, K. Kobbacy, and N. Proudlove, “Intelligent systems in man- ufacturing: current developments and future prospects,” Integrated Manufacturing Systems, 2000.
[2] X. Yao, J. Zhou, J. Zhang, and C. R. Boër, “From intelligent manufacturing to smart manufacturing for industry 4.0 driven by next generation artificial intelligence and further on,” in 2017 5th international conference on enterprise systems (ES). IEEE, 2017, pp. 311–318.
[3] L. Lattanzi, C. Cristalli, D. Massa, S. Boria, P. Lépine, and M. Pellicciari, “Geomet- rical calibration of a 6-axis robotic arm for high accuracy manufacturing task,” The International Journal of Advanced Manufacturing Technology, vol. 111, no. 7, pp. 1813–1829, 2020.
[4] P. R. Jeyaraj and E. R. S. Nadar, “Computer vision for automatic detection and classi- fication of fabric defect employing deep learning algorithm,” International Journal of Clothing Science and Technology, 2019.
[5] O. Badmos, A. Kopp, T. Bernthaler, and G. Schneider, “Image-based defect detection in lithium-ion battery electrode using convolutional neural networks,” Journal of
Intelligent Manufacturing, vol. 31, no. 4, pp. 885–897, 2020.
[6] D. Griol and Z. Callejas, “An architecture to develop multimodal educative appli- cations with chatbots,” International Journal of Advanced Robotic Systems, vol. 10, no. 3, p. 175, 2013.
[7] A. Dersingh, P. Srisakulpinyo, S. Rakkarn, and P. Boonkanit, “Chatbot and visual management in production process,” , pp. 274–277, 2017.
[8] T.-Y. Chen, “Improve user intent classification performance using ymcl model,” Master’s thesis, National Central University, 2020.
[9] H. Chen, X. Liu, D. Yin, and J. Tang, “A survey on dialogue systems: Recent advances and new frontiers,” Acm Sigkdd Explorations Newsletter, vol. 19, no. 2, pp. 25–35, 2017.
[10] E. Hosseini-Asl, B. McCann, C.-S. Wu, S. Yavuz, and R. Socher, “A simple language model for task-oriented dialogue,” arXiv preprint arXiv:2005.00796, 2020.
[11] B. Peng, C. Li, J. Li, S. Shayandeh, L. Liden, and J. Gao, “Soloist: Few-shot task-oriented dialog with a single pretrained auto-regressive model,” arXiv preprint arXiv:2005.05298, 2020.
[12] D. Adiwardana, M.-T. Luong, D. R. So, J. Hall, N. Fiedel, R. Thoppilan, Z. Yang, A. Kulshreshtha, G. Nemade, Y. Lu et al., “Towards a human-like open-domain chatbot,” arXiv preprint arXiv:2001.09977, 2020.
[13] S. Roller, E. Dinan, N. Goyal, D. Ju, M. Williamson, Y. Liu, J. Xu, M. Ott, K. Shuster, E. M. Smith et al., “Recipes for building an open-domain chatbot,” arXiv preprint arXiv:2004.13637, 2020.
[14] N. Reithinger and M. Klesen, “Dialogue act classification using language models,” in Fifth European Conference on Speech Communication and Technology, 1997.
[15] A. Stolcke, E. Shriberg, R. Bates, N. Coccaro, D. Jurafsky, R. Martin, M. Meteer, K. Ries, P. Taylor, C. Van Ess-Dykema et al., “Dialog act modeling for conversa- tional speech,” in AAAI Spring Symposium on Applying Machine Learning to Dis­ course Processing, 1998, pp. 98–105.
[16] A. Stolcke, K. Ries, N. Coccaro, E. Shriberg, R. Bates, D. Jurafsky, P. Taylor, R. Martin, C. V. Ess-Dykema, and M. Meteer, “Dialogue act modeling for auto- matic tagging and recognition of conversational speech,” Computational linguistics, vol. 26, no. 3, pp. 339–373, 2000.
[17] R. Fernandez and R. W. Picard, “Dialog act classification from prosodic features using support vector machines,” in Speech Prosody 2002, International Conference, 2002.
[18] S. Grau, E. Sanchis, M. J. Castro, and D. Vilar, “Dialogue act classification using a bayesian approach,” in 9th Conference Speech and Computer, 2004.
[19] R. Li, C. Lin, M. Collinson, X. Li, and G. Chen, “A dual-attention hierar- chical recurrent neural network for dialogue act classification,” arXiv preprint arXiv:1810.09154, 2018.
[20] T. Saha, S. Srivastava, M. Firdaus, S. Saha, A. Ekbal, and P. Bhattacharyya, “Ex- ploring machine learning and deep learning frameworks for task-oriented dialogue act classification,” in 2019 International Joint Conference on Neural Networks (IJCNN). IEEE, 2019, pp. 1–8.
[21] T. Saha, A. Patra, S. Saha, and P. Bhattacharyya, “Towards emotion-aided multi- modal dialogue act classification,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 4361–4372.
[22] N. Dahlbäck, A. Jönsson, and L. Ahrenberg, “Wizard of oz studies—why and how,” Knowledge­based systems, vol. 6, no. 4, pp. 258–266, 1993.
[23] S. Paul, R. Goel, and D. Hakkani-Tür, “Towards universal dialogue act tagging for task-oriented dialogues,” arXiv preprint arXiv:1907.03020, 2019.
[24] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.