| 研究生: |
蔡時富 Shih-Fu Tsai |
|---|---|
| 論文名稱: |
使用大型語言模型進行機器控制指令的自動化生成 Automated Generation of Machine Control Commands Using Large Language Models |
| 指導教授: |
蘇木春
Mu-Chun Su |
| 口試委員: | |
| 學位類別: |
碩士 Master |
| 系所名稱: |
資訊電機學院 - 資訊工程學系 Department of Computer Science & Information Engineering |
| 論文出版年: | 2024 |
| 畢業學年度: | 112 |
| 語文別: | 中文 |
| 論文頁數: | 80 |
| 中文關鍵詞: | 自動程式碼生成 、3D列印 、機器控制 、智慧機器 、大型語言模型 |
| 外文關鍵詞: | Auto Generation, 3D Printing, Machine Control, Smart Machines, Large Language Models |
| 相關次數: | 點閱:13 下載:0 |
| 分享至: |
| 查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
本研究探討了如何透過大型語言模型(Large Language Model, LLM),將自然語言轉換為程式碼來控制機器。研究內容涵蓋背景知識、文獻回顧、研究方法、實驗設計與結果。
在背景知識和文獻回顧部分,首先介紹了大型語言模型的研究現況、智慧機器、人工智慧物聯網的應用場景與3D列印技術的發展現況。接著,回顧了自動程式碼生成在機器控制領域應用、運動學研究與機器人控制與3D列印近年來發展的相關文獻。
研究方法部分描述了硬體設計的流程細節,包括模型設計軟體、檔案輸出格式、3D列印的使用,以及馬達與開發版的介紹。軟體設計流程方面,介紹了運動模擬環境、運動學開發,與大型語言模型以及其官方應用程式介面的使用,最後,系統架構章節詳細介紹了系統架構圖、系統流程圖等整體程式框架。
實驗設計與結果部分包含三個實驗。分別為機械臂的基本控制、機械臂應用於畫圖與機械臂在自動運輸車上的應用,其中展示了機械結構設計圖、函數設計、下達指令的格式、實驗過程縮圖以及最後的實驗成效總結。
而實驗結果顯示,使用大型語言模型生成程式碼來控制機器的方式擁有相當高的準確度,尤其在有較明確的機器函式庫的前提下,更能透過少量的指令輸入,獲得高品質生成效率和準確度。然而,隨著硬體的增多和系統的複雜性增加,也面臨了一些需要克服的機械性失誤。未來的研究將繼續優化系統的穩定性和精確度,進一步提升其應用價值。
This study explores how to use Large Language Models (LLMs) to translate natural language into code to control machines. The research encompasses background knowledge, literature review, research methods, experimental design, and results.
In the background knowledge and literature review section, the current state of research on large language models, intelligent machines, applications of AI in the Internet of Things, and the development status of 3D printing technology are introduced. Next, relevant literature on the application of automatic code generation in the field of machine control, kinematics research, robotic control, and recent developments in 3D printing are reviewed.
The research methods section details the hardware design process, including the model design software, file output formats, the use of 3D printing, and an introduction to motors and development boards. Regarding the software design process, the motion simulation environment, kinematics development, and the use of large language models and their official application programming interfaces are introduced. Finally, the system architecture chapter provides detailed descriptions of the system architecture diagram, system flowchart, and the overall program framework.
The experimental design and results section includes three experiments: basic control of a robotic arm, the application of the robotic arm in drawing, and the application of the robotic arm on an automatic transport vehicle. This section showcases mechanical design diagrams, function design, command formats, experiment process snapshots, and a summary of the final experimental outcomes.
The experimental results indicate that using large language models to generate code to control machines demonstrates considerable accuracy. Especially when clear machine function libraries are available, high-quality generation efficiency and accuracy can be achieved with minimal command input. However, with the increase in hardware and system complexity, some mechanical errors need to be addressed. Future research will continue to optimize system stability and accuracy, further enhancing its application value.
[1] D. Reed, D. Gannon, and J. Dongarra, Reinventing high performance computing: Challenges and opportunities, 2022.
[2] W. X. Zhao, K. Zhou, J. Li, et al., A survey of large language models, 2023.
[3] Y. Liu, T. Han, S. Ma, et al., “Summary of chatgpt-related research and perspective towards the future of large language models,” Meta-Radiology, vol. 1, no. 2, p. 100 017,
Sep. 2023.
[4] H. Touvron, L. Martin, K. Stone, et al., Llama 2: Open foundation and fine-tuned chat
models, 2023.
[5] G. Team, R. Anil, S. Borgeaud, et al., Gemini: A family of highly capable multimodal
models, 2024.
[6] Y. Feng, S. Vanam, M. Cherukupally, W. Zheng, M. Qiu, and H. Chen, “Investigating
code generation performance of chatgpt with crowdsourcing social data,” in 2023 IEEE
47th Annual Computers, Software, and Applications Conference (COMPSAC), 2023,
pp. 876–885.
[7] K. Chowdhary and K. Chowdhary, “Natural language processing,” Fundamentals of artificial intelligence, pp. 603–649, 2020.
[8] Y. Kim, Y. Jernite, D. Sontag, and A. Rush, “Character-aware neural language models,”
in Proceedings of the AAAI conference on artificial intelligence, vol. 30, 2016.
[9] T. Wolf, L. Debut, V. Sanh, et al., “Transformers: State-of-the-art natural language processing,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Q. Liu and D. Schlangen, Eds., Online: Association for Computational Linguistics, Oct. 2020, pp. 38–45.
[10] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, 2019.
[11] C. Cui, Y. Ma, X. Cao, et al., A survey on multimodal large language models for autonomous driving, 2023.
[12] R. Cai, Z. Song, D. Guan, et al., Benchlmm: Benchmarking cross-style visual capability
of large multimodal models, 2023.
[13] J. Zhang and D. Tao, “Empowering things with intelligence: A survey of the progress,
challenges, and opportunities in artificial intelligence of things,” IEEE Internet of Things
Journal, vol. 8, no. 10, pp. 7789–7817, 2021.
[14] I. Ahmed, G. Jeon, and F. Piccialli, “From artificial intelligence to explainable artificial
intelligence in industry 4.0: A survey on what, how, and where,” IEEE Transactions on
Industrial Informatics, vol. 18, no. 8, pp. 5031–5042, 2022.
[15] R. Cupek, J. C.-W. Lin, and J. H. Syu, “Automated guided vehicles challenges for artificial intelligence,” in 2022 IEEE International Conference on Big Data (Big Data), 2022,
pp. 6281–6289.
[16] T. Le Nguyen and T. T. H. Do, “Artificial intelligence in healthcare: A new technology
benefit for both patients and doctors,” in 2019 Portland International Conference on
Management of Engineering and Technology (PICMET), 2019, pp. 1–15.
[17] M. Sarcar, K. M. Rao, and K. L. Narayan, Computer aided design and manufacturing.
PHI Learning Pvt. Ltd., 2008.
[18] C. Elanchezhian and G. S. Sundar, Computer aided manufacturing. Firewall Media,
2007.
[19] G. Thyer, Computer numerical control of machine tools. Elsevier, 2014.
[20] D. Pham and R. Gault, “A comparison of rapid prototyping technologies,” International
Journal of Machine Tools and Manufacture, vol. 38, no. 10, pp. 1257–1287, 1998.
[21] K. V. Wong and A. Hernandez, “A review of additive manufacturing,” International
scholarly research notices, vol. 2012, 2012.
[22] N. Shahrubudin, T. C. Lee, and R. Ramlan, “An overview on 3d printing technology:
Technological, materials, and applications,” Procedia Manufacturing, vol. 35, pp. 1286–
1296, 2019.
[23] I. Hager, A. Golonka, and R. Putanowicz, “3d printing of buildings and building components as the future of sustainable construction?” Procedia Engineering, vol. 151, pp. 292–
299, 2016.
[24] S. K. Moon, Y. E. Tan, J. Hwang, and Y.-J. Yoon, “Application of 3d printing technology
for designing light-weight unmanned aerial vehicle wing structures,” International Journal of Precision Engineering and Manufacturing-Green Technology, vol. 1, pp. 223–228,
2014.
[25] M. Chinthavali, “3d printing technology for automotive applications,” in 2016 International Symposium on 3D Power Electronics Integration and Manufacturing (3D-PEIM),
IEEE, 2016, pp. 1–13.
[26] T. G. Papaioannou, D. Manolesou, E. Dimakakos, G. Tsoucalas, M. Vavuranakis, and
D. Tousoulis, “3d bioprinting methods and techniques: Applications on artificial blood
vessel fabrication,” Acta Cardiologica Sinica, vol. 35, no. 3, p. 284, 2019.
[27] Z. Liu, M. Zhang, B. Bhandari, and Y. Wang, “3d printing: Printing precision and application in food sector,” Trends in Food Science & Technology, vol. 69, pp. 83–94, 2017.
[28] P. Vaithilingam, T. Zhang, and E. L. Glassman, “Expectation vs. experience: Evaluating the usability of code generation tools powered by large language models,” in Chi
conference on human factors in computing systems extended abstracts, 2022, pp. 1–7.
[29] S. Vemprala, R. Bonatti, A. Bucker, and A. Kapoor, Chatgpt for robotics: Design principles and model abilities, 2023.
[30] J. Liang, W. Huang, F. Xia, et al., “Code as policies: Language model programs for
embodied control,” in 2023 IEEE International Conference on Robotics and Automation
(ICRA), 2023, pp. 9493–9500.
[31] S. Kucuk and Z. Bingul, Robot kinematics: Forward and inverse kinematics. INTECH
Open Access Publisher London, UK, 2006.
[32] A. K. Singh, N. Baranwal, and G. C. Nandi, “Development of a self reliant humanoid
robot for sketch drawing,” Multimedia Tools and Applications, vol. 76, pp. 18 847–18 870,
2017.
[33] R. Siemasz, K. Tomczuk, and Z. Malecha, “3d printed robotic arm with elements of
artificial intelligence,” Procedia Computer Science, vol. 176, pp. 3741–3750, 2020.
[34] J. Z. Gul, M. Sajid, M. M. Rehman, et al., “3d printing for soft robotics–a review,” Science and technology of advanced materials, vol. 19, no. 1, pp. 243–262, 2018.
[35] X. Zhang, M. Li, J. H. Lim, et al., “Large-scale 3d printing by a team of mobile robots,”
Automation in Construction, vol. 95, pp. 98–106, 2018.
[36] autodesk. “Autodesk-fusion-360.” (2024), [Online]. Available: https://www.autodesk.
com.tw/ (visited on 06/07/2024).
[37] 3dsystems. “Stl-formate.” (2024), [Online]. Available: https:// www. 3dsystems. com/
(visited on 06/07/2024).
[38] creality. “Creality-k1-max-3d-printer.” (2024), [Online]. Available: https://www.creality.
com/ (visited on 06/07/2024).
[39] T. Pro. “Sg90 servo motor.” (2024), [Online]. Available: https://www.servodatabase.
com/servo/towerpro/sg90 (visited on 06/07/2024).
[40] T. Pro. “Mg90s servo motor.” (2024), [Online]. Available: https://www.servodatabase.
com/servo/towerpro/mg90s (visited on 06/07/2024).
[41] T. Pro. “Mg996r servo motor.” (2024), [Online]. Available: https://www.servodatabase.
com/servo/towerpro/mg996r (visited on 06/07/2024).
[42] R. P. Foundation. “Raspberry pi 4 model b.” (2024), [Online]. Available: https://www.
raspberrypi.com/products/raspberry-pi-4-model-b/ (visited on 06/07/2024).
[43] E. Systems. “Esp32-s3-devkitc-1.” (2024), [Online]. Available: https://www.espressif.
com/en/products/devkits/esp32-s3-devkitc-1/overview (visited on 06/07/2024).
[44] E. Systems. “Esp32-devkitc.” (2024), [Online]. Available: https://www.espressif.com/
en/products/devkits/esp32-devkitc/overview (visited on 06/07/2024).
[45] STMicroelectronics. “L298n motor driver module.” (2024), [Online]. Available: https:
//www.st.com/en/motor-drivers/l298.html (visited on 06/07/2024).
[46] N. Semiconductors. “Pca9685 16-channel 12-bit pwm servo driver.” (2024), [Online].
Available: https://www.nxp.com/products/interfaces/ic - bus/ic - led - controllers/16 -
channel-12-bit-pwm-fm-plus-ic-bus-led-controller:PCA9685 (visited on 06/07/2024).