跳到主要內容

簡易檢索 / 詳目顯示

回結果列表
研究生： 莊惟婷
Wei-Ting Chuang
論文名稱： 結合AI影像技術與自製末端執行器之餐具分揀系統
指導教授： 王文俊
Wen-June Wang
口試委員:
學位類別： 碩士
Master
系所名稱： 資訊電機學院 - 電機工程學系
Department of Electrical Engineering
論文出版年： 2025
畢業學年度： 114
語文別： 中文
論文頁數： 69
中文關鍵詞： 餐具分揀 、機械手臂 、YOLOv8分割模型 、朝向估測 、物件辨識 、自動化
外文關鍵詞： cutlery sorting, robotic arm, YOLOv8 segmentation model, orientation estimation, object recognition, automation
相關次數： 點閱：15  下載：0 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 本論文旨在設計與實現一套全自動化金屬餐具(刀、叉、湯匙)分揀系統，使用者僅需將餐具倒置於待分類區域，系統可藉由機械手臂控制，自動將所有餐具分揀完成。金屬餐具廣泛應用於餐廳、航空餐飲與大型宴會場域，大量的餐具使用後清洗完畢，往往依靠人工進行分揀收拾作業，不但耗時也耗力。若是利用傳統影像辨識與機械手臂抓取來工作，金屬餐具因具備高反光、顏色相近和隨機堆疊等特性，實務上也面臨相當挑戰。因此，本論文提出結合影像辨識、末端執行器設計與機械手臂控制流程的完整系統架構，以解決現有的高人工成本與以上實務面對的問題。
    本系統涵蓋三大部分：於影像辨識方面，考量到實際場景中的餐具複雜堆疊、角度隨機與光照多變，結合資料擴增技術建立的金屬餐具資料集，訓練出最佳模型進行即時物件辨識與分割；於餐具定位方面，採用主成分分析方法(Principal Component Analysis, PCA)作為朝向角度演算基礎，並建立抓取點估算流程，能有效定位餐具並保持分揀過程的穩定性與成功率；在硬體方面，設計一個具備電磁彈性觸發機制的末端執行器模組，整合電磁鐵、微動開關與彈簧等組件，成功補償深度攝影機於反光表面上的量測誤差，大幅提升操作精度與系統整體效能。
    實驗結果中，YOLOv8x-seg模型為最佳，在遮罩(Mask) mAP0.5-0.95的指標分數為0.865，於測試集的實驗驗證中，餐具朝向分析與抓取點演算法之準確率達99%。在光照條件700 lux實驗場域下，整體系統分揀成功率為96.19%。基於上述分揀策略，本研究亦延伸設計自動化餐具擺盤系統，成功率達90%。
    綜合而言，本論文成功建立一套從影像辨識、抓取到分類與擺盤的完整自動化流程，不僅有效降低人力負擔，亦展現了智慧餐飲自動化的應用潛力，對後續相關研究與實務部署具有重要的參考價值。

    This thesis aims to design and implement a fully automated system for sorting metallic cutlery (knives, forks, and spoons), where users place cutlery in a designated area for the robotic arm to handle automatically. Metallic cutlery is widely used in restaurants, airline catering, and large-scale banquet venues. After extensive washing, the sorting and packing process typically relies on manual labor, which is both time-consuming and labor-intensive. Utilizing traditional image recognition and robotic arm grasping for this task also presents significant challenges due to the high reflectivity, similar colors, and random stacking characteristics of metallic cutlery. Therefore, this study proposes a comprehensive system architecture that integrates image recognition, end-effector design, and robotic arm control, thereby overcoming the problems of high labor costs and insufficient depth sensing accuracy.
    The proposed system consists of three main components. For image recognition, a dedicated dataset of metallic cutlery was established, with data augmentation to account for random orientations, complex stacking, and variable lighting conditions. The dataset was then used to train an optimized model capable of real-time object recognition and segmentation. For cutlery localization, the system used Principal Component Analysis (PCA) to estimate orientation, followed by a grasp-point estimation procedure to ensure accurate localization and stable sorting. Regarding hardware design, we developed an end-effector module with an electromagnetic elastic triggering mechanism. This module integrates electromagnets, micro-switches, and springs to compensate for depth camera measurement errors on reflective surfaces, thereby improving overall precision and system performance.
    Experimental results demonstrated that the YOLOv8x-seg model achieved the best performance, with a mask mAP0.5–0.95 of 0.865. On the test set, the cutlery orientation and grasp-point estimation algorithm achieved 99% accuracy. With lighting below 700 lux, the overall sorting success rate reached 96.19%. Furthermore, based on the proposed sorting strategy, an extended automated cutlery placement system was developed, achieving a 90% success rate. In summary, this thesis establishes a complete automated workflow that covers image recognition, grasping, classification, and placement. The system not only effectively reduces manual labor but also demonstrates the practical potential of intelligent automation in the food service industry.

    摘要 i ABSTRACT ii 致謝 iii 目錄 iv 圖目錄 vi 表目錄 viii 第一章 緒論 1 1.1 研究背景與動機 1 1.2 文獻回顧 1 1.3 論文目標 4 1.4 論文架構 4 第二章 系統架構與軟硬體介紹 6 2.1 系統環境與運作流程簡介 6 2.2 硬體裝置 8 2.2.1 主控電腦 8 2.2.2 立體攝影機 9 2.2.3 機械手臂與末端執行器相關裝置 10 2.2.4 餐具待分類盤與收納盒 13 2.3 系統通訊及軟體 14 2.3.1 系統通訊 14 2.3.2 軟體平台 15 第三章 末端執行器設計與整合 18 3.1 設計原理與目標 18 3.2 機構組成設計 18 3.3 通訊配置與控制邏輯 20 第四章 餐具辨識與定位演算法設計 21 4.1 餐具資料集建立與標註流程 21 4.2 資料擴增與模型訓練 22 4.2.1 幾何旋轉擴增 22 4.2.2 亮度變化擴增 23 4.2.3 模型訓練與設定 23 4.3 餐具朝向分析 24 4.3.1 相機設定與格式轉換 25 4.3.2 分割遮罩與輪廓檢測 26 4.3.3 主軸朝向估計與吸取點推算 27 4.4 特殊場景處理與吸取策略 29 4.4.1 作業區域ROI擷取與處理 30 4.4.2 交錯堆疊餐具的偵測與處理 31 4.4.3 側立式叉子的偵測與處理 31 4.5 角度與座標轉換 32 4.5.1 餐具朝向角度轉換 32 4.5.2 座標轉換 32 第五章 機械手臂控制流程 36 5.1 控制邏輯與流程 36 5.2 交錯餐具撥動處理流程 37 5.3 側立叉子撥動處理流程 38 5.4 正常搬運流程 39 第六章 餐具擺盤系統設計與整合 40 6.1 擺盤系統設計與方法 40 6.1.1 餐盤辨識與擺放點定義 40 6.1.2 餐具辨識與定位 41 6.1.3 最終檢查機制 42 6.2 控制邏輯與動作流程 42 第七章 實驗結果與討論 45 7.1 YOLOv8分割模型結果 45 7.2 餐具朝向分析結果比較 49 7.3 餐具分揀系統之整體準確率 50 7.4 餐具擺盤系統之整體成功率 52 第八章 結論與未來展望 53 8.1 結論 53 8.2 未來展望 53 參考文獻 55

    [1] "財政統計資料庫查詢, " 中華民國財政部, 2025. [Online]. Available: https://web02.mof.gov.tw/njswww/WebMain.aspx?sys=100&funid=defjspf2.
    [2] "免洗餐具限制使用對象及實施方式, " 行政院環保署, 2019. [Online]. Available: https://enews.moenv.gov.tw/page/3b3c62c78849f32f/e39ca2e3-809f-4872-ba90-e30118e366a5.
    [3] D. Zhu, H. Seki, T. Tsuji, and T. Hiramitsu, "Mechanism and Design of Tableware Tidying-up Robot for Self-Service Restaurant," in 2021 IEEE International Conference on Mechatronics and Automation (ICMA), Aug. 2021, pp. 825–830.
    [4] Z. Fu, H. Seki, T. Tsuji, and T. Hiramitsu, "Tableware Recognition for Tidying-up Robot System," in 2024 IEEE International Conference on Mechatronics and Automation (ICMA), Aug. 2024, pp. 626–631.
    [5] 朱飞, "基于图像处理的中餐宴会台面餐具识别与定位研究," 硕士, 湖北工业大学, 2019.
    [6] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You Only Look Once: Unified, Real-Time Object Detection," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016, pp. 779–788.
    [7] R. Varghese and M. S, "YOLOv8: A Novel Object Detection Algorithm with Enhanced Performance and Robustness," in 2024 International Conference on Advances in Data Engineering and Intelligent Computing Systems (ADICS), April 2024, pp. 1–6.
    [8] A. Roudbari, M. Dalvand, N. Pasiar, N. Asadi, M. Yousefi, and M. T. Masouleh, "Autonomous Vision-based Robotic Grasping of Household Objects: A Practical Case Study," in 2023 11th RSI International Conference on Robotics and Mechatronics (ICRoM), Dec. 2023, pp. 918–924.
    [9] Y. Inagaki, R. Araki, T. Yamashita, and H. Fujiyoshi, "Detecting Layered Structures of Partially Occluded Objects for Bin Picking," in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Nov. 2019, pp. 5786–5791.
    [10] H. Yuan, W. Li, and Y. Cheng, "Real-time Detection Algorithm for Occluded Objects Based on Improved YOLOv5s," in 2024 5th International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT), March 2024, pp. 661–666.
    [11] J. Yoon et al., "RGBD Fusion Grasp Network with Large-Scale Tableware Grasp Dataset," in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct. 2023, pp. 2947–2954.
    [12] J. Canny, "A Computational Approach to Edge Detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-8, no. 6, pp. 679–698, 1986.
    [13] A. Maćkiewicz and W. Ratajczak, "Principal Components Analysis (PCA)," Computers & Geosciences, vol. 19, no. 3, pp. 303–342, March 1993.
    [14] X. Wang, Q. Li, K. Liu, K. Zhang, Z. Zhu, and C. Feng, "A Grasping Pose Detection Algorithm for Occluded Objects," in 2023 China Automation Congress (CAC), Nov. 2023, pp. 2287–2292.
    [15] Q. Dai, Y. Zhu, Y. Geng, C. Ruan, J. Zhang, and H. Wang, "GraspNeRF: Multiview-based 6-DoF Grasp Detection for Transparent and Specular Objects Using Generalizable NeRF," in 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023, pp. 1757–1763.
    [16] N. S. Patil and M. D. Jaybhaye, "Autonomous Sorting with 6 DOF Robotic Arm using Machine Vision," in 2023 International Conference on Network, Multimedia and Information Technology (NMITCON), Sept. 2023, pp. 1–6.
    [17] R. Vermelho and L. A. Alexandre, "Grasping and Sorting Cutlery in an Unconstrained Environment with a 6 DoF Robotic Arm and an RGB+D Camera," in 2022 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), April 2022, pp. 3–8.
    [18] L. S. Yim et al., "WFH-VR: Teleoperating a Robot Arm to set a Dining Table across the Globe via Virtual Reality," in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct. 2022, pp. 4927–4934.
    [19] M. S. Choi, D. H. Lee, and D. Pyo, "Development of a Multi-joint gripper for Tableware Grasping," in 2024 24th International Conference on Control, Automation and Systems (ICCAS), 2024, pp. 1597–1598.
    [20] M. G. Selvamuthu and R. Tadakuma, "Development of a Bendable and Extendable Soft Gripper Driven by Differential Worm Gear Mechanism," in 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct. 2024, pp. 2958–2963.
    [21] H. Seong, Y. Hong, H. R. Jung, M. Doh, J. Kim, and H. Moon, "Passive Transformable Fingertip to Augment Tableware Grasp Capability," in 2022 22nd International Conference on Control, Automation and Systems (ICCAS), 2022, pp. 938–943.
    [22] S. W. Kim, C. g. Hwang, S. Yoo, Y. Ko, and S. Kang, "Novel Gripper with Rotatable Distal Joints for Home Robots: Picking and Placing Tableware," in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct. 2023, pp. 4865–4872.
    [23] Y. Fukuzawa, Z. Wang, Y. Mori, and S. Kawamura, "A Robotic System Capable of Recognition, Grasping, and Suction for Dishwashing Automation," in 2021 27th International Conference on Mechatronics and Machine Vision in Practice (M2VIP), Nov. 2021, pp. 369–374.
    [24] "ZED 2 Camera Overview & Datasheet." https://www.mouser.tw/datasheet/3/6026/1/ZED%202%20Datasheet%20v1.1.pdf. (accessed May, 2025)
    [25] Stereolabs. "zed-sdk." https://github.com/stereolabs/zed-sdk (accessed May, 2025).
    [26] UFACTORY. "UFACTORY Lite 6 开发者手册" https://www.cn.ufactory.cc/_files/ugd/896670_411642a7d18b4767b6aa8c3df0eafe0e.pdf. (accessed May, 2025)
    [27] xArm Developer. "xArm-Python-SDK." https://github.com/xArm-Developer/xArm-Python-SDK (accessed May, 2025).
    [28] Ultralytics. "Explore Ultralytics YOLOv8." https://docs.ultralytics.com/zh/models/yolov8/ (accessed May, 2025).
    [29] Kentaro Wada. "LabelMe. "
    https://github.com/LabelMe/labelme (accessed May, 2025).
    [30] Lauren Sell. "LabelImg. "
    https://github.com/HumanSignal/labelImg (accessed May, 2025).
    [31] T.-Y. Lin et al., "Microsoft COCO: Common Objects in Context," CoRR, vol. abs/1405. 2014.
    [32] OpenCV. "Morphological Transformations." https://docs.opencv.org/4.x/d9/d61/tutorial_py_morphological_ops.html (accessed September, 2025).
    [33] D. Marr and E. Hildreth, "Theory of edge detection," Proceedings of the Royal Society of London. Series B. Biological Sciences, vol. 207, no. 1167, pp. 187–217, 1980.
    [34] R. O. Duda and P. E. Hart, "Use of the Hough Transformation to Detect Lines and Curves in Pictures," Commun. ACM, vol. 15, no. 1, pp. 11–15, 1972.
    [35] P. Sturm, "Pinhole Camera Model," in Computer Vision: A Reference Guide, K. Ikeuchi Ed. Boston, MA: Springer US, 2014, pp. 610–613.
    [36] Karen Chan. "刀叉只是基本!西式餐桌禮儀15個NG行為做了就尷尬" https://blog.pinkoi.com/hk/lifestyle/2201-table-manner/ (accessed September, 2025).

    QR CODE
    :::