跳到主要內容

簡易檢索 / 詳目顯示

回結果列表
研究生： 黎哲瑋
Che-Wei Li
論文名稱： 適用於3D虛擬打擊樂器之多感測器融合技術
A multi-sensor fusion technology for 3D virtual percussion instrument
指導教授： 施國琛
Timothy K. Shih
口試委員:
學位類別： 碩士
Master
系所名稱： 資訊電機學院 - 資訊工程學系
Department of Computer Science & Information Engineering
論文出版年： 2016
畢業學年度： 104
語文別： 中文
論文頁數： 61
中文關鍵詞： 虛擬樂器 、打擊樂器 、角速度感測器 、加速度感測器
外文關鍵詞： IMUduino, Percussion Instrument, Groscope
相關次數： 點閱：8  下載：0 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 在這科技日新月異的時代，電腦與人之間已經不再是只有滑鼠鍵
    盤與電腦螢幕的關係了，隨著人機互動的發達，各式各樣的感應器、
    視訊鏡頭、甚至能提供虛擬實境的頭戴式顯示器等等，都更加的讓我
    們能體驗到電腦資訊的進步。微軟的深度攝影機Kinect 便在一開始與
    Xbox 結合，提供了全然不同的遊戲體驗方式。多了深度資訊，除了
    內建的人體骨架追蹤外，我們也能藉此利用在3D 呈像上，著實的為
    工程師們增加了開發各式各樣的應用程式的機會與可能。在Kinect 提
    供的人體骨架追蹤上，利用這些關節的資訊，我們實現了開發虛擬樂
    器的可能，然而，如打擊樂器中，常會利用手指的關節控制鼓棒來擊
    打樂器，但骨架追蹤並無法分辨出細至手指的程度，如此便不能達到
    如真實樂器般的表演感受。故我們便使用了IMUduino 加速度感應器
    黏置於鼓棒上提供即時快速的位置及加速度。藉由利用人體骨架的位
    置來定位擊打位置，以及加速度感應器的加速度以及角速度感測器的
    即時回饋，我們利用此些特性便能簡易的製作出各種打擊樂器，藉由
    改變音源庫的音色以及各擊打位置，便能隨意地轉換演奏樂器。

    People play music for enjoying lives. However, there are always
    some obstacle like the fund issue, space limitation or troubles carrying the instruments around. Human-Computer Interaction has been developed rapidly these years. Keyboards and mouse from the very beginning evolve to all kinds of sensor and cameras. Music instruments made by technology has no longer a fantasy in this decade. This kind of thoughts first came out from gaming product like Wii and XBox. But their purposes are mostly for fun, not for replace the real instruments.
    Bypass the RGB camera which occurs occlusion, infrared cameras
    have more abilities and capability recognizing users. Kinect , produced by Microsoft, contains a RGB camera and a infrared camera along with their own SDK which can track human skeleton and faces. For developer, it is more convenient of having the position of body parts to fulfill the program.
    To fully achieve our project, we adopt Kinect 2 as our devices
    to track skeletons and IMUduino , a smallest and lightest Arduino Leonardo board with motion sensors, to detect a trigger point. With IMUduino on the drumsticks, we can solve the low speed of human joints and the motion blur caused by high-speed hitting. Moreover, user interface are filled with 3D object and user’s body which formed by pointcloud to simulate the real playing environment.

    ontents 頁次 中文摘要. . . . . . . . . . . . . . . . . . . . . . . . . . . . . i Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii Acknoledgment . . . . . . . . . . . . . . . . . . . . . . . . . . iii Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . vii List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Chapter 1、Introduction . . . . . . . . . . . . . . . . . . . . 1 1-1 Background . . . . . . . . . . . . . . . . . . . . 1 1-2 Motivation . . . . . . . . . . . . . . . . . . . . . 3 1-3 Thesis Organization . . . . . . . . . . . . . . . . 3 Chapter 2、Related Works . . . . . . . . . . . . . . . . . . . 4 2-1 Applications by Cameras . . . . . . . . . . . . . 4 2-2 Applications by motion sensor . . . . . . . . . . 6 2-3 Applications by music midi controller . . . . . . 8 2-4 The evaluation of existing method . . . . . . . . 10 Chapter 3、Proposed method . . . . . . . . . . . . . . . . . 12 3-1 Virtual drum system . . . . . . . . . . . . . . . 12 3-1-1 The work flow of system . . . . . . . . . . . . . 12 3-1-2 Environment . . . . . . . . . . . . . . . . . . . . 13 3-1-3 IMUduino and Angular velocity . . . . . . . . . 16 3-1-4 trigger point detection . . . . . . . . . . . . . . 19 3-1-5 Positioning of drum system . . . . . . . . . . . . 20 3-2 Virtual xylophone system . . . . . . . . . . . . . 21 3-2-1 The work flow of system . . . . . . . . . . . . . 21 3-2-2 Environment . . . . . . . . . . . . . . . . . . . . 22 3-2-3 Color space transformation . . . . . . . . . . . . 23 v 3-2-4 Background removal and ROI . . . . . . . . . . 24 3-2-5 Depth-Color information detection . . . . . . . . 25 Chapter 4、Applications . . . . . . . . . . . . . . . . . . . . 29 4-1 Virtual Drum . . . . . . . . . . . . . . . . . . . 29 4-1-1 User Interface . . . . . . . . . . . . . . . . . . . 30 4-1-2 Drum triggering . . . . . . . . . . . . . . . . . . 31 4-2 Virtual Xylophone . . . . . . . . . . . . . . . . 32 4-2-1 User Interface . . . . . . . . . . . . . . . . . . . 32 4-2-2 Discussion . . . . . . . . . . . . . . . . . . . . . 33 4-3 MIDI . . . . . . . . . . . . . . . . . . . . . . . . 35 4-4 VST and Sound Generator . . . . . . . . . . . . 36 Chapter 5、Experimental Results . . . . . . . . . . . . . . . 38 5-1 Environment . . . . . . . . . . . . . . . . . . . . 38 5-2 Experiment . . . . . . . . . . . . . . . . . . . . 38 Chapter 6、Conclusion . . . . . . . . . . . . . . . . . . . . . 42 6-1 Conclusions . . . . . . . . . . . . . . . . . . . . 42 6-2 Future Works . . . . . . . . . . . . . . . . . . . 43 6-2-1 Special Effect . . . . . . . . . . . . . . . . . . . 44 6-2-2 Playing Style . . . . . . . . . . . . . . . . . . . 44 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

    References
    [1] L. Vera, J. Gimeno, I. Coma, M. Fernández, Augmented mirror:
    interactive augmented reality system based on kinect. Human-
    Computer Interaction–INTERACT 2011, 483-486.
    [2] C. Phonpatchara, T. Mekrungroj, T. Matsumaru, M. Fernández,
    Real-time gesture recognition with finger naming by RGB camera
    and IR depth sensor.. IEEE International Conference on Robotics
    and Biomimetics (ROBIO), 2014.
    [3] A. Rosa-Pujazón, I. Barbancho, L. J. Tardón, A. M. Barbancho,
    Drum-hitting gesture recognition and prediction system using kinect.
    I simposio espanol de entrenimiento digital seed , 108-118 , 2013.
    [4] M. H. Hsu, WGCW Kumara, T. K. Shih, Z. Cheng, Spider King:
    Virtual musical instruments based on microsoft Kinect. Awareness
    Science and Technology and Ubi-Media Computing (iCASTUMEDIA),
    2013 International Joint Conference on, 707–713 , 2013.
    [5] C. Levinson, TedStick: A Tangible Electrophonic Drumstick. International
    conference on computer interfaces for musical expression ,
    2012.
    [6] D. Young, I. Fujinaga, Aobachi: A new interface for japanese drumming.
    Proceedings of the 2004 conference on New interfaces for
    musical expression , 23–26 , 2004.
    [7] J. N. Bott, J. G. Crowley, J. J LaViola Jr, Exploring 3D gestural
    interfaces for music creation in video games. Proceedings of the 4th
    international Conference on Foundations of Digital Games, 18-25 ,
    2009.
    46
    [8] C. Kiefer, N. Collins, G. Fitzpatrick, Evaluating the wiimote as a
    musical controller. Proceedings of the 2008 International Computer
    Music Conference, 2008.
    [9] G. Odowichuk, S. Trail, P. Driessen, W. Nie, W. Page, Wyatt, Sensor
    fusion: Towards a fully expressive 3d music control interface.
    Communications, Computers and Signal Processing (PacRim), 2011
    IEEE Pacific Rim Conference on, 836–841 , 2011.
    [10] S. Jordà, G. Geiger, M. Alonso, M. Kaltenbrunner, The reacTable:
    exploring the synergy between live music performance and tabletop
    tangible interfaces. Proceedings of the 1st international conference
    on Tangible and embedded interaction, 139-146 , 2007.
    [11] A. R. Tindale, A. Kapur, G. Tzanetakis, P. Driessen, A. Schloss,
    A comparison of sensor strategies for capturing percussive gestures.
    Proceedings of the 2005 conference on New interfaces for musical
    expression, 200-203 , 2005.
    [12] S. Trail, M. Dean and T. F. Tavares, Non-invasive sensing and
    gesture control for pitched percussion hyper-instruments using the
    Kinect. International conference on computer interfaces for musical
    expression, 2012.

    QR CODE
    :::