基於多個Kinect的虛擬球體樂器｜國立中央大學博碩士論文系統

簡易檢索 / 詳目顯示

回結果列表

研究生：	呂名峰 Ming-Fong Lu
論文名稱：	基於多個Kinect的虛擬球體樂器 3D Sphere Virtual Instrument with multiple Kinect
指導教授：	施國琛 Timothy K. Shih
口試委員:
學位類別：	碩士 Master
系所名稱：	資訊電機學院 - 資訊工程學系 Department of Computer Science & Information Engineering
論文出版年：	2015
畢業學年度：	103
語文別：	英文
論文頁數：	91
中文關鍵詞：	人機互動、Kinect 、MIDI 、虛擬樂器、OpenGL 、感應器校正、點雲、點雲對齊
外文關鍵詞：	HCI, Kinect, MIDI, Virtual Instument, OpenGL, Kinect calibration, PointCloud, ICP
相關次數：	點閱：13 下載：0
分享至:	分享至facebook 分享至twitter

查詢本校圖書館目錄查詢臺灣博碩士論文知識加值系統勘誤回報

人機互動的領域在現代被廣泛地運用，在這篇論文中我們設計一個虛擬樂器，主要是使用微軟公司的KINECT感應器，能夠偵測人體的骨架來操作音樂，KINECT本身有限制人體面對KINECT的角度而導致側面無法被偵測，本論文使用骨架當作點雲，並使用三台KINECT來對齊骨架而達到側面也可以被偵測。在樂器中我們使用一顆球作為介面，介面中的球有12個表面並可以彈奏單音或和弦，為了達到12的表面都可以被彈奏，我們使用ICP演算法來對齊三台KINECT讓KINECT側面可以被偵測到，來達到使用者可以盡情的彈奏樂器而不會受到正面彈奏的限制。在介面的部份我們使用OpenGL來做為繪圖的基礎，論文中會介紹基礎的三維觀念和OpenGL基礎，並理解如何創造一顆虛擬球體且放入人體的骨架當作樂器，並介紹MIDI相關的知識與如何使用MIDI來彈奏音樂，最後實驗的部分會顯示出骨架對其的準確度與顯示骨架對齊的圖示。

Human Computer Interaction is wildly used in our life, in this paper we design a virtual instrument by the Microsoft Kinect sensor. Kinect sensor can detect human body by representing a skeleton, and the program use the skeleton to control the music signal. Kinect has human body angle limits, if user’s side face to Kinect then Kinect cannot detect skeleton, this paper uses skeletons as point cloud and calibrate three Kinect to achieve side face detection. In our instrument we use a sphere for interface, inside the sphere has twelve blocks and the user can play a single note or chords. In order to play all of the blocks, we use the ICP algorithm to calibrate three Kinect and the users can the play instrument without side face limits. At program interface we use OpenGL for rendering the objects, and we will introduce the basic three dimensional concepts and OpenGL concept, explains how to put the skeleton inside the sphere and introduce MIDI concept to use MIDI for playing music. Finally the experiment result shows the accuracy of the skeleton in calibration and show the status of skeleton in figures.

摘要    vi
Abstract    vii
Acknowledgements    viii
Contents    ix
List of Figures    xi
List of Tables    xv
Chapter 1.    Introduction    1
Chapter 2.    Related work    3
2.1    3D Concept    5
1.    3D effect    6
2.    Coordinate systems    6
3.    Cartesian coordinates    7
4.    3D Cartesian Coordinates    7
5.    Projection    8
6.    Orthographic projection    9
7.    Perspective projection    9
8.    Coordinate Clipping    9
9.    Viewport    10
2.2    Kinect    11
1.    Detect user    13
2.    Kinect SDK    13
3.    Kinect skeleton    16
2.3    OPENGL    17
1.    Vertex basic    17
2.    Pipeline    18
3.    OpenGL API    18
4.    GLUT    19
5.    OPENGL TRIANGLE EXAMPLE    22
6.    Opengl translate    23
7.    Rotated    24
2.4    MIDI    25
2.5    ICP algorithm    31
1.    ICP  Principle    32
Chapter 3.    Proposed method    34
3.1    Creation of virtual space    34
3.2    Two Kinect Calibration    43
1.    The old method    43
2.    The new method    47
Chapter 4.    System architecture    56
Chapter 5.    Experiment result and Analysis    61
Experiment in 30 degrees with 6 samples    67
Chapter 6.    Conclusion and Future works    72
1.    Future work    72
2.    Conclusion    73
References    74


                                

[1] D. Jung, S. Laing, M. H. Jensen, P. Hunkin, A. Löf, and N. Tims, Requirements on dance-driven 3-D camera interaction: a collaboration between dance, graphic design and computer science. In Proceedings of the 12th Annual Conference of the New Zealand Chapter of the ACM Special Interest Group on Computer-Human Interaction (pp. 25-32). ACM. (2011, July).
[2] M. Tang, "Recognizing hand gestures with microsoft’s kinect." Palo Alto: Department of Electrical Engineering of Stanford University:[sn] (2011).
[3] W. Tao, Y. Qiao, and B. Lee. "Kinect Skeleton Coordinate Calibration for Remote Physical Training." MMEDIA 2014, The Sixth International Conferences on Advances in Multimedia. 2014.
[4] S. Kaenchan, P. Mongkolnam, B. Watanapa, and S. Sathienpong, "Automatic multiple kinect cameras setting for simple walking posture analysis." Computer Science and Engineering Conference (ICSEC), 2013 International. IEEE, 2013.
[5] M. H. Hsu, W. G. C. W. Kumara, T. K. Shih, and Z. Cheng, “Spider King: Virtual musical instruments based on microsoft Kinect,” in 2013 International Joint Conference on Awareness Science and Technology and Ubi-Media Computing (iCAST-UMEDIA), Ulaanbaatar, 2013, pp. 707-713.
[6] S. Sentürk, S. W. LEE, A. Sastry, A. Daruwalla, and G. Weinberg "Crossole: A gestural interface for composition, improvisation and performance using kinect." Proc. NIME. Vol. 2012. 2012.
[7] MIDI MANUFACTURES ASSOCIATION. MIDI [Online].Available: https://www.midi.org/index.php
[8] Microsoft. KinectSDK. [Online].Available:https://msdn.microsoft.com/en-us/library/hh855347.aspx

[9] Solarian.(2013,05,22). OpenGL101: Matrices - projection, view, model.[Online].Available:https://solarianprogrammer.com/2013/05/22/opengl-101-matrices-projection-view-model/
[10] Y. Chen and G. Medioni. Object modeling by registration of multiple range images. Image and Vision Computing (IVC), 10(3):145-155, 1992. 2.3, 3.5
[11] P. Besl, and N. McKay, "A Method for Registration of 3-D Shapes", IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no.2, pp.239-256, Feb., 1992
[12] R. S. Wright, and B. Lipchak, (2004). OpenGL superbible.
[13] Sorkine, O., 2007. Least-squares rigid motion using svd, [Online].Available http://igl.ethz.ch/projects/ARAP/svd_rot.pdf.
[14] F. Colas, 2011. Iterative Closest Point Algorithm [Online]. Available: http://www.asl.ethz.ch/education/master/info-process-rob/ICP.pdf
[15] K. Low. Linear least-squares optimization for point-to-plane icp surface registration. Technical report, Technical Report TR04-004, Department of Computer Science, University of North Carolina at Chapel Hill, 2004.
[16] J. Wilm, 2013. Iterative Closest Point matlab code [Online]. Available:
http://www.mathworks.com/matlabcentral/fileexchange/27804-iterative-closest-point

[17] H. Kjer, and J. Wilm. Evaluation of surface registration algorithms for PET motion correction. Master's thesis, Technical University of Denmark, 2010. 7.

[18] dfrec 2010.12.07 OpenGL入门教程 [Online]. Available:
http://wenku.baidu.com/view/8096a02bcfc789eb172dc892.html

[19] Y. H. Hsiao, An Application of MIDI in Guitar''s Scoring. Master’s thesis, Tamkang University, 1999.

[20] J. wolfe, 2005. Note names, MIDI numbers and frequencies [Online]. Available:
http://newt.phys.unsw.edu.au/jw/notes.html

[21] S. Kaenchan, P. Mongkolnam, B. Watanapa, and S. Sathienpong, Automatic multiple Kinect cameras setting for simple walking posture analysis Computer Science and Engineering Conference (ICSEC), 2013 International, Nakorn Pathom, pp.245-249, 4-6 Sept. 2013.

[22] A. Saeed Matlab code icp.m [Online]. Available:
http://staffhome.ecm.uwa.edu.au/~00053650/code.html

簡易檢索 / 詳目顯示

相關論文