隨著資訊與網路的蓬勃發展，許多個人化的服務提供也隨之興起，衍伸而出的是許多商機。除了電子商務領域中的個人化服務推薦的應用之外，亦有個人化數位學習的應用。而這些個人化服務的提供最首要的一件事情就是「獲得使用者於使用服務時的使用動作/操作行為」。在針對如何獲得使用者操作行為的這個領域中，目前在學術上與網路上也已經有許多關於狀態獲取與辨識的文獻與機制，但這些狀態辨識機制卻仍不夠全面性地辨識使用者操作時所點選的項目資訊。
為了能夠更全面地辨識使用者操作時所點選的項目資訊，本研究提出實驗狀態回饋機制（Experimental State Feedback Mechanism, ESFM），能夠即時捕捉使用者在Windows作業系統中的操作動作。並以CSEP雲端安全實驗平台為例，CSEP是一個具有互動式教學的數位學習平台。故可利用ESFM來捕捉使用者在虛擬實驗主機上的操作動作，進而回饋實驗的下一步驟給CSEP，讓使用者能夠一步步地接續進行爾後的實驗。
本研究的實驗狀態回饋機制（ESFM）經實驗證明可適切地進行捕捉使用者的動作以及辨識視窗與元件資訊，且對於系統效能的影響很低，也能夠改善目前CSEP案例中所使用的Sikuli機制的不足。因此本研究的ESFM可在不影響使用者操作的情況下，獲得使用者在虛擬實驗主機上操作動作的資訊，並提供CSEP未來在設計案例時所用。

With the rapid development of Internet, many personalized services also rise and bring many business opportunities. In addition to the application of personalized service recommendation in the field of e-commerce, some application of personal e-learning also exists in the field of providing personalized services. While providing these personalized services, the most important thing is "To get user's action behavior when they using the service." There are many academic literatures and mechanisms about how do we access and identify user's action behavior, but this information is still not enough to show the details of user's operating behavior.
In order to identify user's action behavior more accurately, our study presents the Experimental Status Feedback Mechanism (ESFM). ESFM could instantly capture user's action on Windows OS. Our study use Cloud Security Experimental Platform (CSEP) as an example, CSEP is an e-learning platform and provided with interactive teaching. Therefore, through ESFM could capture user's action when they do some operations on the virtual machine. When ESFM capture this information, it will send this information to CSEP server, allowing users to follow steps and finishing their experiments.
The ESFM we proposed has been proven that it can aptly capture the user's actions and recognize the window and widgets information. ESFM not only has lower impact on system performance, but improve the Sikuli mechanism used in CSEP experiments. Therefore, ESFM could capture the user's operation without affecting them on the virtual machine and used for future design of CSEP experiments.

[1] 劉旭哲，「互動式線上教學之互動點推薦機制研究」，國立中央大學資訊管理學系碩士論文，2012。
[2] 黃瀞瑩，「一個提供彈性虛擬資料中心的雲端服務平台」，國立中央大學資訊工程學系碩士論文，2011。
[3] Y. Wang, J. Lin, M. Annavaram, Q. A. Jacobson, J. Hong, B. Krishnamachari, and N. Sadeh, “A Framework of Energy Efficient Mobile Sensing for Automatic User State Recognition,” in Proceedings of the 7th International Conference on Mobile Systems, Applications, and Services, New York, NY, USA, pp. 179–192, 2009.
[4] S. Keates, F. Hwang, P. Langdon, P. J. Clarkson, and P. Robinson, “Cursor Measures for Motion-impaired Computer Users,” in Proceedings of the Fifth International ACM Conference on Assistive Technologies, New York, NY, USA, pp. 135–142, 2002.
[5] D. R. Kaufman, V. L. Patel, C. Hilliman, P. C. Morin, J. Pevzner, R. S. Weinstock, R. Goland, S. Shea, and J. Starren, “Usability in the Real World: Assessing Medical Information Technologies in Patient’s Homes,” J Biomed. Inform., Vol. 36, No. 1/2, pp. 45–60, Feb. 2003.
[6] B. Cannon and E. Wohlstadter, “Enforcing Security for Desktop Clients Using Authority Aspects,” in Proceedings of the 8th ACM International Conference on Aspect-oriented Software Development, New York, NY, USA, pp. 255–266, 2009.
[7] V. K. Santhalia and S. Singh, “Design and Development of a Desktop Monitoring System,” in Proceedings of the Second International Conference on Computational Science, Engineering and Information Technology, New York, NY, USA, pp. 620–625, 2012.
[8] R. Atterer, M. Wnuk, and A. Schmidt, “Knowing the User’s Every Move: User Activity Tracking for Website Usability Evaluation and Implicit Interaction,” in Proceedings of the 15th International Conference on World Wide Web, New York, NY, USA, pp. 203–212, 2006.
[9] Y.M. Chen, C.E. Chuang, H.C. Liu, C.Y. Ni, and C.T. Wang, “Using Agent in Virtual Machine for Interactive Security Training,” in Security Technology, T. Kim, H. Adeli, W. Fang, J. G. Villalba, K. P. Arnett, and M. K. Khan, Eds. Springer Berlin Heidelberg, pp. 65–74, 2011.
[10] S.J. Chen, J.Y. Huang, C.T. Huang, and W.J. Wang, “SAMEVED: A System Architecture for Managing and Establishing Virtual Elastic Datacenters,” Int J Grid High Perform Comput, Vol. 5, No. 2, pp. 27–42, Apr. 2013.
[11] U. Kukreja, W. E. Stevenson, and F. E. Ritter, “RUI: Recording user input from interfaces under Windows and Mac OS X,” Behav. Res. Methods, Vol. 38, No. 4, pp. 656–659, Nov. 2006.
[12] J. Alexander, A. Cockburn, and R. Lobb, “AppMonitor: A tool for recording user actions in unmodified Windows applications,” Behav. Res. Methods, Vol. 40, No. 2, pp. 413–421, May 2008.
[13] K. Mera and T. Ichimura, “Classification of EGC output and Mental State Transition Network using Self Organizing Map,” in Proceedings of the 2011 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 3155–3160, 2011.
[14] C. Epp, M. Lippold, and R. L. Mandryk, “Identifying Emotional States Using Keystroke Dynamics,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New York, NY, USA, pp. 715–724, 2011.
[15] N. Bidargaddi, A. Sarela, and I. Korhonen, “Physiological state characterization by clustering heart rate, heart rate variability and movement activity information,” in Proceedings of the 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2008. EMBS 2008, pp. 1749–1752, 2008.
[16] T. Tani and S. Yamada, “User state identification through desktop interaction,” in Proceedings of the 2012 Joint 6th International Conference on Soft Computing and Intelligent Systems (SCIS) and 13th International Symposium on Advanced Intelligent Systems (ISIS), pp. 395–398, 2012.
[17] T. Ji, P. Tuo-Yu, Z. Zhen-Min, and L. Kai, “A personalized recommendation model based on the user-state awareness,” in Proceedings of the 1st IEEE Symposium on Web Society, 2009. SWS ’09, pp. 10–13, 2009.
[18] F. Zhang, J. Chen, H. Chen, and B. Zang, “CloudVisor: Retrofitting Protection of Virtual Machines in Multi-tenant Cloud with Nested Virtualization,” in Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles, New York, NY, USA, pp. 203–216, 2011.
[19] R. Ando and K. Suzaki, “A Lightweight Access Log Filter of Windows OS Using Simple Debug Register Manipulation,” in Security Technology, T. Kim, H. Adeli, W. Fang, J. G. Villalba, K. P. Arnett, and M. K. Khan, Eds. Springer Berlin Heidelberg, pp. 215–227, 2011.
[20] A. Hurst, S. E. Hudson, and J. Mankoff, “Automatically Identifying Targets Users Interact with During Real World Tasks,” in Proceedings of the 15th International Conference on Intelligent User Interfaces, New York, NY, USA, pp. 11–20, 2010.
[21] T. Yeh, T.-H. Chang, and R. C. Miller, “Sikuli: Using GUI Screenshots for Search and Automation,” in Proceedings of the 22Nd Annual ACM Symposium on User Interface Software and Technology, New York, NY, USA, pp. 183–192. , 2009
[22] “以電子商務推薦系統打造個人化數位學習服務.” [Online]. Available: http://newsletter.teldap.tw/news/InsightReportContent.php?nid=4653&lid=533.
[23] “How can we deliver more personalized service to our guests?” [Online]. Available: http://www.sas.com/content/dam/SAS/en_us/doc/solutionbrief/deliver-more-personalized-service-to-guests-105156.pdf.
[24] “中央大學雲端安全實驗平台.” [Online]. Available: http://csep.mgt.ncu.edu.tw/.
[25] “Hooks.” [Online]. Available: http://msdn.microsoft.com/en-us/library/windows/desktop/ms632589(v=vs.85).aspx.
[26] “The GTK+ Project.” [Online]. Available: http://www.gtk.org/.
[27] “Oxford Learner’s Dictionaries.” [Online]. Available: http://www.oxfordlearnersdictionaries.com/.
[28] “Sikuli Script.” [Online]. Available: http://www.sikuli.org/.
[29] “Introducing Spy++.” [Online]. Available: http://msdn.microsoft.com/en-us/library/dd460756.aspx.
[30] “UI Automation.” [Online]. Available: http://msdn.microsoft.com/en-us/library/windows/desktop/ee684009(v=vs.85).aspx.
[31] “WinSpy++ 1.7.” [Online]. Available: http://www.catch22.net/software/winspy-17.
[32] “MS Spy++ style Window Finder.” [Online]. Available: http://www.codeproject.com/Articles/1698/MS-Spy-style-Window-Finder.
[33] “Penetration Testing Software, Pen Testing Security,” Metasploit. [Online]. Available: http://www.metasploit.com/.
[34] “Home Keylogger Free Edition.” [Online]. Available: http://www.kmint21.com/keylogger/.
[35] “ISO/IEC 25010:2011.” [Online]. Available: http://www.iso.org/iso/home/store/catalogue_ics/catalogue_detail_ics.htm?csnumber=35733.
[36] “User mode and kernel mode.” [Online]. Available: http://msdn.microsoft.com/en-us/library/windows/hardware/ff554836(v=vs.85).aspx.
[37] “Windows API Index.” [Online]. Available: http://msdn.microsoft.com/en-us/library/windows/desktop/ff818516(v=vs.85).aspx.
[38] “SetWindowsHookEx function.” [Online]. Available: http://msdn.microsoft.com/en-us/library/windows/desktop/ms644990(v=vs.85).aspx.
[39] “Virtual-Key Codes.” [Online]. Available: http://msdn.microsoft.com/en-us/library/windows/desktop/dd375731(v=vs.85).aspx.
[40] “Control Library.” [Online]. Available: http://msdn.microsoft.com/en-us/library/windows/desktop/bb773169(v=vs.85).aspx.
[41] “RARLAB.” [Online]. Available: http://www.rarlab.com/download.htm.
[42] “pcman - OpenFoundry.” [Online]. Available: http://www.openfoundry.org/of/projects/744.