跳到主要內容

簡易檢索 / 詳目顯示

研究生: 黃柏軒
Po-Hsuan Huang
論文名稱: 對於大量天文資料之分散式小行星軌跡探索
Distributed Asteroid Track Discovery for Large Astronomical Data
指導教授: 蔡孟峰
Meng-Feng Tsai
口試委員:
學位類別: 碩士
Master
系所名稱: 資訊電機學院 - 資訊工程學系
Department of Computer Science & Information Engineering
論文出版年: 2015
畢業學年度: 103
語文別: 中文
論文頁數: 69
中文關鍵詞: 大量資料雲端運算小行星軌跡Hough Transform演算法
相關次數: 點閱:9下載:0
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 小行星(asteroid)遍布於整個太陽系當中,受到重力吸引而造成其移動的現象(moving object)。隨著硬體設備技術的進步,天文資料的獲取越來越快速。於大量的望遠鏡觀測資料當中尋找出小行星軌跡,將可以幫助天文學家了解更深層的天文意義,以及對於軌跡的路徑是否對地球造成危害作更進一步的預測。然而天文觀測資料的龐大,透過手動或人工的方式計算軌跡過於費時費力,且精確性較低。
    為了要減少硬碟的寫入寫出,解決記憶體不足的問題,本論文將導入處理大量資料的方法,採用分散式系統,運用分散式的演算法,以具更系統性的方法尋找小行星軌跡。本論文將使用分散式的檔案系統與資料庫來做為儲存的設備,以良好可靠性與擴充性作為考量。同時應用分散式的Hough Transform演算法,搭配Open Source的雲端計算環境,更有效率地尋找小行星軌跡。
      在實驗的部分,本論文將於Palomar Transient Factory(PTF)天文觀測資料中找出小行星軌跡。本論文的方法明顯的讓軌跡探索的效率有所改進,對於未來新獲取的觀測資料也能進行有效率的更新,並提供視覺化的介面來觀察小行星軌跡的移動。


    Asteroids are spread in the whole Solar System. They are attracted by gravity which makes them a moving object. As the hardware device improve, acquiring astronomical data becomes faster. Discovering asteroid track in the large observed data can benefit astronomical researchers in understanding some astronomical phenomena and can be examined for further prediction to whether it causes disasters to Earth. Due to large astronomical data, finding out asteroid track by manual is absolutely hard, inefficient and low accuracy.
    In order to reduce disk I/O overhead and solve the problem of memory insufficient. This research proposes to adopt a distributed Hough Transform method with Big-Data data management and cloud computing technique for systematically searching asteroid tracks. This research utilizes distributed file system and database as the storage based on good reliability and scalability. It also uses an open source cloud computing environment for more efficiency in searching asteroid tracks.
      In the experiments, this research discovers asteroid tracks in Palomar Transient Factory astronomical observed data. The results represent that our method can improve the efficiency apparently. New observed data can be also updated efficiently and provide the interface of visualization for observing the moving of asteroid.

    摘要 i Abstract ii 誌謝 iii 目錄 iv 圖目錄 vi 一、 緒論 1 1-1 研究背景 1 1-2 研究動機與目的 2 1-3 論文架構 2 二、 文獻探討 4 2-1 巡天計畫 4 2-2 小行星 5 2-3 MapReduce架構 5 2-4 Hadoop 6 2-5 OpenStack 7 2-6 小行星軌跡探索 8 三、 系統架構 9 3-1 雲端運算平台 9 3-2 HDFS檔案系統 10 3-3 HBase資料庫 11 3-4 前處理階段(Preprocessing) 14 3-5 Hough Transform階段(HT Stage) 16 3-6 Track Discovery階段(TD Stage) 16 四、 研究方法 17 4-1 Hough Transform演算法 17 4-1-1 分散式Hough Transform演算法 20 4-1-2 天文資料的分割 21 4-1-3 HT Stage的MapReduce 23 4-2 小行星軌跡的延伸 24 4-2-1 小行星軌跡的更新 26 4-2-2 TD Stage的MapReduce 30 4-3 視覺化查詢界面 32 五、 實驗 36 5-1 HT Stage執行時間 37 5-1-1 HT Stage基於HDFS 37 5-1-2 HT Stage基於HBase 41 5-2 TD Stage執行時間 49 5-2-1 小行星軌跡延伸 49 5-2-2 小行星軌跡更新 51 六、 結論 54 參考文獻 55

    [1] R. Duda and P. Hart, “Use of the Hough Transformation to Detect Lines and Curves in Pictures”, Communications of the ACM CACM Homepage archive Volume 15 Issue 1, Pages 11-15, Jan. 1972.
    [2] Apache Hadoop, http://hadoop.apache.org/
    [3] Apache HBase, http://hbase.apache.org/
    [4] OpenStack, https://www.openstack.org/
    [5] C. Zhai, M. Shao, B. Nemati, T. Werne, H. Zhou, S. G. Turyshev, J. Sandhu, G. Hallinan, and L. K. Harding, “Detection of A Faint Fast-moving Near-earth Asteroid Using The Synthetic Tracking Technique”, The Astrophysical Journal, 792:60 (14pp), 2014 September 1.
    [6] Random Thoughts on Coding, http://codingjunkie.net/secondary-sort/
    [7] P. Gural, J. Larsen, and A. Gleason, “Matched filter processing for asteroid detection”, Astronomical Journal vol. 130 pp. 1951-1960 2005.
    [8] Hadoop 101: Programming MapReduce with Native Libraries, Hive, Pig, and Cascading, http://blog.pivotal.io/pivotal/products/hadoop-101-programming-mapreduce-with-native-libraries-hive-pig-and-cascading
    [9] Palomar Transient Factory, http://www.ptf.caltech.edu/
    [10] B. Shucker and J. Stuart, “Detecting asteroids with a multi-hypothesis velocity matched filter”, Proc. Asteroids, Comets, and Meteors, 2008.
    [11] D. Han and E. Stroulia, “A Three-Dimensional Data Model in HBase for Large Time-Series Dataset Analysis”, MESOCA, pp. 47-56, 2012.
    [12] Coding Hassle, http://codinghassle.blogspot.tw/2015/03/hadoop-shuffle-and-sort-phase.html
    [13] S. Daruru, S. Dhandapani, G. Gupta, I. Iliev, W. Xu, P. Navratil, N. Marin,J. Ghosh, “Distributed, Scalable Clustering for Detecting Halos in Terascale Astronomy Datasets”, Data Mining Workshops (ICDMW), pp. 138-147, 2010.
    [14] B. MA, A. Shoshani, A. Sim, K. Wu, Y. Byun, J. Hahm, M.-S. Shin, “Efficient Attribute-based Data Access in Astronomy Analysis”, High Performance Computing, Networking, Storage and Analysis (SCC), pp. 562-571, 2012.
    [15] D. Carstoiu, E. Lepadatu, M. Gaspar, “HBase - non SQL Database, Performances Evaluation”, International Journal of Advancements in Computing Technology
    Volume 2, Number 5, December 2010.
    [16] G. Mackey, S. Sehrish, J. Bent, J. Lopez, S. Habib, J. Wang, “Introducing Map-Reduce to High End Computing”, Petascale Data Storage Workshop, pp. 1-6, Nov. 2008.
    [17] Y. Liu, B. Chen, W. He,Y. Fang, “Massive Image Data Management using HBase and MapReduce”, Geoinformatics (GEOINFORMATICS), pp. 1-5, 2013.
    [18] Y. Chen, W. Li, J. Li, and T. Wang, “NOVEL PARALLEL HOUGH TRANSFORM ON MULTI-CORE PROCESSORS”, Acoustics, Speech and Signal Processing, 2008. ICASSP, pp. 1457-1460, 2008.
    [19] R. K. Satzoda, S. Suchitra, and T. Srikanthan, “Parallelizing the Hough Transform Computation”, IEEE SIGNAL PROCESSING LETTERS, VOL. 15, pp. 297-300, 2008.
    [20] L. Denneau, R. Jedicke, T. Grav, et al., “The Pan-STARRS Moving Object Processing System”, PASP 125, pp. 357–395, 2013.
    [21] M. Shao, B. Nemati, C. Zhai, S. G. Turyshev, J. Sandhu, G. W. Hallinan, L. K. Harding, “Finding Very Small Near-Earth Asteroids using Synthetic Tracking”, The Astrophysical Journal, 782:1 (10pp), 2014 February 10.
    [22] L. Cai, S. Huang, “Performance analysis and testing of HBase based on its architecture”, Computer and Information Science (ICIS), pp. 353-358, 2013.
    [23] V. Krishna Konishetty, K. Arun Kumar, K. Voruganti, G. V. Prabhakara Rao, “Implementation and evaluation of scalable data structure over HBase”, International Conference on Advances in Computing, Communications and Informatics (ICACCI-2012), pp. 1010-1018, 2012.
    [24] J. Bhogal, I. Choksi, “Handling Big Data using NoSQL”, Advanced Information Networking and Applications Workshops (WAINA), pp. 393-398, 2015.
    [25] S. Ibrahim, H. Jin, L. Lu, L. Qi, S. Wu, X. Shi, “Evaluating MapReduce on Virtual Machines: The Hadoop Case”, Cloud Computing Lecture Notes in Computer Science Volume 5931, pp 519-528, 2009.
    [26] S. Moon, J. Lee, Y. Suk Kee, “Introducing SSDs to the Hadoop MapReduce Framework”, Cloud Computing (CLOUD), pp. 272-279, 2014.
    [27] S. S. Sathyanarayana, R. K. Satzoda, and T. Srikanthan, “Exploiting Inherent Parallelisms for Accelerating Linear Hough Transform”, Image Processing, IEEE Transactions on Volume:18 Issue: 10, pp. 2255-2264, 2009.
    [28] The Khangaonkar Report, http://khangaonkar.blogspot.tw/2013/04/using-hbase-part-2-architecture.html
    [29] Big data, http://hadoopbigdatas.blogspot.tw/2013/03/hbase-architecture.html
    [30] Sachin Puttur: Big Data: Overview of apache Hadoop, http://www.sachinpbuzz.com/2014/01/big-data-overview-of-apache-hadoop.html
    [31] Hough Transform, http://en.wikipedia.org/wiki/Hough_transform
    [32] Chair for Computer Aided Medical Procedures & Augmented Reality Lehrstuhl für Informatikanwendungen in der Medizin & Augmented Reality, http://campar.in.tum.de/Students/DaPentenrieder
    [33] J. Dean and S. Ghemawat, “MapReduce: Simplified Data Processing on Large Clusters”, OSDI'04: Sixth Symposium on Operating System Design and Implementation,
    San Francisco, CA, December, 2004.
    [34] 小行星, https://zh.wikipedia.org/wiki/%E5%B0%8F%E8%A1%8C%E6%98%9F

    QR CODE
    :::