| 研究生: |
林冠辰 Kuan-Chen Lin |
|---|---|
| 論文名稱: |
基於CSP與最佳狀態序列之擷取程式驗證 Template and Schema Guided Wrapper Verification based on CSP and Best State Sequence |
| 指導教授: | 張嘉惠 |
| 口試委員: | |
| 學位類別: |
碩士 Master |
| 系所名稱: |
資訊電機學院 - 資訊工程學系在職專班 Executive Master of Computer Science & Information Engineering |
| 論文出版年: | 2013 |
| 畢業學年度: | 101 |
| 語文別: | 中文 |
| 論文頁數: | 39 |
| 中文關鍵詞: | 擷取程式驗證 、有限狀態機 、限制滿足問題 、網頁資料擷取 |
| 外文關鍵詞: | wrapper verification, Finite State Machine, Constraint Satisfaction Problems, web data extraction |
| 相關次數: | 點閱:11 下載:0 |
| 分享至: |
| 查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
這是資訊爆炸的時代,幾乎任何資訊都可透過網路取得,因此近期網頁資料擷取領域,相繼有許多研究提出各種的非監督式擷取方法,能夠快速有效擷取資料進行後續加值應用。但是網路環境快速變動的特定,也讓非監督式擷取程式面臨挑戰。非監督式擷取程式在產生擷取規則時,需要相當複雜計算,需要耗費相當的時間,不可能每次擷取資料都重新產生新的擷取規則,因此擷取程式後續驗證與維護機制必然越來越重要。
擷取程式會解析網頁內容,產生網頁模板(Template)與資料結構(Schema)來擷取資料。擷取程式驗證機制主要目的是確保時間點t產生的模板(Template)與資料結構結構(Schema)仍能適用於時間點t’的資料擷取。在網頁XML DOM樹狀結構中,資料內容是存在於葉節點位置,因此可運用有限狀態機模型檢測網頁葉節點轉換規律是否與時間點t的網頁模板(Template)與資料結構(Schema)一致。本論文嘗試簡化建立有限狀態機模型過程,使其更為快速有效,並以CSP快速刪減候選狀態節點的數量,加速模型驗證效率,並搭配狀態的序列組合(Sequence Probability)來檢測網頁模板(Template)與資料結構結構(Schema)有效性,以實驗測試其效率及有效性。
Wrapper induction is a complex process that takes a considerable amount of time. However, data extraction that requires constant wrapper induction is inefficient for a more sophisticated Web site design. Therefore, wrapper verification and its maintenance are becoming two of a few major subjects for research. This paper focuses on wrapper verification of unsupervised information extraction. The researcher uses the leaf nodes of the dom tree as the source of transformation and obtains a FSM (Finite State Machine) for schema verification. If the new page could pass the verification, it would be considered a simultaneous data layout.
This paper attempts to simplify the process of creating the finite state machine model, and proposes Schema Guided Wrapper Verification based on CSP (Constraint Satisfaction Problems) to reduce the number of candidate states to accelerate validation efficiency. The approach this study proposes not only improves validation efficiency, but also finds a better Best State Sequence to improve the accuracy of data extraction.
[1] C.-H. Chang, M. Kayed, M. R. Girgis, and K. F. Shaalan, "A Survey of Web
Information Extraction Systems," IEEE Transactions on Knowledge and Data
Engineering, vol. 18, no. 10, pp. 1411-1428, 2006.
[2] C.-H. Chang, S. Yang, C.-M. Liou, and M. Kayed, "Gadget creation for
personal information integration on web portals," IEEE International
Conference on Information Reuse and Integration, 2008.
[3] M. Dontcheva, S. M. Drucker, G. Wade, D. Salesin, and M. F. Cohen,
"Summarizing personal web browsing sessions," Proceedings of the 19th
annual ACM symposium on User interface software and technology, 2006, pp.
115-124.
[4] O. Etzioni, M. Cafarella, D. Downey, A.-M. Popescu, T. Shaked, S. Soderland,
D. S. Weld, and A. Yates, "Methods for domain-independent information
extraction from the web: an experimental comparison," Proceedings of the
19th national conference on Artifical intelligence, 2004, pp. 391-398.
[5] C.-N. Hsu and C.-C. Chang, "Finite-state transducers for semi-structured text
mining," Proceedings of IJCAI-99 Workshop on Text Mining: Foundations,
Techniques and Applications, 1999, pp. 38-49.
[6] M. Kayed and C.-H. Chang, "Page-Level Web Data Extraction from Template
Pages," IEEE Transactions on Knowledge and Data Engineering, vol. 22, no.
2, pp. 249-263, 2010.
[7] N. Kushmerick, "Wrapper Induction for Information Extraction." Ph.D.
University of Washington, Seattle, WA, 1997.
[8] N. Kushmerick, "Regression testing for wrapper maintenance," Proceedings of
the sixteenth national conference on Artificial intelligence and the eleventh
Innovative applications of artificial intelligence conference innovative
applications of artificial intelligence, 1999, pp. 74-79.
[9] N. Kushmerick, "Wrapper Verification," World Wide Web Journal, vol. 3, no. 2,
pp. 79-94, 2000.
[10] N. Kushmerick, "Wrapper Induction: Efficiency and Expressiveness,"
Artificial Intelligence, vol. 118, no. 1-2, pp. 15-68, 2000.
[11] K. Lerman, S. N. Minton, and C. A. Knoblock, "Wrapper Maintenance: A
Machine Learning Approach," Journal of Artificial Intelligence Research, vol.
18, no. 1, pp. 149-181, 2003.
[12] J.-H. Li, "Differentiating Templates and Data Values from Semi-Structured
Web Pages." Master's Computer Science and Information Engineering at
National Center University, 2005.
[13] L. Liu, C. Pu, and W. Han, "XWRAP: An XML-Enabled Wrapper
Construction System for Web Information Sources," Proceedings of the 16th
International Conference on Data Engineering, 2000, pp. 611-621.
[14] X. Meng, D. Hu, and C. Li, "Schema-Guided Wrapper Maintenance for
Web-Data Extraction," Proceedings of the 5th ACM international workshop on
Web information and data management, 2003, pp. 1-8.
[15] X. Meng, H. Lu, M. Gu, and H. Wang, "SG-WRAP: A Schema-Guided
Wrapper Generator," Proceedings of the 18th International Conference on
Data Engineering, 2002, p. 331.
[16] I. Muslea, S. Minton, and C. A. Knoblock, "Hierarchical Wrapper Induction
for Semistructured Information Sources," Autonomous Agents and Multi-Agent
Systems, vol. 4, no. 1-2, pp. 93-114, 2001.
[17] E.-H. Pek, X. Li, and Y. Liu, "Web Wrapper Validation," In Proceedings of
APWeb, 2003.
[18] Y.-L. Lin, "Page-level Wrapper Verification based on Structure, Semantic
and Schema." Master's Computer Science and Information Engineering
at National Center University, 2008.
[19] C. A. Knoblock Projects,
http://www.isi.edu/integration/people/knoblock/projects/prj_wrapper_maintain.html
[20] Document Object Model (DOM), http://www.w3.org/DOM/
[21] XML Schema Definition Language (XSD), http://www.w3.org/TR/xmlschema11-1/