隨著交通方便性的提升，旅行已成為現代人們的常態，旅遊型態也漸漸地產生了改變，不再是單純的觀光而是希望可以更深入的體會當地的風土民情，其中參與當地的特色活動便是一種可以體會當地特色的方式。然而，在網站中搜尋當地的活動對不熟悉當地的人卻是一個負擔，因為無論是政府組織亦或是民間組織，他們通常都將活動的資訊更新在他們自己的網站中，散落在WWW中，因此在這邊我們希望可以使用一個智慧的爬蟲系統，可以自動且有效率的探勘並收集「活動來源網頁」。此文主要是想要介紹我們要如何訓練一個智慧爬蟲模型，讓他可以從起始頁面網站去探勘此網站的活動來源網頁，因為我們認為每個網站的活動來源頁面的個數都是不同的，因此智慧爬蟲於每個網站中走的步伐數皆為變動的，在這邊我們也會提及如何設定閥值讓我們的模型知道是否該停止探索此網站，在此模型中我們採用了強化式學習(Reinforcement learning)並結合了多任務學習(Multitask Learning)來訓練，也因為我們只有有限的標記資料，因此我們採用兩階段的訓練架構進行訓練，第一階段會運用少量的標記資料先進行預訓練而後再透過未標記資料與我們的「活動來源網頁分類器」進行微調模型，最後藉由我們所提出的方法，我們的爬蟲模型於真實世界的資料上達到了74\%的準確度。

With the convenience of transportation, traveling is no longer about sightseeing or taking a professional photo but more about joining local event to experience local culture. Most event organizers such as governments, enterprises and organizations will update event information somewhere on their website. How to efficiently find the page where event announcements are listed for any given website is called the problem of event source discovery. In this paper, we show a deep reinforcement learning model to train our event source discovery agent. We use two stages to train our crawler, pre-training and fine-tuning. In the pre-training phase, the model is trained with limited labeled data, where each episode has a fixed time step. In the fine-tuning phase, the agent is trained using unlabeled data and a reward system based on an event source page classifier. The agent learns whether to continue exploring or stop exploring through an adaptive threshold, so the number of steps in each episode changes during the fine training. The proposed agent achieves \textcolor{red}{74\%} Return-On-Investment (i.e. precision) with \textcolor{red}{1.3} unit cost (the number of clicks for each event source page) on the real word data set.

[1] A. Amalia, D. Gunawan, A. Najwan, and F. Meirina. Focused crawler for the acquisition of health articles. In 2016 International Conference on Data and Software Engineering (ICoDSE), pages 1–6, 2016.
[2] Jonathan Baxter. A bayesian/information theoretic model of learning to learn via multiple task sampling. Machine Learning, 28:7–39, 2004.
[3] M. G. Bellemare, Y. Naddaf, J. Veness, and M. Bowling. The arcade learning environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research, 47:253–279, Jun 2013.
[4] R. Caruana. Multitask learning: A knowledge-based source of inductive bias. In ICML, 1993.
[5] Soumen Chakrabarti, Martin van den Berg, and Byron Dom. Focused crawling: a new approach to topic-specific web resource discovery. Computer Networks, 31(11):1623–1640, 1999.
[6] Ronan Collobert and Jason Weston. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th International Conference on Machine Learning, ICML ’08, pages 160–167, New York, NY, USA, 2008. ACM.
[7] Saraswathi Devaraj and A. Krishnakumar. Effective search engine spam classi- fication. 8:1541–1545, 08 2019.
[8] Gary William Flake, Steve Lawrence, and C. Lee Giles. Efficient identification of web communities. In Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’00, page 150–160, New York, NY, USA, 2000. Association for Computing Machinery.
[9] Miyoung Han, Pierre-Henri Wuillemin, and P. Senellart. Focused crawling through reinforcement learning. In ICWE, 2018.
[10] H. V. Hasselt, A. Guez, and D. Silver. Deep reinforcement learning with double q-learning. In AAAI, 2016.
[11] N. Heess, TB Dhruva, S. Sriram, Jay Lemmon, J. Merel, Greg Wayne, Y. Tassa, T. Erez, Ziyu Wang, S. Eslami, Martin A. Riedmiller, and D. Silver. Emergence of locomotion behaviours in rich environments. ArXiv, abs/1707.02286, 2017.
[12] Bing Liu, Robert Grossman, and Yanhong Zhai. Mining data records in web pages. In Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’03, page 601–606, New York, NY, USA, 2003. Association for Computing Machinery.
[13] Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Philip S. Yu. Learning multiple tasks with multilinear relationship networks. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 1593–1602, Red Hook, NY, USA, 2017. Curran Associates Inc.
[14] Houqing Lu, Donghui Zhan, Lei Zhou, and Dengchao He. An improved focused crawler: Using web page classification and link priority evaluation. Mathemat- ical Problems in Engineering, 2016:1–10, 01 2016.
[15] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, Ioannis Antonoglou, Daan Wierstra, and Martin A. Riedmiller. Playing atari with deep reinforcement learning. ArXiv, abs/1312.5602, 2013.
[16] V. Mnih, K. Kavukcuoglu, D. Silver, Andrei A. Rusu, J. Veness, Marc G. Belle- mare, A. Graves, Martin A. Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, S. Petersen, C. Beattie, A. Sadik, Ioannis Antonoglou, Helen King, D. Kumaran, Daan Wierstra, S. Legg, and Demis Hassabis. Human-level control through deep reinforcement learning. Nature, 518:529–533, 2015.
[17] Shakir Mohamed and Danilo J. Rezende. Variational information maximisation for intrinsically motivated reinforcement learning. In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2, NIPS’15, page 2125–2133, Cambridge, MA, USA, 2015. MIT Press.
[18] Ioannis Partalas, G. Paliouras, and I. Vlahavas. Reinforcement learning with classifier selection for focused crawling. In ECAI, 2008.
[19] Xiaoguang Qi and Brian D. Davison. Web page classification: Features and algorithms. ACM Comput. Surv., 41(2), February 2009.
[20] John Schulman, F. Wolski, Prafulla Dhariwal, A. Radford, and Oleg Klimov. Proximal policy optimization algorithms. ArXiv, abs/1707.06347, 2017.
[21] D. Silver, G. Lever, N. Heess, T. Degris, Daan Wierstra, and Martin A. Ried- miller. Deterministic policy gradient algorithms. In ICML, 2014.
[22] Aixin Sun, Ee-Peng Lim, and Wee-Keong Ng. Web classification using sup- port vector machine. In Proceedings of the 4th International Workshop on Web Information and Data Management, WIDM ’02, page 96–99, New York, NY, USA, 2002. Association for Computing Machinery.
[23] Aixin Sun, Ee-Peng Lim, and Wee-Keong Ng. Web classification using sup- port vector machine. In Proceedings of the 4th International Workshop on Web Information and Data Management, WIDM ’02, page 96–99, New York, NY, USA, 2002. Association for Computing Machinery.
[24] R. Sutton and A. Barto. Reinforcement learning: An introduction. IEEE Transactions on Neural Networks, 16:285–286, 2005.
[25] Richard S. Sutton, David McAllester, Satinder Singh, and Yishay Mansour. Pol- icy gradient methods for reinforcement learning with function approximation. In Proceedings of the 12th International Conference on Neural Information Pro- cessing Systems, NIPS’99, page 1057–1063, Cambridge, MA, USA, 1999. MIT Press.
[26] Qifan Wang, Bhargav Kanagal, Vijay Garg, and D. Sivakumar. Constructing a comprehensive events database from the web. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM ’19, page 229–238, New York, NY, USA, 2019. Association for Computing Ma- chinery.
[27] Chris Watkins. Learning from delayed rewards. 1989.
[28] W. Yan and L. Pan. Designing focused crawler based on improved genetic algorithm. In 2018 Tenth International Conference on Advanced Computational Intelligence (ICACI), pages 319–323, March 2018.
[29] B. W. Yohanes, H. Handoko, and H. K. Wardana. Focused crawler optimiza- tion using genetic algorithm. TELKOMNIKA Telecommunication Computing Electronics and Control, 9:403–410, 2011.
[30] Banu Yohanes, Peter Handoko, and Hartanto Wardana. Focused crawler opti- mization using genetic algorithm. Telkomnika, 9, 12 2011.
[31] Jia Zhu, Qing Xie, Shoou-I Yu, and Wai Hung Wong. Exploiting link structure for web page genre identification. Data Min. Knowl. Discov., 30(3):550–575, May 2016.