跳到主要內容

簡易檢索 / 詳目顯示

研究生: 葉以新
I-Hsin Yeh
論文名稱: 在深度網路架構中探討時間衰變函數對應用注意力機制的推薦系統之影響
Analyze the influence of the time decay function in the recommendation system through the attention mechanism in the deep network architecture
指導教授: 陳彥良
Yen-Liang Chen
口試委員:
學位類別: 碩士
Master
系所名稱: 管理學院 - 資訊管理學系
Department of Information Management
論文出版年: 2021
畢業學年度: 109
語文別: 英文
論文頁數: 59
中文關鍵詞: 推薦系統BERT模型深度學習注意力機制時間衰退函數
外文關鍵詞: Recommendation system, Bert model, Deep learning, Attention mechanism, Time decay
相關次數: 點閱:12下載:0
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 過去,協同過濾(CF)技術在推薦系統中得到了廣泛的應用。其中由於潛在因子模型擁有將評分值分解為用戶表示向量和項目表示向量的乘積,並用以進行推薦之特性,許多研究者將其應用於深度學習架構下的協同過濾推薦系統之研究。
    不同於以往之研究,本研究對於深度學習架構下的潛在因子模型提出了三個方面的改進。首先對於不同的目標項目,由於關注目標的變化,用戶表示向量應該根據目標項目的內容進行動態調整。因此我們在推薦系統中加入了注意力機制,可以根據用戶的歷史購買經驗與目標產品的關係動態調整用戶表示向量。
    除此之外,考慮到在現實中人們的偏好通常會隨著時間而改變。因此基於上述注意力模型,本研究應用了兩個時間衰變函數來強調用戶的近期偏好變化。第一個衰變函數考慮的是最近的評分行為比很久以前的評分行為更重要的情況,因為最近的評分行為反映了用戶當前的興趣。第二時間衰變函數考慮了用戶通常更喜歡最近發布的電影而不是很久以前發布的電影的情況。
    通過將前述兩個時間衰減函數與注意力模型相結合,我們提出了一種用於項目得分預測的時間衰變自適應潛在分解機(TDADLFM)模型,並將此模型應用於集成 Movielens-10M 和 HetRec2011 的數據集,證明三個因素皆可以提高推薦性能。


    In the past, collaborative filtering (CF) technology has been widely used in recommendation systems. Usually, the latent factor model is used as the basis for implementing CF recommendation in the deep learning system. It will decompose the rating value into the product of the user embedding vector and the item embedding vector. This study is different from previous studies in three aspects. First, for different target items, due to changes in focus, the user embedding vector should be dynamically adjusted according to the content of the target item. Therefore, we have added an attention mechanism to the recommendation system, which can dynamically adjust the user's embedding vector according to the relationship between the user's historical purchase experience and the target product. However, in reality, people's preferences usually change over time. Therefore, based on the above attention model, this study considers two time decay functions to emphasize the user's recent preferences. The first decay function considers the situation where the most recent rating is more important than the rating a long time ago, because the most recent rating reflects the user's current interest. The second time decay function takes into account the situation where users generally prefer recently released movies to movies released a long time ago. By combining these two time decay functions with the attention model, we propose a time decay adaptive latent decomposition machine (TDADLFM) model for item score prediction. This study applies this model to a dataset integrating Movielens-10M and HetRec2011, and proves that all three new considerations can improve recommendation performance.

    摘要 i ABSTRACT ii List of Figures vi List of Tables vii 1. Introduction 1 1-1 Research background 1 1-2 Motivations 3 1-3 Contributions 6 2. Related Work 8 2-1 Traditional recommendation system 8 2-1-1 Content based 8 2-1-2 Model based collaborative filtering 8 2-2 Latent factor model 9 2-3 Recommendation system in deep learning 10 2-4 Time decay recommendation system 12 3. Recommendation Algorithm 15 3-1 TDLFM module 15 3-2 Input information 16 3-3 BERT 17 3-3-1 BERT embedding 17 3-4 Attention Mechanism 19 3-4-1 Attention mechanism A 20 3-4-2 Attention mechanism B 21 3-5 Time decay mechanism 22 3-5-1 Convex curve decay function 22 3-5-2 Linear curve decay function 23 3-5-3 Concave curve decay function 23 3-6 Adaptive time-decay function 23 3-7 Latent factor model 24 3-8 Release time decay model C 25 3.9. Loss function 26 4. Experiments 27 4-1 Datasets and preprocess 27 4-2 Measurement metric 28 4-3 Baseline 29 4-4 Experimental benchmark 30 4-5 Experimental platform 31 5. Experimental results and discussion 32 5-1 Experiment 1 32 5-2 Experiment 2 32 5-3 Experiment 3 33 5-4 Experiment 4 33 5-5 Experiment 5 33 5-6 Experiment 6 34 6. Conclusion and Future Work 36 6-1 Conclusion 36 6-1 Future work 36 References 38 Appendix A 42

    [ 1 ] Aggarwal, C.C., An introduction to recommender systems, in Recommender systems. 2016, Springer. p. 1-28.
    [ 2 ] Herlocker, J.L., et al., Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems (TOIS), 2004. 22(1): p. 5-53.
    [ 3 ] Garcin, F., et al. Offline and online evaluation of news recommender systems at swissinfo. ch. in Proceedings of the 8th ACM Conference on Recommender systems. 2014.
    [ 4 ] Blake, M.B. and M.F. Nowlan. A web service recommender system using enhanced syntactical matching. in IEEE International Conference on Web Services (ICWS 2007). 2007. IEEE.
    [ 5 ] Sarwar, B., et al. Item-based collaborative filtering recommendation algorithms. in Proceedings of the 10th international conference on World Wide Web. 2001.
    [ 6 ] Rendle, S., et al., BPR: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618, 2012.
    [ 7 ] Herlocker, J.L., et al. An algorithmic framework for performing collaborative filtering. in Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval. 1999.
    [ 8 ] Koren, Y., R. Bell, and C. Volinsky, Matrix factorization techniques for recommender systems. Computer, 2009. 42(8): p. 30-37.
    [ 9 ] Koren, Y. Factorization meets the neighborhood: a multifaceted collaborative filtering model. in Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining. 2008.
    [ 10 ] McAuley, J. and J. Leskovec. Hidden factors and hidden topics: understanding rating dimensions with review text. in Proceedings of the 7th ACM conference on Recommender systems. 2013.
    [ 11 ] Diao, Q., et al. Jointly modeling aspects, ratings and sentiments for movie recommendation (JMARS). in Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 2014.
    [ 12 ] Safoury, L. and A. Salah, Exploiting user demographic attributes for solving cold-start problem in recommender system. Lecture Notes on Software Engineering, 2013. 1(3): p. 303-307.
    [ 13 ] Zheng, L., V. Noroozi, and P.S. Yu. Joint deep modeling of users and items using reviews for recommendation. in Proceedings of the tenth ACM international conference on web search and data mining. 2017.
    [ 14 ] Han, J., et al., Adaptive deep modeling of users and items using side information for recommendation. IEEE transactions on neural networks and learning systems, 2019. 31(3): p. 737-748.
    [ 15 ] Atkinson, R.C. and R.M. Shiffrin, Human memory: A proposed system and its control processes, in Psychology of learning and motivation. 1968, Elsevier. p. 89-195.
    [ 16 ] Baddeley, A.D., Human memory: Theory and practice. 1997: psychology press.
    [ 17 ] Wixted, J.T. and E.B. Ebbesen, Genuine power curves in forgetting: A quantitative analysis of individual subject forgetting functions. Memory & cognition, 1997. 25(5): p. 731-739.
    [ 18 ] Wittmann, B.C., et al., Reward-related FMRI activation of dopaminergic midbrain is associated with enhanced hippocampus-dependent long-term memory formation. Neuron, 2005. 45(3): p. 459-467.
    [ 19 ] Pazzani, M.J. and D. Billsus, Content-based recommendation systems, in The adaptive web. 2007, Springer. p. 325-341.
    [ 20 ] Mnih, A. and R.R. Salakhutdinov, Probabilistic matrix factorization. Advances in neural information processing systems, 2007. 20: p. 1257-1264.
    [ 21 ] Van Den Oord, A., S. Dieleman, and B. Schrauwen. Deep content-based music recommendation. in Neural Information Processing Systems Conference (NIPS 2013). 2013. Neural Information Processing Systems Foundation (NIPS).
    [ 22 ] Blei, D.M., A.Y. Ng, and M.I. Jordan, Latent dirichlet allocation. the Journal of machine Learning research, 2003. 3: p. 993-1022.
    [ 23 ] Ling, G., M.R. Lyu, and I. King. Ratings meet reviews, a combined approach to recommend. in Proceedings of the 8th ACM Conference on Recommender systems. 2014.
    [ 24 ] Jhamb, Y. and Y. Fang, A dual-perspective latent factor model for group-aware social event recommendation. Information Processing & Management, 2017. 53(3): p. 559-576.
    [ 25 ] Kim, D., et al. Convolutional matrix factorization for document context-aware recommendation. in Proceedings of the 10th ACM conference on recommender systems. 2016.
    [ 26 ] Manotumruksa, J., C. Macdonald, and I. Ounis. Matrix factorisation with word embeddings for rating prediction on location-based social networks. in European Conference on Information Retrieval. 2017. Springer.
    [ 27 ] Salakhutdinov, R., A. Mnih, and G. Hinton. Restricted Boltzmann machines for collaborative filtering. in Proceedings of the 24th international conference on Machine learning. 2007.
    [ 28 ] Li, S., J. Kawale, and Y. Fu. Deep collaborative filtering via marginalized denoising auto-encoder. in Proceedings of the 24th ACM international on conference on information and knowledge management. 2015.
    [ 29 ] Wu, Y., et al. Collaborative denoising auto-encoders for top-n recommender systems. in Proceedings of the ninth ACM international conference on web search and data mining. 2016.
    [ 30 ] He, X., et al. Neural collaborative filtering. in Proceedings of the 26th international conference on world wide web. 2017.
    [ 31 ] Vartak, M., et al., A meta-learning perspective on cold-start recommendations for items. 2017.
    [ 32 ] Wei, J., et al., Collaborative filtering and deep learning based recommendation system for cold start items. Expert Systems with Applications, 2017. 69: p. 29-39.
    [ 33 ] Xia, C., et al. Dynamic item-based recommendation algorithm with time decay. in 2010 Sixth International Conference on Natural Computation. 2010. IEEE.
    [ 34 ] Richards, D., et al., Good times, bad times, and the diversionary use of force: A tale of some not-so-free agents. Journal of Conflict Resolution, 1993. 37(3): p. 504-535.
    [ 35 ] Ding, Y. and X. Li. Time weight collaborative filtering. in Proceedings of the 14th ACM international conference on Information and knowledge management. 2005.
    [ 36 ] Lee, T.Q., Y. Park, and Y.-T. Park, A time-based approach to effective recommender systems using implicit feedback. Expert systems with applications, 2008. 34(4): p. 3055-3062.
    [ 37 ] Gong, S. and G. Cheng. Mining user interest change for improving collaborative filtering. in 2008 Second International Symposium on Intelligent Information Technology Application. 2008. IEEE.
    [ 38 ] Wu, D., et al. Temporal social tagging based collaborative filtering recommender for digital library. in International Conference on Asian Digital Libraries. 2012. Springer.
    [ 39 ] Chen, T., et al. A simple framework for contrastive learning of visual representations. in International conference on machine learning. 2020. PMLR.
    [ 40 ] Chen, P.-C., et al. Dynamic time-aware attention to speaker roles and contexts for spoken language understanding. in 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). 2017. IEEE.

    QR CODE
    :::