跳到主要內容

簡易檢索 / 詳目顯示

回結果列表
研究生： 泰提潘
Tipajin Thaipisutikul
論文名稱： 基於上下文信息的新型順序推薦深度學習模型
A Novel Sequential Recommendation Deep Learning Model based on Contextual Information
指導教授： 施國琛
Timothy K. Shih
口試委員:
學位類別： 博士
Doctor
系所名稱： 資訊電機學院 - 資訊工程學系
Department of Computer Science & Information Engineering
論文出版年： 2021
畢業學年度： 109
語文別： 英文
論文頁數： 108
中文關鍵詞： 機器學習 、順序行為 、循環神經網絡 、注意力機制 、情境意識
外文關鍵詞： Machine learning, Sequential Behavior, Recurrent neural network, Attention mechanism, Context awareness
相關次數： 點閱：14  下載：0 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 過去幾年中，基於深度學習的模型（DL）受到了很多關注，尤其是在順序推薦任務領域。由於其處理複雜數據的能力，當前的順序DL研究工作已經超越了傳統模型，例如基於Markov鍊和基於因子分解的模型。但是，基於順序DL的模型的研究仍有改進的空間。特別是，如何設計有效的DL模型來處理不同場景下的順序推薦任務。在這種情況下，本文通過考慮現有方法的當前局限性，重點研究基於DL的順序推薦系統。具體來說，我們演示了順序推薦過程的概述概念，介紹了相關的最新算法，總結了影響基於DL的模型性能的關鍵因素，提出了用於復雜環境下順序推薦任務的新型基於DL的方法，並進行相應的評估，以顯示我們提出的模型在最新方法上的有效性。最後，我們通過系統地概述當前的挑戰，未來的方向以及我們在該領域的貢獻來結束我們的論文。最後，我們認為我們提出的項目對現有的序列感知推薦工作具有很高的積極貢獻。

    Deep learning based models (DL) have received a lot of attention in the past few years especially with the domain of sequential recommendation tasks. Due to its capability to deal with complex data, currently sequential DL research works have surpassed traditional models such as Markov chain-based and factorization-based models. However, there is a room for improvement on the studies of Sequential DL-based models. Particularly, how to design an effective DL model to handle the sequential recommendation tasks under different scenarios. In this view, this thesis focuses on the DL-based sequential recommender systems by taking the current limitations of existing methods into consideration. Specifically, we demonstrate the overview concept of sequential recommendation processes, present the related state-of-the-art algorithms, summarize the key factors affecting the performance of DL-based models, propose the novel DL-based for sequential recommendation tasks under complex settings, and conduct corresponding evaluations to show the effectiveness of our proposed models over the state-of-the-art methods. We finally conclude our thesis by systematically outlining current challenges, future directions and our contributions in this field. At last, we believe that our proposed project has high positive contributions to the existing sequence-aware recommendation works.

    摘 要 v ABSTRACT vi Authorship Attribution vii Acknowledgements viii Table of Content ix List of Figures xi List of Tables xiii Chapter I Introduction 1 Chapter II Overview of Sequential Recommendation 5 2.1 Concept Definition 5 2.2 Influential Factors on Sequential DL-based Models 6 2.2.1. Input Module 7 2.2.2. Data Pre-Processing 10 2.2.3. Model Structure 13 2.2.4. Model Training 18 2.2.5. Model Evaluation 22 Chapter III Related Work 24 3.1. Traditional POI Recommendation 24 3.2. Deep Learning POI Recommendation 25 3.3. Conventional Recommendation 27 3.4. Sequential Recommendation 28 3.5. Neural Attentive Recommendation 29 3.6. Additional Topics For The Future Works 33 3.7. Our Target Research Directions 34 Chapter VI A Learning-Based POI Recommendation with 39 Spatio-temporal Context-Awareness 39 4.1. Introduction 39 4.2. Preliminary 42 4.3. Proposed Recommendation System: DeNavi 42 1) Feature Extraction and Embedding 43 2) Learning Model and Training 43 DeNavi-LSTM: 44 3) Prediction Module of DeNavi 51 4.4. Performance Evaluation 52 A. Experiment Setup 52 B. ACC@N Performance 55 C. Precision@N, Recall@N and F-measure@N Performance 56 D. The Effectiveness of Time and Distance Transition Contexts 58 E. Scalability Discussion 60 F. Discussion of Parameter Settings 61 G. Interpretability Discussion 63 4.5. Conclusion 66 Chapter VII A Novel Context-aware Recommender System Based on A Deep Sequential Learning Approach (CReS) 67 5.1. Introduction 67 5.2. Preliminaries 71 5.3. The proposed model architecture: CRES 73 5.3.1. Feature Extraction and Input 74 5.3.2. Session Representation 75 5.3.3User Long-Term Preferences 77 5.3.4Top-N Recommendations 78 5.3.5Interpretable Recommendations 79 5.3.6Model Learning 79 5.4. Experimental Setting 80 5.4.1Datasets and Data-Preparation 80 5.4.2 Sequential Implication for Datasets 81 5.4.3 The advantage of CReS over the sequential datasets 83 5.6. Experimental Results 86 5.6.1. Overall Performance (RQ1) 86 5.6.2. Performance on other measures (RQ1) 87 5.6.3. Analysis on Generalization versus Specification of CReS (RQ1) 89 5.6.4. Analysis on Model Components (RQ2) 92 5.6.5. Effects of different hyper-parameter settings on CReS (RQ3) 94 5.6.6. Case Study (RQ4) 97 5.6.7. Discussion of all observations and future work propositions (RQ1-4) 98 5.7. Conclusion 99 Chapter VIII Conclusion 100 References 102

    References

    [1] Van M. (2020). Recommender System for Small and Medium-sized Businesses Finding Business Partners. Recommender Systems Intelligent Information Systems, 253-265.
    [2] Ying, H., Zhuang, F., Zhang, F., Liu, Y., Xu, G., Xie, X., Wu, J. (2018). Sequential Recommender System based on Hierarchical Attention Networks. Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence.
    [3] Huang, X., Qian, S., Fang, Q., Sang, J., & Xu, C. (2018). Csan. 2018 ACM Multimedia Conference on Multimedia Conference - MM '18.
    [4] Gao C., He X., Gan D., Chen X., Feng F., Li Y., & Chua. T. (2018). Learning recommender systems from multi-behavior data. arXiv preprint arXiv:1809.0816.
    [5] Greenstein-Messica, A., Rokach, L., & Friedman, M. (2017). Session-Based Recommendations Using Item Embedding. Proceedings of the 22nd International Conference on Intelligent User Interfaces.
    [6] Pennington, J., Socher, R., & Manning, C. (2014). Glove: Global Vectors for Word Representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP).
    [7] Mikolov T., Chen K., Corrado G., & Dean J.. (2013). E cient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
    [8] Barkan, O., & Koenigstein, N. (2016). ITEM2VEC: Neural item embedding for collaborative filtering. 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP).
    [9] Wu, C., & Yan, M. (2017). Session-aware Information Embedding for E-commerce Product Recommendation. Proceedings of the 2017 ACM on Conference on Information and Knowledge Management.
    [10] https://medium.com/@madhuramiah/bi-directional-rnn-basics-of-lstm-and-gru-
    e114aa4779bb
    [11] Bahdanau D., Cho K., & Bengio Y. (2015). Neural machine translation by Jointly Learning to Align and Translate. - ICLR.
    [12] Li, J., Ren, P., Chen, Z., Ren, Z., Lian, T., & Ma, J. (2017). Neural Attentive Session- based Recommendation. Proceedings of the 2017 ACM on Conference on Information and Knowledge Management.
    [13] Vaswani A., Shazeer N., Parmar N., Uszkoreit J., & Jones L. (2017) Attention is all you need. – ICONIP.
    [14] Zhang S.,Tay Y., Yao L.,& Sun A. (2018). Next item recommendation with self- attention -.arXiv:Information Retrieval.
    [15] Liu, Q., Zeng, Y., Mokhosi, R., & Zhang, H. (2018). Stamp. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.

    [16] Ren, P., Chen, Z., Li, J., Ren, Z., Ma, J., & Rijke, M. D. (2019). RepeatNet: A Repeat Aware Neural Recommendation Machine for Session-Based Recommendation. Proceedings of the AAAI Conference on Artificial Intelligence, 33, 4806-4813.
    [17] Sachdeva, N., Gupta, K., & Pudi, V. (2018). Attentive neural architecture incorporating song features for music recommendation. Proceedings of the 12th ACM Conference on Recommender Systems
    [18] Bai, T., Nie, J., Zhao, W. X., Zhu, Y., Du, P., & Wen, J. (2018). An Attribute-aware Neural Attentive Model for Next Basket Recommendation. The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval.
    [19] Kang, W., & Mcauley, J. (2018). Self-Attentive Sequential Recommendation. 2018 IEEE International Conference on Data Mining (ICDM).
    [20] Sun, F., Liu, J., Wu, J., Pei, C., Lin, X., Ou, W., & Jiang, P. (2019). BERT4Rec.
    Proceedings of the 28th ACM International Conference on Information and Knowledge
    Management.
    [21] Huffer, K. M., & Reed, J. W. (2017). Situational awareness of network system roles
    (SANSR). Proceedings of the 12th Annual Conference on Cyber and Information
    Security Research - CISRC '17.
    [22] Jannach, D., & Ludewig, M. (2017). When Recurrent Neural Networks meet the Neighborhood for Session-Based Recommendation. Proceedings of the Eleventh ACM Conference on Recommender Systems.
    [23] Hidasi B., Karatzoglou A., Baltrunas L., & Tikk D. (2016) Session-based recommendations with recurrent neural networks. - ICLR ’16.
    [24] Tang, J., & Wang, K. (2018). Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding. Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining - WSDM '18.
    [25] Chen, M., Liu. Y., & Yu. X. (2014). NLPMM: A Next Location Predictor with Markov Modeling. Procedding of Advances in Knowledge Discovery and Data Mining Lecture Notes in Computer Science (PAKDD 2014), pp. 186–197.
    [26] Gambs, S., Killijian, M., & Cortez, M. (2012). Next place prediction using mobility Markov chains. Proceedings of the First Workshop on Measurement, Privacy, and Mobility (MPM 2012), no. 3, pp. 1-6.
    [27] Cheng, C., Yang, H., Lyu, M., &King., I. (2013). Where you like to go next: successive point-of-interest recommendation. Proceedings of the 23th international joint conference on Artificial Intelligence (IJCAI 2013), pp. 2605-2611.
    [28] He, J., Li, X., Liao, L., Song, D., & Cheung, K. (2016). Inferring a personalized next point-of-interest recommendation model with latent behavior patterns. Proceedings of the 13th AAAI Conference on Artificial Intelligence (AAAI 2016), pp. 137-143.
    [29] Zhang, J., & Chow, C. (2015). Spatiotemporal Sequential Influence Modeling for Location Recommendations. ACM Transactions on Intelligent Systems and Technology (TIST 2015), vol. 7, no. 1, pp. 1–25.
    [30] Mehta, R. & Rana, K. (2107). A review on matrix factorization techniques in recommender systems. The 2nd International Conference on Communication Systems, Computing and IT Applications (CSCITA 2017), pp. 269-274.
    [31] Xiong, L., Chen, X., Huang, T., Schneider, J. & Carbonell, J. (2010). Temporal Collaborative Filtering with Bayesian Probabilistic Tensor Factorization. Proceedings of the 2010 SIAM International Conference on Data Mining (SDM 2010), pp. 211-222.
    [32] Rendle, S., Freudenthaler, C. & Schmidt-Thieme, L. (2010). Factorizing personalized Markov chains for next-basket recommendation,” Proceedings of the 19th international conference on World Wide Web (WWW 2010), pp. 811-820.
    [33] Fu, Y., Liu, B., Ge, Y., Yao, Z. & Xiong, H. (2014). User Preference Learning with Multiple Information Fusion for Restaurant Recommendation. Proceedings of the 2014 SIAM International Conference on Data Mining.
    [34] Liu, Y., Liu, C., Liu, B., Qu, M. & Xiong, H. (2016). Unified Point-of-Interest Recommendation with Temporal Interval Assessment,” Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
    [35] Liu, B., Xiong, H., Papadimitriou, S., Fu, Y. & Yao, Z. (2015). A General Geographical Probabilistic Factor Model for Point of Interest Recommendation,” IEEE Transactions on Knowledge and Data Engineering, vol. 27, no. 5, pp. 1167–1179.
    [36] T. Qian, B. Liu, Q. V. H. Nguyen, & H. Yin, (2019). Spatiotemporal Representation Learning for Translation-Based POI Recommendation,” ACM Transactions on Information Systems, vol. 37, no. 2, pp. 1–24.
    [37] Wang, W., Yin, H., Du, X., Nguyen, H. & Zhou, X. (2018). TPM. ACM Transactions on Intelligent Systems and Technology, vol. 9, no. 6, pp. 1–25.
    [38] Wang, W., Yin, H., Sadiq, S., Chen, L., Xie, M. & Zhou, X. (2016). SPORE: A sequential personalized spatial item recommender system. IEEE 32nd International Conference on Data Engineering.
    [39] Yin, H., Cui, B., Zhou, H., Wang, W., Huang, Z. & Sadiq, S. (2016). Joint Modeling of User Check-in Behaviors for Real-time Point-of-Interest Recommendation. ACM Transactions on Information Systems, vol. 35, no. 2, pp. 1–44.
    [40] Yin, H., Wang, W., Wang, H., Chen, L. & Zhou, X. (2017). Spatial-Aware Hierarchical Collaborative Deep Learning for POI Recommendation,” IEEE Transactions on Knowledge and Data Engineering, vol. 29, no. 11, pp. 2537–2551.
    [41] Cheng, H., Ispir, M., Anil, R., Haque, Z., Hong, L., Jain, V., Liu, X., Shah, H., Koc, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., Anderson, G., Corrado, G. & Chai, W. (2016). Wide & Deep Learning for Recommender Systems. Proceedings of the 1st Workshop on Deep Learning for Recommender Systems (DLRS 2016), pp. 7-10.
    [43] Manotumruksa, J., Macdonald, C., & Ounis, I. (2017). A Deep Recurrent Collaborative Filtering Framework for Venue Recommendation. Proceedings of the 2017 ACM on Conference on Information and Knowledge Management (CIKM 2017), pp. 1429-1438.
    [44] Feng, J., Li, Y., Zhang, C., Sun, F., Meng, F., Guo, A. & Jin, D. (2018). DeepMove: Predicting Human Mobility with Attentional Recurrent Networks. Proceedings of the 2018 World Wide Web Conference on World Wide Web (WWW 2018), pp. 1459-1468.
    [45] Manotumruksa, J., Macdonald, C., & Ounis, I. (2018). A Contextual Attention Recurrent Architecture for Context-Aware Venue Recommendation,” Proceedings of the 41st Internaional ACM SIGIR Conference on Research & Development in Information Retrieval (SIGIR 2018), pp. 555-564.
    [46] Zhao, S., Lyu, M. & King, I. (2018). Geo-Teaser: Geo-Temporal Sequential Embedding Rank for POI Recommendation. Springer Briefs in Computer Science Point-of-Interest Recommendation in Location-Based Social Networks, pp. 57–78.
    [47] Zhu, Y., Li, H., Liao, Y., Wang, B., Guan, Z., Liu, H. & Cai, D. (2017). What to Do Next: Modeling User Behaviors by Time-LSTM,” Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI 2017), pp. 3602-3608.
    [48] Liu, Q., Wu, S., Wang, L. & Tan, T. (2016). Predicting the next location: a recurrent model with spatial and temporal contexts,” Proceedings of the 13th AAAI Conference on Artificial Intelligence (AAAI 2016), pp.194-200.
    [49] Al-Molegi, A., Jabreel, M. & Ghaleb, B. (2016). STF-RNN: Space Time Features-based Recurrent Neural Network for predicting people next location. IEEE Symposium Series on Computational Intelligence. pp. 1-7.
    [50] Yao, D., Zhang, C., & Huang, J. (2017). Serm: A recurrent model for next location prediction in semantic trajectories. Proceedings of the 2017 ACM on Conference on Information and Knowledge Management.
    [51] Sarwar, B., Karypis, G., Konstan, J., Reidl, J. (2001). Item-based collaborative filtering recommendation algorithms. Proceedings of the Tenth International Conference on World Wide Web.
    [52] Li, G., Wang, L., Ou, W. (2015). Robust Personalized Ranking from Implicit Feedback. International Journal of Pattern Recognition and Artificial Intelligence, 30(01), 1659001.
    [53] Gu, W., Dong, S., Zeng, Z. (2014). Increasing recommended effectiveness with markov chains and purchase intervals. Neural Computing and Applications, 25(5), 1153-1162.
    [54] Kapetanakis, S., Polatidis, N., Alshammari, G., Petridis, M. (2019). A novel recommendation method based on general matrix factorization and artificial neural networks. Neural Computing and Applications, 32(16), 12327-12334.
    [55] Zhou, J. P., Cheng, Z., Perez, F., Volkovs, M. (2020). TAFA: Two-headed Attention Fused Autoencoder for Context-Aware Recommendations. Fourteenth ACM Conference on Recommender Systems.
    [56] He, R., Mcauley, J. (2016). Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation. 2016 IEEE 16th International Conference on Data Mining (ICDM).
    [57] Hidasi, B., Karatzoglou, A. (2018). Recurrent Neural Networks with Top-k Gains for Session-based Recommendations. Proceedings of the 27th ACM International Conference on Information and Knowledge Management.
    [58] Tan, Y. K., Xu, X., Liu, Y. (2016). Improved Recurrent Neural Networks for Session-based Recommendations. Proceedings of the 1st Workshop on Deep Learning for Recommender Systems.
    [59] Quadrana, M., Karatzoglou, A., Hidasi, B., Cremonesi, P. (2017). Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks. Proceedings of the Eleventh ACM Conference on Recommender Systems.
    [60] Sakar, C. O., Polat, S. O., Katircioglu, M., Kastro, Y. (2018). Real-time prediction of online shoppers’ purchasing intention using multilayer perceptron and LSTM recurrent neural networks. Neural Computing and Applications, 31(10), 6893-6908.
    [61] Xu, H., Jiang, C. (2019). Research on context-aware group recommendation based on deep learning. Neural Computing and Applications, 32(6), 1745-1754.
    [62] Zhang, H., Sun, Y., Zhao, M., Chow, T., Wu, Q. (2020). Bridging User Interest to Item Content for Recommender Systems: An Optimization Model. IEEE Transactions on Cybernetics, 50(10), 4268-4280.
    [63] Zhang, H., Huang, W., Liu, L., Chow, T. (2020). Learning to Match Clothing From Textual Feature-Based Compatible Relationships. IEEE Transactions on Industrial Informatics, 16(11), 6750-6759.
    [64] Sornalakshmi, M., Balamurali, S., Venkatesulu, M., Krishnan, M. N., Ramasamy, L. K., Kadry, S., Muthu, B. A. (2020). Hybrid method for mining rules based on enhanced Apriori algorithm with sequential minimal optimization in healthcare industry. Neural Computing and Applications.
    [65] Hansen, C., Hansen, C., Maystre, L., Mehrotra, R., Brost, B., Tomasi, F., Lalmas, M. (2020). Contextual and Sequential User Embeddings for Large-Scale Music Recommendation. Fourteenth ACM Conference on Recommender Systems.
    [66] Lin, J., Pan, W., Ming, Z. (2020). FISSA: Fusing Item Similarity Models with Self-Attention Networks for Sequential Recommendation. Fourteenth ACM Conference on Recommender Systems.
    [67] Li, Z., Zhao, H., Liu, Q., Huang, Z., Mei, T., Chen, E. (2018). Learning from History and Present. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
    [68] Kang, K., Park, J., Kim, W., Choe, H., Choo, J. (2019). Recommender System Using Sequential and Global Preference via Attention Mechanism and Topic Modeling. Proceedings of the 28th ACM International Conference on Information and Knowledge Management.
    [69] Huang, J., Ren, Z., Zhao, W. X., He, G., Wen, J., Dong, D. (2019). Taxonomy-Aware Multi-Hop Reasoning Networks for Sequential Recommendation. Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining.
    [70] Afchar, D., Hennequin, R. (2020). Making Neural Networks Interpretable with Attribution: Application to Implicit Signals Prediction. Fourteenth ACM Conference on Recommender Systems.
    [71] Kingma, D. P., Jimmy, B. (2017). Adam: A Method for Stochastic Optimization.
    [72] Abdi, M., Okeyo, G., & Mwangi, R. (2018). Matrix Factorization Techniques for Context-Aware Collaborative Filtering Recommender Systems: A Survey. Computer and Information Science, vol. 11, no. 2.
    [73] Graves, A. (2012). Supervised Sequence Labelling. Studies in Computational Intelligence Supervised Sequence Labelling with Recurrent Neural Networks, pp. 5–13.
    [74] Zhang, Y., Dai, H., Xu, C., Feng, J., Wang, T., Bian, J., Wang, B., & Liu, T. (2014). Sequential click prediction for sponsored search with recurrent neural networks. Proceedings of the 28th AAAI Conference on Artificial Intelligence (AAAI 2014), pp. 1369-1375.
    [75] Ye, M., Yin, P., Lee, W., & Lee, W. (2011). Exploiting geographical influence for collaborative point-of-interest recommendation,” Proceedings of the 34th international ACM SIGIR conference on Research and development in Information (SIGIR 2011), pp. 325-334.
    [76] Zhang, B. & Feng, Y. (2016). Improving Temporal Recommendation Accuracy and Diversity via Long and Short-Term Preference Transfer and Fusion Models,” Web Technologies and Applications Lecture Notes in Computer Science, pp. 174–185.
    [77] Waters, J. (2010). All You Need to Know About Participating in Today’s Most Popular Online Communities, pp. 202-211.
    [78] Cho, E., Myers, S., & Leskovec, J. (2011). Friendship and mobility: user movement in location-based social networks. Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining (KDD 2011), pp. 1082-1090.
    [79] He, X., He, Z., Song, J., Liu, Z., Jiang, Y., Chua, T. (2018). NAIS: Neural Attentive Item Similarity Model for Recommendation. IEEE Transactions on Knowledge and Data Engineering, 30(12), 2354-2366.
    [80] Wu, Y., Li, K., Zhao, G., Qian, X. (2020). Personalized Long- and Short-term Preference Learning for Next POI Recommendation. IEEE Transactions on Knowledge and Data Engineering.
    [81]
    Silveira, T., Zhang, M., Lin, X., Liu, Y., Ma, S. (2017). How good your recommender system is? A survey on evaluations in recommendation. International Journal of Machine Learning and Cybernetics, 10(5), 813-831.
    [82] Chen, Y., Thaipisutikul, T., Shih, T. K. (2020). A Learning-Based POI Recommendation With Spatiotemporal Context Awareness. IEEE Transactions on Cybernetics, 1-14.

    [83] Zhou J., Cui G., Zhang Z., Yang C., Liu Z., Sun M. (2018). Graph neural networks: A review of methods and applications on arXiv:1812.08434.
    [84] Wu, S., Tang, Y., Zhu, Y., Wang, L., Xie, X., Tan, T. (2019). Session-Based Recommendation with Graph Neural Networks. Proceedings of the AAAI Conference on Artificial Intelligence, 33, 346–353.
    [85] Abdollahi, B., Nasraoui, O. (2017). Using Explainability for Constrained Matrix Factorization. Proceedings of the Eleventh ACM Conference on Recommender Systems.
    [86] Mathur, N., Baldwin, T., Cohn, T. (2020). Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.
    [87] Lin, C.-Y., Hovy, E. (2002). Manual and automatic evaluation of summaries. Proceedings of the ACL-02 Workshop on Automatic Summarization.
    [88] Fang, H., Zhang, D., Shu, Y., Guo, G. (2021). Deep Learning for Sequential Recommendation. ACM Transactions on Information Systems, 39(1), 1–42.

    QR CODE
    :::