Collaborative Filtering Part 3: Neural Networks
by Yifei Zhou
1. Introduction
In recent years, neural networks have shown significant advantages in many machine learning domains, such as image recognition, speech recognition, and natural language processing. Some researchers also tried to apply neural networks in recommender systems. Neural networks are a big concept in the deep learning domain. Based on the purpose of machine learning, neural networks can be divided into different categories, including Densed Neural Networks (DNNs), Convolutional Neural Networks (CNNs), Graph Neural Networks (GNNs), and Recurrent Neural Networks (RNNs), etc.
The structure of densed neural networks (DNNs) |
In this post, I mainly focus on which are most correlated to the recommendation task, and see how neural networks perform a recommendation task. The figure above shows the structure of DNNs.
2. Multi-Layer Perceptron (MLP)
He proposed a Neural Collaborative Filtering (NCF) framework in 2017, and the core idea is to model the users and items as the one-hot encoding to the input layer, and outputs a probality as prediction score. The general framework is shown in figure below.
Multi-Layer Perceptron (MLP) is a good illustration of DNNs as a typical usage for a classification task. However, most recommendation tasks are unsupervised learning. What is the connection between a classification task and an unsupervised learning task? The NCF framework provides a bridge to integrate them. In this case, we do not build a user-item rating matrix like model-based collaborative filtering methods. Both positive and negative instances are required in a classification task. How do we generate those positive and negative samples in a recommendation task? He treated each sample (one user-item interaction) in the database as a positive instance and paired it with four unseen samples randomly selected by the user as negative instances. These data are prepared as the input for the training process.
Neural Collaborative Filtering (NCF) Framework |
To predict whether a user likes an item, we just input the user's and the item's one-hot encoding vector into the neural networks and compare the output probability. If the output probability is more significant than 0.5, that user will probably like this item; Otherwise, not like it.
3. Generalized Matrix Factorization (GMF)
As I mentioned in previous blog, matrix factorization is a type of method in model-based collaborative filtering. In fact, the matrix factorization can also be conducted by neural networks. We just remove the deep hidden layers in the MLP and only keep the embedding layer and output layer, and extract the weights from the embedding layer as the user-latent and item-latent feature vectors.
4. Auto-Encoders
Besides, auto-encoders can also be applied in recommendation tasks. As we know, an autoencoder is a particular type of neural network that simulates the output like the input. Autoencoders have been widely used to recognize handwritten digits. In other words, the autoencoder encoders an image into a lower dimensional latent space and then decodes the latent spaces back to the image. In 2016, Wu et al. argued an approach, 'Collaborative Denosing Auto-encoders', that comprises the database into autoencoders. In this framework, the number of input and output nodes is the same as the number of item nodes in the database. In other words, each row of the user-item matrix is represented as the input for each user, where the non-zero entries are the inputs for the items, and the output is the same as the input for that user. Therefore, the whole user-item matrix is used to fit for input and output in the training process.
In addition, we can also conduct it in a column-based way, which means the each item-vector is used for input and output.
Collaborative Denosing Auto-Encoders |
Conclusion
In conclusion, we can see that the neural networks can be applied effectively and flexibly to a recommendation task. The methods outlined in this blog are the most commonly used in recommendation tasks. Besides, some complicated methods also integrate with other machine learning algorithms, like 'Deep Matrix Factorization'. In short, neural networks are a good practice for performing a recommendation task. However, the training time will be explosive with the increasing number of parameters. On the other hand, there is also a potential risk of overfitting if the model structure is too complicated. In the next post, I will look at the recommendation task at an angle of the graph and see how we use graph-based algorithms to solve a recommendation task.
References
-
He, Xiangnan, et al. “Neural collaborative filtering.” Proceedings of the 26th international conference on world wide web. 2017.
-
Wu, Yao, et al. “Collaborative denoising auto-encoders for top-n recommender systems.” Proceedings of the ninth ACM international conference on web search and data mining. 2016.