Are there any readers who do not visit YouTube at least once a day? Service has already become an integral part of our lives. It is hard to imagine that we will use something else to watch videos. YouTube offers a huge content base. In total, the service has 1.9 billion active users every month. According to statistics, 79% of Internet users have a YouTube account. So how does Google manage to keep such a huge product running and how the youtube algorithm works? In this article, we will look at the principle of the YouTube suggestions algorithm, and it is interesting, believe me.
This material will be based on the official Google publication , which explains the principle of operation of YouTube recommendation algorithms based on neural networks. Why did I decide to study this issue? The fact is that not so long ago, before going to bed, I decided to turn on the sounds of the waterfall (white noise) in order to fall asleep faster. The next evening at the same time, I noticed that the same video was in the very first place in the recommendations. I turned it on again. On the third day at the same time, this video was again in the same first position. And this despite the fact that at any other time YouTube recommends completely different videos to me.
And then I finally realized that YouTube algorithms work much more complicated than we think. At a minimum, they are able to adapt to your preferences at different times of the day. Then I decided to study how YouTube algorithms work and came across some interesting information that I am ready to share with readers.
YouTube developers faced several problems when developing the algorithm:
- A huge number of videos in various topics, which complicates the optimal selection in the recommendations
- High dynamics of service. Hundreds to thousands of hours of videos are uploaded to YouTube every hour. The recommendation system needs to be flexible and dynamic
- The inconsistency of the interests of the audience
- Optimization of resources for the selection of recommendations, since the operation of selection algorithms is a complex process that requires a lot of power
YouTube recommender architecture
Millions of videos are submitted to enter the system, and at the exit it offers the same dozens of videos that hit the user on the screen in the “Recommendations” tab.
The system consists of two convolutional neural networks: “candidate generation” and “ranking” (ranking). The first network of millions of videos selects hundreds of the most relevant ones, the second neural network ranks the resulting selection from the most to the least interesting to the user. When compiling a sample, the system takes into account the entire history of the user and the context. Context refers to, for example, time of day, age, gender, geographic location. Also, at the time of creating the sample, A / B testing occurs, when for the sake of experiment, the user is shown different samples, if any of the samples is more viewable, the system self-learns and adapts to this sample.
When evaluating the sample, not only the viewing time is taken into account, but also the CTR (click through rate) – the number of users who started watching the video in relation to the number of users who saw the video in the recommendations.
At the ranking stage, the selection is based on the expected watch time indicator, so the longer users watch a video, the higher the chance that it will get into the top recommendations. YouTube is not just based on click through rate, as a video can be just clickbait. The goal of training the ranking neural network is to predict the video viewing time.
Conclusion
YouTube recommendations are formed from two neural networks. The first neural network is responsible for selecting videos on the topic, the second-level neural network among the selected ones cuts off clickbait and uninteresting videos with low user engagement. That is why videos that are watched longer, more often likes and comments get to the very first places in the recommendations, if they correspond to topics that are of interest to the user.
