Better youtube recommendations
- More autonomy when choosing what to watch
- Very customizable
- Recommendations for two or more people
- You can install it locally for completely private recommendations
- Sharing and browsing information bubbles
- Less clickbait
How does it work?
- For every video you have liked on youtube, its recommended videos are collected.
- This way, we create a graph, where two videos are connected if one recomends the other.
- Now we divide this graph into clusters (groups of videos around common theme). For example we can have a
folk rockcluster, or ascience podcastscluster or atravel vlogscluster. - Small clusters are a part of larger clusters. For example
folk rockandboomer rockare inside ofrockcluster, androckis insidemusic. - This forms a tree, with big branches (like
music), splitting into smaller and smaller branches and twigs. - Now, to choose what to watch you can start at the trunk, and "climb" this tree, by choosing which branch to go into.
- Note, that some clusters are too big to be clearly labeled, but by looking at the videos in them, you can usually get a general idea about this cluster's theme.
How are the clusters found?
- When a group of videos is densely connected, it's assumed do be a cluster. When two clusters are well connected, they are joined into a bigger cluster. The exact method we use is here.
Why rely on youtube recommendations instead of providing our own and having more control over them?
- Creating a recommender system from scratch is much harder than you may think at first. In addition to having accurate recommendations, you also have to defend against attacks, like click farms trying to boost some content, or intelligence agencies spreading misinformation. You also have to detect illegal or NSFL stuff, and filter it out. See this to get a sense of how hard this is.
- Another critical problem is the network effect. To build a good recommender system, we need data from a lot of users. To have a lot of users, we need a good recommender system.
- For these reasons, it's better to start with an existing recomender system as a "bottom layer", and then build any new features we want, on top of it.
git clone https://github.com/filyp/yourtube.git
cd yourtube
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[fast]"Fetch your liked and watch later playlists using yt-dlp (uses browser cookies for authentication):
.venv/bin/yourtube-fetch-playlists
.venv/bin/yourtube-scrapePlaylist data will be saved to ~/.yourtube/playlists/.
.venv/bin/yourtubeYourTube should be now available at: http://localhost:8866/
To automatically fetch playlists and scrape weekly, enable the systemd user timer:
ln -s "$(pwd)/yourtube-update.service" ~/.config/systemd/user/
ln -s "$(pwd)/yourtube-update.timer" ~/.config/systemd/user/
systemctl --user enable --now yourtube-update.timerThe timer uses Persistent=true, so it will catch up if your machine was off during the scheduled time.
To check the timer status:
systemctl --user status yourtube-update.timer
systemctl --user list-timersIt's fun to share some interesting clusters you find with your friends. To do this, set up a sync of your ~/.yourtube/saved_clusters/ folders, for example using Syncthing.
The saved clusters are self-contained, so others can open them even if they never scraped the videos from that cluster.
You can also go a step further and sync your ~/.yourtube/videos folders, which will merge your recommendation graphs.