Recommender Systems

The Recommender Systems course at Leiden University ran as a series of build it yourself assignments, each about predicting what a user will interact with next, ending in a class contest. My group of three worked through all of them, reimplementing a few well known models from scratch and then putting what we had learned into the contest.

Links · 💻 Code · 🤗 Checkpoints · 📄 Neural CF · 📄 SASRec · 📄 TIGER · 📄 Contest

The models

We started with Neural Collaborative Filtering, a neural take on matrix factorisation, as a baseline on MovieLens, where a careful ablation showed the negative sampling ratio mattered more than any of the fancier architecture choices. Then SASRec, a self attentive sequential recommender that predicts the next item with a causal Transformer, which we evaluated the honest way by ranking against the full catalogue rather than a handful of sampled negatives. Then TIGER, a stranger idea where each item is turned into a short semantic id by a residual quantiser and the model generates the next item’s id token by token. SASRec and TIGER are reimplementations of published work, and both land a good way below the original papers’ numbers because we used the harder evaluation and a small compute budget, and the reports say so plainly.

The contest

Here the lesson was that simple and robust beats clever. Our best submission fuses two sequential models, SASRec and BERT4Rec, by reciprocal rank fusion, and that plain blend scored higher than every heavier thing we tried, including our own TIGER, graph models and gradient boosted rerankers, all of which looked great on our own validation set and then dropped on the real leaderboard. The final submission landed at 0.02013, just short of the top score of 0.02126.