What are recommendation systems?

Recommendation systems are filtering systems used on digital platforms. Sometimes called recommender systems, these are the mechanisms that suggest movies or shows to watch on Netflix, books to read on Amazon, and people to follow on Instagram based on your user preferences, purchases, or viewing history.

The use of recommendation systems helps brands that offer products or services to learn more about users and also helps new users to navigate platforms and apps more easily. If a user is confused by what’s on offer and can’t find what they like or want, the chances are greater that they’ll quit the app or won’t renew their membership. E-commerce relies on metrics such as purchase history and browsing history to suggest similar items to customers.

The recommendation engine also considers user ratings and reviews as feedback to gauge what a user likes and dislikes. This level of analysis based on information that the user offers simply by using an app or website is raising some concern over privacy. There are questions around whether recommender and other content decision systems undermine individual privacy because the technique relies upon collecting and processing personal information and creating user profiles.

Recommender system regulations

From March 1st 2022, China enforced recommender system regulations known in English as Internet Information Service Algorithmic Recommendation Management Provisions. It’s the first case of a major economy implementing blanket rules around how the internet is managed. 

However, it’s part of a broader initiative started by the Chinese government in 2020, introducing laws on data privacy, data security, and antitrust among others. As Cambrian AI analyst Alberto Romero wrote in the wake of the restrictions being imposed, “Imagine a world in which Facebook couldn’t show you what it wanted – optimized to keep you engaged – but what you wanted or needed.” One website which does exist to show people the information that they want or need is MovieLens, a site for “non-commercial personalized movie recommendations”. It consists of a virtual community of members who allow their reviews and ratings to be used by the site’s recommender system.

Recommendation models were likely created with good intentions for an optimised user experience. But as Romero highlights in his article In Search of an Algorithm for Well-being they have become less about engagement and more about addiction. Similarly, in the 2020 documentary The Social Dilemma, statistician Edward Tufte contemplates the fact that, “There are only two industries that call their customers ‘users’: illegal drugs and software.” To add to this, an online landscape created by collaborative filtering recommenders is as the name suggests, “filtered”. This risks a biassed perception of reality as has been seen in the culture wars that have taken root throughout social media and spilled into the real world in the past few years. Romero quotes tech ethics expert Gemma Galdón on the matter, who says, “We have allowed the tech industry a very anomalous space of non-accountability in our society. And it must be subjected to the same controls as any innovation space that surrounds us.”

The Cold Start Problem is a non-fiction book by Andrew Chen that was published in December 2021. It explores the “cold start problem” of scalability when a product or service relies on network effects by interviewing founding members of LinkedIn, Twitch, Zoom, Dropbox, Tinder, Uber, Airbnb, Pinterest, and more. Interestingly, Chen says that “The technology ecosystem is downright hostile to new products – competition is fierce, copycats abound, and marketing channels are ineffective.” This raises questions about what happens when a small selection of big tech companies dominate the space.

How does a recommendation system work?

Recommendation algorithms use a machine learning technique called collaborative filtering. They can be built using the Python library, TensorFlow Ranking. Collaborative filtering methods rely on user behaviour and feedback, from the links the user clicks on to what they read, watch, and view. This helps build the algorithm that then suggests recommended items or content. A collaborative filtering recommender compares the behaviour of similar users as well to ascertain the best suggestions for any one user. This creates a similarity score amongst different users. Essentially, a number of users within the dataset are helping ‘vote’ on what a similar user might like to read, watch, or purchase based on their own and shared online interests and activities.

Matrix factorisation is a class of collaborative filtering approaches that is considered extremely effective for recommendation systems. Funk MF is the original algorithm proposed by Simon Funk. It factorises the user-item rating matrix as the product of two lower dimensional matrices, the first one being a row for each user, while the second has a column for each item. In practice, this enables a system to gauge the customer’s exact purpose for purchasing new items, scan data within pages, and then shortlist and rank product recommendations that meet the customer’s requirements.

Depending on settings and the terms and conditions that the user has agreed to by using an app or the site, an API (application programming interface) may communicate in real time with other apps or sites to share information. The recent prevalence of open APIs and open data sources in big data has provided the opportunity for mashup development using matrix factorisation. Data can be read by systems from embeddings and metadata that helps to create classes and subsets which can then be utilised in matrix factorisation.

Help further data science with a master’s

Artificial intelligence can help humans to do many things but outputs can only be as effective as their inputs. Using data analytics strategically and ethically can offer insights that inform how we train computers and neural networks to benefit society. Recommendation systems are a case in point – they can either take up our time and attention or they can offer up what we want when we want it. 

Ready to disrupt the ways that things are done? Discover the fundamentals of deep learning and data mining with a 100% online MSc Computer Science with Data Analytics from Keele University.