Closing the Gap in Online Personalized Recommender Systems

in Digital Technology Sector, Emerging Research, Information Systems, News
January 23rd, 2013

From “A Hidden Markov Model for Collaborative Filtering,” MIS Quarterly, 36(4), 1329-1356.

Nachiketa SahooCommercial websites are constantly suggesting new products and content to us—a mechanized, cyber-age form of the old urging, “if you liked that, you’ll love this!” In tech terms, the systems that generate these suggestions are called personalized recommender systems. But how can these computer systems account for the age-old human tendency to change our desires as time goes on?

A new study by Boston University’s Nachiketa Sahoo and co-authors Param Vir Singh and Tridas Mukhopadhyay is one of the first to address this problem.

Sahoo is an assistant professor in information systems at Boston University School of Management; Singh and Mukhopadhyay are faculty members at the David A. Tepper School of Business at Carnegie Mellon University. Their paper, “A Hidden Markov Model for Collaborative Filtering,” appearing in MIS Quarterly‘s special issue on business intelligence research, suggests using a stochastic algorithm called a hidden Markov model (HMM) to process data about user activity and preferences, rather than the common algorithms used now by most personalized recommender systems. The authors show that the HMM, a more dynamic model, allows online personalized recommender systems to account for changing user preferences.

A New Model to Address Changing User Preferences

The authors point out that dynamic, not static, user tastes and desires are integral to the consumer experience, particularly with the repeat consumption of so-called “experience goods,” such as movies, music, and news. “This causes problems for a recommender system that has been trained to identify customers’ preferences from their past ratings of products,” the authors write.

Sahoo et al. propose a customized HMM algorithm to estimate user preferences and make recommendations. They evaluate their approaches using three real-world datasets: one containing employees’ blog reading activity in a Fortune 500 IT services firm, one documenting users’ movie watching behavior in the Netflix Prize dataset, and one tracking users’ music listening behavior on last.fm. Comparing the performance of their algorithm with that of several other popular algorithms in recommender systems, the authors show that the HMM-based algorithm performs as well or better than the other algorithms, particularly as user preferences change.

Their approach is based on the intuition that older data, rather than being discounted—as they are in some current personalized recommender systems—could instead be used to learn about that user’s preference and then applied to another user. “Data from a user’s past may not be useful for making recommendation for the user now,” they argue, since “her preference has changed, but it might be useful for making a recommendation for someone who currently has that preference.”

Read more about ”A Hidden Markov Model for Collaborative Filtering.”

Banner photo is a visualization of related movies found by a computer algorithm created for Netflix Prize. Each movie is represented by a dot, and colored lines signify a similarity between pairs. Photo courtesy of flickr user chef_ele.