Expected Reciprocal Rank (ERR) is a metric used to evaluate the performance of information retrieval systems, particularly in the context of search engines and recommendation systems. ERR quantifies how well a system ranks relevant documents or items, taking into account the likelihood that users will find what they are looking for based on the ranking order.
The core idea behind ERR is to model user behavior when interacting with search results. It is based on the assumption that users are more likely to select items appearing at the top of a ranked list. The metric incorporates the probability of a user examining each rank position and the likelihood of finding the relevant item. This is done using a concept called the reciprocal rank, which is the inverse of the rank at which the first relevant item is found.
To calculate ERR, the ranking of items is assessed, and the expected value of the reciprocal rank is computed. This involves assigning a score to each rank position based on the probability of the user stopping at that position to examine the result. The formula for ERR can be expressed as:
ERR = Σ (P(i) * (1 / rank(i)))
where P(i) is the probability that a user finds the relevant item at rank i, and rank(i) is the position in the ranked list.
ERR is particularly valuable because it accounts for both the position of relevant items in the ranking and the likelihood of user engagement, making it a more nuanced measure than traditional metrics like Precision or Recall. By focusing on the user experience and satisfaction, ERR helps developers fine-tune their systems to deliver better results.