Consistency of expert product reviews: an application to wine guides

Abstract

Purpose. The purpose of this study is to examine the internal consistency of wine guides by comparing the judgements of experts who taste and review wines. It further classifies the wines into categories to establish whether expert reviews of similar wines are coherent. Design/methodology/approach. Sentiment analysis based on natural language processing techniques was used to compare quantitative and qualitative reviews between experts. In addition, a finite mixture model was used to classify wines into categories to analyse internal consistency between ratings. Findings. The results for a sample of more than 200,000 Wine Enthusiast ratings reveal significant differences between the experts’ reviews, which shows that there are no standard criteria for reviewing the wines included in the guide. Originality. Wine guides are amongst the most widely used marketing resources in the wine industry. They provide a signal to consumers about the quality of wines, guiding their purchase decisions, whilst also influencing the reputation of brands and the performance of the companies producing these wines. Thus, the main contribution of this study is to improve comparisons of the assessments by wine guide experts in relation to the wines they taste. This analysis will make it possible to verify their effectiveness as a source of information for consumers and will allow wine producers to make more accurate assessments of the potential impact of experts on their performance.