AI and Analytics for Business

Research Paper Series

Sequential Search with Refinement: Model and Application with Click-Stream Data

We propose a structural model of consumer sequential search under uncertainty about attribute levels of products. Our identification of the search model relies on exclusion restriction variables that separate consumer utility and search cost. Because such exclusion restrictions are often available in online clickstream data, the identification and corresponding estimation strategy is generalizable for many online shopping websites where such data can be easily collected. Furthermore, one important feature of online search technology is that it gives consumers the ability to refine search results using tools such as sorting and filtering based on product attributes. The proposed model can coherently integrate consumers’ decisions of search and refine. The model is instantiated using consumer clickstream data of online hotel bookings provided by a travel website. The results show that refinement tools have significant effects on consumer behavior and market structure. We find that the refinement tools encourage 34% more searches and enhance the utility of purchased products by 18%. However, most websites by default rank search results according to their qualities or relevance to consumers (e.g., Google). When consumers are unaware of such default ranking rules, they may engage in disproportionately more searches using refinement tools. Consequently, overall welfare surplus may deteriorate when search cost outweighs the enhanced utility. In contrast, if the website simply informs consumers that the default ranking already reflects product quality or relevance, consumers search less and the welfare surplus improves. We also find that refinement tools lead to a less concentrated market structure.

Keywords: consumer search, click-stream data analysis, electronic commerce, consumer behavior