Evaluating metrics for comparing the use of web sites: a case study of two consumer health web sites

Abstract

Use is plainly an important characteristic in determining the success or otherwise of a web site and in making comparisons between sites. The source of most use data are the web logs that record user activity on a real-time and continuous basis. There is much demand from sponsors, site owners and marketing departments for this information. The authors examine the kinds of comparison that can be made, the performance of the respective metrics in comparing two sites - something that is still relatively rare - and discuss the methodological problems associated with making such comparisons. Two consumer health sites - NHS Direct Online and SurgeryDoor - are compared. Data were collected for the month of November 2000. During this period the two sites were visited by more than 60,000 people who recorded more than 1 million page views. The paper shows that the most widely used and heralded metric of use, page views (impressions) is heavily dependent on site architecture and this causes difficulties in comparing the performance of sites.