The prediction of a dramatic increase in online grocery shopping over the next 5 years is only likely to be fulfilled if online supermarkets present a more efficient and logical shopping experience. Therefore, usability testing is playing an increasingly important role in the development of e-commerce websites. To date, commerce usability research has only focused on the ordering of singular items and the issue of multiple item ordering has not been researched.
This study aims to start to fill this gap in usability research. Based on the results gathered, a set of ten usability criteria ere established, with a focus on the ordering systems for sites selling multiple items and quantities. Keywords Usability, World Wide Web, Electronic Commerce, Web Site Design, Internet Retail Store Design, Shopping. INTRODUCTION The Problem Context The issue of usability plays a vital role in the success of an online store. However, to date there has been no published research into this issue as it relates to multiple item ordering.
In the field of grocery shopping there is much competition between the two major physical supermarket chains in Australia, Woolworth and Coles, both in traditional stores and online. Online, the major competitor is Shafts, which does not have a traditional supermarket storefront to complement its online operations. The purpose of this study was to conduct a comparative analysis into the usability of ordering systems of online supermarkets that supply to Australian consumers in 2002, focusing specifically on the web sites belonging to Coles, Woolworth and Shafts.
This issue has been identified as a major problem by such companies, with one of these companies revealing in an interview that they had in excess of 200,000 clients registered, but only 100,000 had ever made a purchase and of these customers, 30,000 had only made one purchase. With figures as shown above it is understandable that these businesses would be interested in ways to increase the usability of their sites, potentially leading to an increased number of sales.
Although numerous studies have been conducted on usability (Dillon, 1998), no research has been done on multiple item ordering, which is of prime importance to online supermarkets. Background to the Research With e- commerce becoming an integral process of many traditional businesses, and with an increasing number of e-commerce businesses, it is important for customers to feel comfortable using the online ordering systems provided if the businesses are to be successful. Despite most online ordering systems requiring similar facilities and information, there is no standard for creating such a system.
The inherent differences of such systems affect the usability of these ordering systems. To date, there appears to be a lack of scientific research into this issue. This study aimed to evaluate the usability of the ordering systems implemented by the three main online permeates available for use by Australian Internet consumers: Shafts, Australian Online Supermarket Usability By Jeffrey using mechanisms from published studies and the perceptions of users, exploring issues of familiarity, navigation, aesthetics, the perception of trust and security, and whether each site was designed to be used intuitively.
For a supermarket to become successful, an interface which is simple to use and navigate is of high importance, with shoppers commonly purchasing multiple items from online supermarkets. Therefore, frustration experienced when locating a single item is likely to be littered in situations where the user repeatedly experiences difficulty. It is probable that poor navigation is a contributing factor in a high percentage of incomplete transactions.
Interfaces were assessed on criteria such as the layout, the ability to navigate through the product selection and ordering system, the visual appeal and general aesthetics, and the time taken for order placement. The impact of these usability aspects on the purchasing patterns and the recurrent use of the online supermarket was assessed to determine the most effective types of layout and ordering, including appropriate content and organization. Usability testing is an extremely important process which is often neglected during the implementation of a web site.
Its ultimate goal is to test the effectiveness of the web site (ACH, 2001). LITERATURE REVIEW Usability If something is said to have usability it has: ease of use, is easy to learn, is efficient, is visually pleasing, and is quick and effective (Bar et al. , 2002; Mandela, 1997; Pierce, 2001). When this interpretation of usability is applied to the Internet, James (2001) argues that if a site is usable, casual Internet shoppers are likely to be converted into e-shoppers.
Most online purchases are made by users who have had over two years Internet experience, and have therefore adapted to the medium and the related purchasing arrangements. New Internet users are unfamiliar with the technology, making usability a more important issue when encouraging them to complete purchase transactions online. Usability Testing The term ‘usability testing’ refers to “a process that employs participants who are representative of the target population to evaluate the degree to which a product meets specific usability criteria” (Rubin, 1994).
Its use as a research tool is based on traditional experimental theology, and allows tests to be conducted under a generic title, rather than being required to specify the particular method to be used. It encompasses a range of methods. Developers are able to gain greater understanding about their site by conducting usability testing, and observing how users interact with the site. This user interaction is often different to the designer’s planned flow of interaction and is an important tool to master, as it can provide valuable feedback on unplanned use and areas requiring improvement.
Those conducting testing must recognize that the aim f usability testing is to identify issues, and that to gather a statistically significant sample is unachievable and unnecessary. Small groups are able to identify issues relevant to the general user population, and a focus on these issues and their solutions should be the aim of the testing. Schaffer principles of efficient three clicks’ web design were applied at the Staples Inc. Web site (staples. Com) (Hicks, 2001). After redesigning the Staples Inc. Be site, the number of people dropping out of the registration process prior to completion was reduced by 53 percent. Reducing boosted overall profits due to a larger customer base. To achieve a more efficient registration process, Staples Inc. Created an internal usability group. This group consisted of Staples Inc. Staff, who were asked to use the site to register. The purpose of employing the test group was to gain feedback from site ‘consumers’, and use this information to create a more user-friendly site.
By allowing user input, the users were seen as a co-developer of the web site, allowing the production of a site that was acceptable to both developers and users. While this method produced positive results, the price of an outside lab conducting usability testing on a complex site in America can be in excess of IIS$30,OHO. In contrast to the behavior of Staples Inc. , many companies and developers assume that they can predict the needs and desires of customers without backup by scientific research or user testing.
However, Kruger claims that when usability testing is implemented it is usually “too little, too late, and for all the wrong reasons” (Kruger, 2000). Usability Testing Goals Usability testing seeks to understand the behavior and needs of consumers, to better serve them through an appropriately designed web site. Smith (2000) states that customers have pre- determined preferences when shopping. By identifying these preferences, and satisfying such preferences through the design of the e-commerce web site, an online store is able to create e-loyalty.
The best way to gather information on user preferences and determine the most effective way to incorporate stickiness drivers (Michael 2002) to satisfy them is through customer research or customer relationship management (CRM) programs. One common tool of such CRM programs is loyalty schemes. E-loyalty is concerned with retaining previous site customers, and encouraging them to be continually purchasing from that retailer. Smith (2000) poses ten design factors that can help create a more usable web site.
These are ; : ; ; ;Make your first impression count Make it simple to solve problems Design for your best customers Create value and engender trust Include features that start and continue dialogue Seize every opportunity to build community Deliver all parts of the sales cycle or subject covered Provide the best service that you can afford Make it easy to recommend your site Create an opportunity cost for defection The final product should aim to satisfy the end-user by creating a web site that is impel and satisfying to learn and use, while providing useful and obvious functionality that meets the needs of the target users (Rubin, 1994). Usable products increase customer satisfaction, therefore leading to greater loyalty and increased sales from repeat business. Often, web site ease of use is the deciding factor for online consumers, with this ease of use providing differentiation for consumers deciding to purchase from online supermarkets.
Only 56 percent of light Internet users have attempted to make a purchase of any type online dames, 2001) with the lack of consumer purchasing being largely attributed to lack of usability. The Background of Supermarkets Traditionally, supermarkets have sold a wide range of consumers have become more discerning due to changing lifestyles, spending priorities and shopping patterns (Fisher, 1998). The focus of supermarket supply is now changing towards provision of fewer products from a more limited range of brand name suppliers. There is also a trend to provide a total solution’ to customers, by including ‘ready to heat’ meals. The aim is to simplify the shopping process for customers. A study of 251 Coos across 10 industries Info, 2001) revealed that 77 percent of the relevant companies will increase their I.
T. Expenditure in the next 3 years. Overall, an average off 53 percent increase in expenditure is planned during this period. In contrast, supermarkets have traditionally been slow to adopt technological innovations. However, as the Internet adoption rate rapidly increases in Australia, supermarkets are releasing the large potential audience, and are attempting to meet the changing demands of customers. The largest supermarket chains in Australia, measured by their value share of all grocery sales, are Woolworth with 38. 3 percent market share, and Coles, with 28. 1 percent market hare (figures for year ending 28/10/2001) (Retail Media, 2002).
The third largest supermarket chain, Matches + Campbell, (commonly trading as ‘GA), has a market share of 1 1. 8 percent. Online Supermarket Usability Developers of online supermarkets need to understand the ‘mental models’ that users associate with grocery shopping in the ‘real-world’ environments (Barded, 2002). Users of online supermarkets are likely to have experience purchasing in a traditional supermarket, and are therefore experienced in determining the aisle locations of items. It is the categorization of items that is important for web site designers and developers, with seers commonly transferring traditional experiences to the online domain.
A search method where the user enters the items they are looking for is also valid. Both methods are present on the sites tested in this research. Navigation is a major usability issue, with web design based around linking units of content (Barded, 2002). It is the organization and classification of this content that allows effective navigation, and promotes usability. Therefore, consistent and logical navigation, as well as content layout (Consumers union of U. S, 2000), is essential on a site that promotes ease of use. Navigation refers to locating the desired information or product, while search capabilities help users locate desired products.
A study revealed that, on average, 70% of site users employ a web site’s search engine, and 43% believe that the search engine is the most important feature on a site (Banister, 2002). It is also important to provide sorting or proportioning capabilities to allow users to better meet their own needs. In the case where users choose to browse the site rather than employ the search facility, it is vital to provide meaningful labels and menu names to aid navigation (Consumers union of U. S, 2000). It is essential for online supermarkets to note that, while the public must shop for groceries, consumers have the choice to use traditional supermarkets or their online counterparts.
While, originally, many Internet users purchased goods via e-commerce, once the novelty wears off, online supermarkets must provide a strong incentive for consumers to purchase their goods online (Goldstein, 2002). To date, this incentive has been advertised as the convenience of being able to purchase from home. However, this concept of convenience must be extended to specific use of the site, not Just the overall idea of nonviolence of shopping online come at a cost, with Woolworth claiming they will be forced to charge an extra 11 percent on a $100 order to cover the cost of picking, packing and delivering the order (CHOICE, 1998). METHODOLOGY Usability Testing Methodology To gain information on the usability of the three major online supermarkets in Australia, task-based usability tests were conducted, followed by a post-test questionnaire.
The concept of the usability testing methodology is taken from the classical approach for conducting a controlled experiment (Rubin, 1994). A comparative analysis of the three supermarket sites, Woolworth, Coles and Shafts, was undertaken. This approach of a “Comparison Test” allows Judgment of a product, such as a web site, against its competitors at the end of the development lifestyle. The objective is to compare two or more different interface styles, with this information used to better understand the advantages and disadvantages of different designs (Rubin, 1994). Purpose The purpose of the usability testing was to gain knowledge of users’ perceptions and their comparisons of the different web sites servicing the online supermarket industry.
Task-based testing helped familiarize participants with the sites, and required them to complete a standard set of tasks across each site. This allowed consistent tests with identical products and tracking of the time required to complete the tasks, providing a basis for comparison. Questionnaires were used to measure those aspects of usability that are not easily measured quantitatively, but are qualitatively-based and Just as significant. Test Methods The following section discusses in detail the three elements of the research. Background/Screening Questionnaire This questionnaire was issued to potential seers to gain basic background information, and to act as the screening process for participants.
The information obtained included the user demographics, computer/ Internet experience and online shopping experience. The purpose of the questionnaire was to ascertain each user’s level of experience, and ensured a representative sample of users participants was tested (Rubin, 1994). Task-based Testing Task-based testing involved supplying given tasks to users to complete (Rubin, 1994). These tests were assessed on two criteria. The first criterion was the time taken to complete the test. The second aspect of the task-based testing involved observing the user carrying out the tests. Nielsen (2001) stated that to gain information about usability the first rule is to watch the users work with the web site, not Just listen to what they say.
Quantitative usability metrics were used to assess the following tasks: ; ; Creating a basic ‘shopping trolley of items, consisting of 10 items (quantities) Creating a complex ‘shopping trolley of items, consisting of 28 items (quantified) The two tests were conducted only until the point where the shopping trolley list was displayed, prior to credit card details being entered, with the time taken by each user o complete the tasks recorded. This quantitative data was then used to study the usability of the web sites via measurement of the speed of completion. The not only highly usable but allows online shoppers to conduct their ordering in a timely manner. Previous studies have demonstrated that decreased completion time is a contributing factor to users shopping online (Dillon, 1998). Task-based tests were used to gain knowledge of how users interact with the web sites.
The quantitative aspects of the testing were used to ascertain the time taken for a user to complete core web site tasks. It will allow all businesses to be assessed on a common basis, and use these findings to compare the web sites. Post-test questionnaire The post- test questionnaire was based around the proposed Centre for Electronic Commerce (CE) web site evaluation framework (Elliot, 2002). This framework is broadly based, and does not narrowly focus on individual perspectives. The sections of the framework used in this questionnaire were the ease of use and the innovation in services and technology section, with a focus on the searching capabilities of each of the web sites. The questionnaire was designed to consist of three main sections.
The first section used the Licker scale, asking questions about the user perceptions of and attitudes towards each of the web sites. The second section was designed for users who used the search facilities. Users were asked to rate the effectiveness of the search facilities using the Licker scale, and record items they had difficulty locating. The third section asked for general comments about the usability of each of the web sites. RESULTS Background Questionnaire The test population consisted of 9 females and 11 males. Most of the test subjects were in the 18-25 age group, with this age bracket presenting the group most likely to adopt online grocery shopping in the future. All of the users were regular computer users.
Internet use statistics were similar to computer use, with 10 using the Internet at least once a day, 9 using the Internet once a day and 1 user using the Internet a least once a week. Details of computer and Internet use indicate that all users fit into the average profile of an Internet shopper Games, 2001). Fourteen of the users that were selected had previously purchased goods and services from the Internet. Of these fourteen Internet shoppers, most had purchased over 10 items online, which is regarded as a significant number of purchases. Three users had previously purchased from an online supermarket: two users had made two or less purchases, and the other user purchased regularly.
Test Script Results The purpose of requiring users to complete two tests was twofold: To provide users with the chance to become familiar with each site, prior to completing rigorous testing To provide a comparison between the two tests; with the Advanced Test requiring users to purchase a greater number of items in varying quantities, after gaining experience with the site, to determine whether users are able to increase their speed ND whether the use of the site is intuitive. Basic Test The basic test was conducted by all test subjects. This test involved the script. Tests undertaken at Coles (mean eminences) were the quickest to finish followed by Woolworth (mean eminences) then Shafts (mean imminence’s).
Basic Test 1200 1 oho seconds 200 202020 WORTHY COLES SHAFTS Figure 1: Basic Test Postbox As can be seen from the postbox above, Shafts experienced the greatest range of test script times taken by users. Woolworth had the smallest time range, with searches returning consistent numbers of results under similar headings, despite different keywords. This led to greater consistency in time across all users. Advanced Test The results obtained for the advanced test were quite similar to those obtained from the basic test. The store that took the shortest time was Coles (mean 1 eminences), followed by Woolworth (mean imminence’s) and Shafts (mean imminence’s).
Advanced Test 2000 1800 41361616 1600 1400 1200 1000800 19 600 400 200 0 19 19 19 Figure 2: Advanced Test Postbox As can be seen above, the range of resulting times from Woolworth and Shafts was greater than Coles. However, all sites experienced cases where users took notably longer than the average. This was molly due to difficulty locating one or two items, with this searching increasing the time taken by several minutes. Summary of Findings The findings from the testing procedure clearly indicated that Coles was the fastest site to use, therefore increasing user confidence and satisfaction. Coles was closely followed by Woolworth, with Shoptalk’s slow test script times reflecting the low user satisfaction.
Possible reasons for the differences in timing and user responses are reviewed in the discussion, with guidelines provided to help designers increase usability and therefore improve the time required to use each site. The following graphs below are on a scale with 1 being poorly rated by the testers and 5 being of a nature that was deemed to be exceptional. What are your perceptions of the overall layout? As shown on the graph below, testers found the overall layout at Coles to be the most usable. Shafts was rated as the least usable, with testers experiencing difficulty finding the search facilities, which were located in the top left hand corner of the screen. Those testers who found the search box commented on its illogical placement.
Testers appreciated being able to view trolley contents while shopping at Shafts, over the placement of the trolley contents made them difficult to read. Testers had few complaints about the layout of the Woolworth site, but also did not note any features which made it especially user-friendly. The placement of the search box and trolley at Coles was preferred by testers, because they were able to see at a glance what they were looking for. Overall Layout 5. 0 4. 5 4. 0 3. 5 3. 0 2. 5 2. 0 Mean Figure : Overall Layout What are your perceptions of the ordering page layout? Despite the Coles ordering page being rated as the most usable, one common implant from testers was the illogical and inconvenient placement of the buy button.
At Coles, the buy button was placed to the left of the item description, while Shafts and Woolworth located the button to the right. Testers also found the ability to change item quantities important to their overall shopping experience. Woolworth was the most usable in this respect, with testers able to enter the quantity when placing the item in the basket. Coles and Shafts both required testers to place the item in the basket, then subsequently edit the quantity, either by going to the trolley contents list or by clicking the buy button repeatedly. While testers appreciated this facility at Woolworth, many were unaware that by placing a tick in the ‘add to trolley box, the quantity defaulted to one (1).
Instead, many testers placed a one (1) in the quantity box, then ticked the ‘add to trolley box, leading to complaints about this excessive effort to purchase a single item. The other area of contention with testers was the layout of the search results. Some testers appreciated the format at Woolworth, where items were returned under category headings. This made it faster for testers to scan the results list. None of the users tested preferred the format of the Shafts exults, which consisted of an extensive list of unsorted items that matched the search criterion. The format of item details was also problematic, with Shafts using the fewest number of columns to divide information. The entire description (including size) was in one column, making it difficult to quickly scan the results list.
The preferred method was the one used at Coles, where item information was formatted into clearly labeled columns, and results were ordered alphabetically by default. Ordering Page Layout How was the navigation to access the items? The navigation on the Coles site was rated as the most usable, as shown in the graph search box visible and the aisles clearly labeled and logically arranged. The aisles at Woolworth were also described as logically arranged. Shafts was the least usable by far. The search facility was difficult to locate, with some testers purchasing all items through the aisles because they were unable to find the search box. Navigation Figure 4. 8: Navigation How was the color scheme?
Testers found both Woolworth and Coles to have pleasant color schemes, with many testers indicating that their inference was increased through the use of company colors. Bright colors, such as those on the Shafts site, offended some testers. Shafts may have been disadvantaged by its lack of traditional storefronts, with testers having much less exposure to Shoptalk’s branding offline. Color Scheme Figure 4. 9: Color Scheme What are your perceptions of the overall process of ordering items? Test subjects rated Coles as having the simplest process for ordering items. However, talking to testers after completing the post-test questionnaire indicated that these results may be reflective more of each tester’s overall