The ROI of Usability

ROI – Return on Investment – is one way of looking at the value of usability to a business. This article was originally published in the Winter 2002 User Experience Magazine as part of “Return on Investment for Usable UI Design” by Aaron Marcus.

Overall Value of Implementing UI Practices

Because there have been many well-documented examples of cost savings with usability engineering, sound statistics can be applied generally to UI development. These statistics serve as benchmarks.

Value proposition: High return on savings and product usability

Some statistics:

“The rule of thumb in many usability-aware organizations is that the cost-benefit ratio for usability is $1:$10-$100. Once a system is in development, correcting a problem costs 10 times as much as fixing the same problem in design. If the system has been released, it costs 100 times as much relative to fixing in design.” (Gilb, 1988)
“The average UI has some 40 flaws. Correcting the easiest 20 of these yields an average improvement in usability of 50%. The big win, however, occurs when usability is factored in from the beginning. This can yield efficiency improvements of over 700%.” (Landauer, 1995)

Development: Reduce Costs

Usability engineering is most effective at the beginning of the product development cycle, especially if it is part of quality functional deployment (QFD), a process used for structuring the development process through a primary focus on customer requirements. Through QFD, reducing development time by one-third to one-half is possible (Scerbo, Bosert, 1991). For example, for new products there is no investment in any particular design, and numerous possibilities can be explored at relatively little cost (Bias & Mayhew, 1994) (see Figure 1). Applying human factors in the initial design can greatly reduce extensive redesign, maintenance, and customer support, which can substantially eat away profits.

Figure 1. The number of possible designs decreases as the cost to make changes increases. (Bias & Mayhew, 1994, p. 80)

Short-term benefits can accumulate during product development and long-term benefits during product release. Customers directly benefit by usability improvements by increases in ease of use, ease of learning, user satisfaction, and user productivity. At the same time, decreases occur in the number of “user” errors, costs for training and support, and maintenance. Taking proactive measures in usability and quality during the initial production stages can produce a cost saving rippling effect.

Value proposition: Save development costs

Some Examples

“Savings from earlier vs. later changes: Changes cost less when made earlier in the development life cycle. Twenty changes in a project, at 32 hours per change and [a minimal] hourly rate of $35, would cost $22,400. Reducing this to 8 hours per change would reduce the cost to $5,600. Savings = $16,800.” (Human Factors International, 2001)
“A financial services company had to scrap an application it had developed, when, shortly before implementation, developers doing a User Acceptance test found a fatal flaw in their assumptions about how data would be entered. By this time, it was too late to change the underlying structure, and the application never implemented.” (Dray, 1995)

Some Statistics

“When managers were polled regarding the reasons for the inaccurate cost estimates, the top four reasons were issues that could have been addressed by following best practices in usability engineering. These include frequent requests for changes by users, overlooked tasks, users’ lack of understanding of their own requirements, and insufficient communication and understanding between users and analysts.” (Barker, 2000)
“A study of software engineering cost estimates showed that 63% of large software projects significantly overran their estimates (Nielsen, 1993). When asked to explain their inaccurate cost estimates, software managers cited 24 different reasons and, interestingly, the four reasons rated as having the highest responsibility were related to usability engineering. Proper usability engineering methodology will prevent most such problems and thus substantially reduce cost overruns in software projects.” (Nielsen, 1993)

Value proposition: Save development time

An Example

“Usability techniques allowed a high-tech company to reduce the time spent on one tedious development task by 40% (Bias & Mayhew, 1994). At another company, usability techniques helped cut development time by 33-50%.” (Bosert, 1991)

Some Statistics

“Conklin (1991) states that speeding up development is a key goal for integrating usability effectively into product development and that a one-quarter delay in bringing a product to market may result in the loss of 50% of the product’s profit.” (Bias & Mayhew, 1994)
“Increased revenues accrue due to the increased marketability of a product with demonstrated usability, increased end-user productivity, and lower training costs. Conklin (1991) states that another usability goal is speeding up market introduction and acceptance by using usability data to improve marketing literature, reach market influencers and early adopters, and demonstrate the product’s usability and reduced training cost.” (Bias & Mayhew, 1994)

Value proposition: Reduce maintenance costs

An Example

“[Usability engineering techniques] are quite effective at detecting usability problems early in the development cycle, when they are easiest and least costly to fix. By correcting usability problems in the design phase, American Airlines reduced the cost of those fixes by 60-90%.” (Bias & Mayhew, 1994)

Some Statistics

“One [well-known] study found that 80 percent of software life-cycle costs occur during the maintenance phase. Most maintenance costs are associated with “unmet or unforeseen” user requirements and other usability problems.” (Pressman, 1992)
“Martin and McClure found that $20-30 billion was spent worldwide on maintenance. Studying backlogs of maintenance work shows that an “invisible” backlog is 167% the size of the declared backlog. Anonymous case study data show that internal development organizations are spending the majority of their resources on maintenance activities and thus cannot initiate development of strategic new systems.” (Martin & McClure, 1983)

Value proposition: Save redesign costs

An Example

“Sun Microsystems has shown how spending about $20,000 could yield a savings of $152 million dollars. Each and every dollar invested could return $7,500 in savings.” (Rhodes, 2000)

Sales: Increase Revenue

Usable products lead to substantial cost savings and sales. Unusable products most often prevent a customer from accomplishing a productivity task or retrieving information necessary to make an e-commerce purchase. Online shoppers spend most of their time and money at Websites with the best usability (Nielsen, 1998). Good navigation and Website design make it easier for users to find what they’re looking for and to buy it once they’ve found it (Donahue, 2001).

Because there are so many poorly designed Websites, when customers find one that “works”, they tend to do repeat business and gain trust in the organization.

Usable products also lead to good product reviews. Publications devote space just to this one factor, and good reviews lead to increased sales.

Value proposition: Increase transactions/purchases

Some Statistics

“You can increase sales on your site as much as 225% by providing sufficient product information to your customers at the right time. In our recent research, we found that the design of product lists directly affected sales. On sites that did not require shoppers to bounce back-and-forth between the list and individual product pages, visitors added more products to their shopping cart and had a more positive opinion of the site. By understanding your customer expectations and needs, and designing your product lists accordingly, you can significantly increase your sales.” (UI Engineering, 2001)
“One study estimated that improving the customer experience increases the number of buyers by 40% and increase order size by 10%.”(Creative Good, 2000)

Value proposition: Increase product sales

Some Examples

“Wixon & Jones did a case study of a usability-engineered software product that increased revenue by more than 80% over the first release of the product (built without usability work) (Wixon). The revenues of the usability-enhanced system were 60% higher than projected. Many customers cited usability as a key factor in buying the new system.” (Bias & Mayhew, 1994)
“After move.com completed the redesign of the home “search” and “contact an agent” features based on a UI consulting firm’s recommendations, users ability to find a home increased from 62% to 98%, sales lead generation to real estate agents increased over 150%, and [move.com’s] ability to sell advertising space on move.com improved significantly.” (Vividence, 2001)

Some Statistics

“The magnitude of usability improvements is usually large. This is not a matter of increasing use by a few percent. It is common for usability efforts to result in a hundred percent or more increase in traffic or sales.” (Nielsen, July 1999)
“Convoluted e-commerce sites can lose up to half of their potential sales if customers can’t find merchandise, according to Forrester Research, Inc.” (Kalin, 1999)

Value proposition: Increase traffic (size of audience)

Some Examples

“IBM’s Web presence has traditionally been made up of a difficult-to-navigate labyrinth of disparate subsites, but a redesign made it more cohesive and user-friendly. According to IBM, the massive redesign effort quickly paid dividends. The company said in the month after the February 1999 re-launch that traffic to the Shop IBM online store increased 120 percent, and sales went up 400 percent.” (Battey, 1999)
“At HomePortfolio.com we monitored site traffic, observed consumers in usability studies and worked with internal business groups. This helped us make changes that made the site’s purpose clearer and increased transaction rates measurably. The change increased the traffic up 129% the week we put it up.” (Interaction Design, Inc., 2001)

Value proposition: Retain customers (frequency of use)

Some Statistics

“More than 83 percent of Internet users are likely to leave a Web site if they feel they have to make too many clicks to find what they’re looking for, according to Andersen’s latest Internet survey.” (Arthur Andersen, 2001)
“A bad design can cost a Web site 40 percent of repeat traffic. A good design can keep them coming back. A few tests can make the difference.” (Kalin, 1999)

Value proposition: Attract more customers (increase appeal)

An Example

“Staples.com determined that the key to online success and increased market share was to make its e-commerce site as usable as possible. Staples.com spent hundreds of hours evaluating users’ work environments, decision-support needs, and tendencies when browsing and buying office products and small business services through the Web. Methods included data gathering, heuristic evaluations, and usability testing. [They achieved these results]:
67% more repeat customers
31-45% reduced drop-off rates
10% better shopping experience
80% increased traffic
Increased revenue” (Human Factors International, 2001)

A Statistic

“In a 1999 study of Web users, respondents were asked to list the five most important reasons to shop on the Web. Even though low prices definitely do attract customers, pricing was only the third-most important issue for respondents. Most of the answers were related to making it easy, pleasant, and efficient to buy. The top reason was “Easy to place an order” by 83% of the respondents.” (Nielson, February 1999)

Value Proposition: Increase market share (competitive edge)

An Example

“Usability is one of our secret weapons.” The secret weapon appears to be working. Schwab’s main Website for U.S. investors,www.schwab.com, handles more than $7 billion in securities transactions a week, with more than 2 million active customer accounts holding $174 billion in assets. With those numbers, one might wonder why Schwab would need to make any changes to its Web site at all. But Schwab knows it cannot afford to coast; as more and more newcomers get online, and the competition for their dollars increases, more e-commerce sites are making ease of use a differentiator. “A year ago, it was a rush to put up applications and functionality,” Thompson says. “It’s now a rush to be useful.” (Kalin, 1999)

Some Statistics

“The importance of having a competitive edge in usability may be even more pronounced for e-commerce sites. Such sites commonly drive away nearly half of repeat business by not making it easy for visitors to find the information they need (Manning).
The repeat customers are most valuable: new users at one e-commerce site studied spent an average of $127 per purchase, while repeat users spent almost twice as much, with an average of $251.” (Nielsen, August 1, 1997).

Use: Improve effectiveness

User-centered design benefits users, the users’ company, and the vendor company. Increased usability increases productivity and job satisfaction while decreasing customer support needs and documentation requirements. When users feel more effective with their work, rates of absenteeism and employee turnover are lowered. All of these benefits are in alignment with fulfilling successful business goals.

Value proposition: Increase success rate, reduce user error

Some Examples

“One study at NCR showed a 25% increase in throughput with an additional 25% decrease in errors resulting from redesign of screens to follow basic principles of good design.” (Gallaway, 1981)
“On Disney.com, for example, when UIE asked users to find the hotel closest to the monorail at Disney World, about 20 percent became lost in Disneyland and didn’t even know it. ‘If one in five people who came to the theme parks got lost,’ Spool says, ‘Disney would fix it.’ Disney Online’s Senior Vice President and General Manager Ken Goldstein notes that Disney Online is already committed to developing an easy-to-use Internet design. While Disney Online did not have anything to do with Spool’s tests, Goldstein is interested in his findings. ‘As the next generation of Disney.com evolves,’ Goldstein says, ‘we will continue to respond to customer input through our own usability testing.'” (Kalin, 1999)

Some Statistics

“A study from Zona Research found that 62% of Web shoppers have given up looking for the item they wanted to buy online (and 20% had given up more than three times during a two-month period).” (Nielsen, October 1998)
“In Jared Spool’s study of 15 large commercial sites, users could only find information 42% of the time even though they were taken to the correct home page before they were given the test tasks.” (Nielsen, October 1998)

Value Proposition: Increase efficiency/productivity (reduce time to complete task)

Some Examples

“With its origins in human factors, usability engineering has had considerable success improving productivity in IT organizations. For instance, a major computer company spent $20,700 on usability work to improve the sign-on procedure in a system used by several thousand people. The resulting productivity improvement saved the company $41,700 the first day the system was used. On a system used by over 100,000 people, for a usability outlay of $68,000, the same company recognized a benefit of $6,800,000 within the first year of the system’s implementation. This is a cost-benefit ratio of $1:$100.” (Bias & Mayhew, 1994)
“To build a model intranet, Bay Networks spent $3 million and two years studying the different ways people think about the same thing. The result: all think alike about the $10 million saved each year.” (Fabris, 1999)

Some Statistics

“Inadequate use of usability engineering methods in software development projects have been estimated to cost the US economy about $30 billion per year in lost productivity (see Tom Landauers’ excellent book The Trouble with Computers). By my estimates, bad intranet Web design will cost $50-100 billion per year in lost employee productivity in 2001 ($50B is the conservative estimate; $100B is the median estimate; you don’t want to hear the worst-case estimate!). Bad design on the open Internet will cost a few billion more, though much of this loss may not show up in gross national products, since it will happen during users’ time away from the office.” (Nielsen, August 28, 1997)
“On a corporate intranet, poor usability means poor employee productivity; usability guru Jakob Nielsen estimates that any investment in making an intranet easier to use can pay off by a factor of 10 or more, especially at large companies.” (Kalin, 1999)

Value Proposition: Increase user satisfaction

An Example

“One airline’s IFE (In-flight Entertainment System) was so frustrating for the flight attendants to use that many of them were bidding to fly shorter, local routes to avoid having to learn and use the difficult systems. The time-honored airline route-bidding process is based on seniority. Those same long-distance routes have always been considered the most desirable. For flight attendants to bid for flights from Denver to Dallas just to avoid the IFE indicated a serious morale problem.” (Cooper, 1999)

A Statistic

“When systems match user needs, satisfaction often improves dramatically. In a 1992 Gartner Group study, usability methods raised user satisfaction ratings for a system by 40%.” (Bias & Mayhew, 1994)

Value Proposition: Increase job satisfaction/decrease job turnover

An Example

“Humantech, Inc., studied ergonomic office environments and productivity for 4000 managerial, technical, and clerical workers in a broad cross-section of North American industries. Surveys showed that video display terminal workers had twice as many complaints of neck and shoulder discomfort, eye strain was reported three times as often, and there were higher rates of absenteeism less job satisfaction, and increased (30%) turnover.” (Schneider, 1985)

Value Proposition: Increase ease of use

A Statistic

“Incorporating ease of use into your products actually saves money. Reports have shown it is far more economical to consider user needs in the early stages of design, than it is to solve them later. For example, in Software Engineering: A Practitioner’s Approach, author Robert Pressman shows that for every dollar spent to resolve a problem during product design, $10 would be spent on the same problem during development, and multiply to $100 or more if the problem had to be solved after the product’s release.” (IBM, 2001)

Value Proposition: Increase ease of learning

A Statistic

“A study by Computer + Software News (1986) found that users rated ease of use second at 6.8 out of 10, while ease of learning was rated fourth at 6.4 on a scale of important purchase factors.” (Bias & Mayhew, 1994)

Value Proposition: Increase trust in systems

An Example

“User trials were used to redesign the EuroClix Website before its launch. In its first six months, it convinced more than 30,000 users to sign up. This study clearly shows that consumers’ trust concerns can significantly be alleviated by providing relevant information when and where users need it.” (Egger & de Groot, 2000)

Value Proposition: Decrease support costs

Some Examples

“At Microsoft several years ago, Word for Windows’s print merge feature was generating a lot of lengthy (average = 45 minutes) support calls. As a result of usability testing and other techniques, the UI for the feature was adjusted. In the next release, support calls ‘dropped dramatically’; Microsoft recognized ‘significant cost savings.” (Bias & Mayhew, 1994)
“A certain printer manufacturer released a printer driver that many users had difficulty installing. Over 50,000 users called support for assistance, at a cost to the company of nearly $500,000 a month. To correct the situation, the manufacturer sent out letters of apology and patch diskettes (at a cost of $3 each) to users; they ended up spending $900,000 on the problem. No user testing of the driver was conducted before its release. The problem could have been identified and corrected at a fraction of the cost if the product had been subjected to even the simplest of usability testing.” wrote the researcher.” (Bias & Mayhew, 1994)

Value Proposition: Reduce training/documentation cost

Some Examples

“In another company, business representatives did a cost-benefit analysis for a new system and estimated that a well-designed GUI front end had an Internal Rate of Return of 32%. This was realized through a 35% reduction in training, a 30% reduction in supervisory time, and improved productivity, among other things.” (Dray & Karat, 1994)
“At one company, end-user training for a usability-engineered internal system was one hour compared to a full week of training for a similar system that had no usability work. Usability engineering allowed another company to eliminate training and save $140,000. As a result of usability improvements at AT&T, the company saved $2,500,000 in training expenses.” (Bias & Mayhew, 1994)

A Statistic

“A study by Computer + Software News (1986) found that information systems managers rated ease of training seventh (out of 10) on a scale of important purchase factors.” (Bias & Mayhew, 1994)

Other ROI Factors

Since the early 1960s, issues related to product safety have led to pro-plaintiff legal precedents in US courts. If a manufacturer has not included usability factors into their product, then it is usually found liable. If a manufacturer has assimilated human factors engineering into its development process, claims on the grounds of usability may be greatly reduced.

Value proposition: Litigation deterrence and safety

Some Examples

“Although software makers don’t seem liable to the same sorts of litigation as, for example, a manufacturer of medical equipment, poor usability may be an element in lawsuits. For example, the Standish Group reported that American Airlines sued Budget Rent-A-Car, Marriott Corporation, and Hilton Hotels after the failure of a $165 million car rental and hotel reservation system project. Among the major causes of the project’s disintegration were “an incomplete statement of requirements, lack of user involvement, and constant changing of requirements and specifications,” all issues directly within usability’s purview.” (Standish, 1995)
“Poor usability is a potential element in lawsuits and other litigation. The US government’s recent case against Microsoft hinged on a usability question: Are users well-served when the browser and operating system are closely integrated?” (Donahue, 2001)

A Statistic

“Chapanis cites two independent studies that showed a 54% reduction in rear-end accidents with the use of human factors improvement: the centered high-mount brake light on autos.” (Chapanis, 1991)

References – Sources for all references cited in this article