Italy leads the world in Design. Italian designers create elegant, beautiful products that people everywhere love. “Quite simply, we are the best”, declared architect Luigi Caccia Dominioni. “We have more imagination, more culture, and we are better mediators between the past and the future.” So why should Italian website designers imagine that they should test their designs with users?

Italian websites can be beautiful and elegant indeed. But for people to love a it, a site must be usable as well. It must enable its users to accomplish their goals quickly, smoothly and accurately. Especially when it’s a public administration site whose job is to serve all of the citizens (and many foreigners too).

Government digital services involve complex interactions with the people who use them. Even the best designers cannot predict with certainty how well users will understand and respond to a design, what they will find confusing or frustrating in even the simplest transaction. Yes, there are guidelines for usable web design, such as the US Government’s research-based guidelines. But even these extensive, detailed guidelines acknowledge usability testing as an essential part of the process.

Providing government services via digital channels has two goals. The obvious one is to save money: the UK’s 2012 Digital Efficiency Report describes the digital channel as much cheaper than the others. For some services, it says, transactions conducted by telephone, by post, and in person cost 20, 30, and 50 times as much, respectively, as transactions conducted digitally. No wonder governments want to get citizens online. According to its 2013 Government Digital Strategy paper, moving to “digital by default” would save the UK government £1.7 to £1.8 billion each year.

That’s approximately 2.4 billion euros in savings every year. With almost as many people as the UK, might not Italy hope to achieve comparable savings?

Less obviously, perhaps, but equally importantly, going digital aims to improve the availability and efficiency of government services. Here’s the UK Government Digital Strategy paper again:

“But this isn’t just about saving money – the public increasingly expects to access services quickly and conveniently, at times and in ways that suit them. We will not leave anyone behind but we will use digital technology to drive better services and lower costs.”

Cost savings, however, cannot be achieved merely by providing services online. If the services are not usable enough, the public will not use them. The October 2013 launch of healthcare.gov, the site by which Americans were to sign up for the new health insurance program, failed disastrously. It failed in part because of usability problems that testing could have found, such as requiring users to create an account before they could find the cost of an insurance policy. In large numbers, Americans had to use the much costlier telephone and in-person channels to access this service, and the site had to undergo expensive revision.

The UK Government approaches usability by requiring its public websites to adopt not only a consistent style but also a user centered process. The Digital by Default Service Standard requires all new transactional government services to meet 26 criteria for digital service provision. The criteria cover goals (“Create a service that is simple and intuitive enough that users succeed first time, unaided”), methods (“Establish performance benchmarks … against which the service will be measured”; “Analyse the prototype service’s success, and translate user feedback into features and tasks for the next phase of development”), management (“Put in place a sustainable multidisciplinary team that can design, build and operate the service, led by a suitably skilled and senior service manager with decision-making responsibility”). The standard explicitly requires user research and usability testing (“Put a plan in place for ongoing user research and usability testing to continuously seek feedback from users”) as part of the process, and it addresses both user performance and user satisfaction.

The USA takes a different approach. Its General Services Administration (GSA) offers a DigitalGov UX program that provides training and support to help agencies develop and apply their own usability competencies. “We provide effective user-centered services”, GSA says, “focused on the interaction between government and the public it serves.”

Usability engineering costs money, you might argue. And you’d be right. Thirty years ago, people argued against software engineering because it was “too expensive”. Eventually we learned that software engineering saves more money than it costs: the earlier a problem is found, the cheaper it is to correct — and a re-do of a fielded system is expensive. The same goes for usability: a prototype is far cheaper to change than is a site that has already gone live. In addition, launching a site that still has major usability problems can give you a bad reputation and can make people reluctant to come back to it, even after you fix the problems. Especially in these days of social media, word spreads fast.

usability testing

So I was testing a web site that allowed users to make travel reservations. In exploring the site to tease out major issues, I noticed that it didn’t handle cities very well by name; it seemed much more comfortable with airport codes. So I chose to have participants make a reservation to Lincoln, Nebraska, figuring that they might guess LIN as the airport code — and end up in Milan, Italy. (Lincoln’s code is LNK; LIN stands for Milan’s Linate Airport.)

Most of the participants in the test had at least some trouble with booking a trip to Lincoln, and more than one actually did wind up with a reservation to Milan. When one of these people realized what had happened, she howled with glee, then turned to me and demanded, “You did that on purpose, didn’t you? You knew that would happen!” (At this point I was free to laugh too, and I had to admit that I had.)

That’s my funniest moment in usability testing. What’s yours?