Windows 8 Tablet: A Usability Test

March 7, 2013 by Andreas Schmitz

tablets

Six months ago, Gotthard Tischner from SAP partner cundus claimed, “Windows 8 ousts Android.” His statement was based largely on the assumption that Surface offers a higher level of security than both the iPad and comparable Android devices – such as Samsung Galaxy Note. Now available for purchase, Surface is Microsoft’s first foray into the tablet market, and its competitors are keeping a watchful eye on its progress.

Professor Jan Borchers of RWTH Aachen University (RWTH) developed a concept for testing three tablets – Apple’s iPad, Samsung’s Galaxy Note 10.1, and Microsoft’s Surface RT – for Markencheck, a consumer information show that appears on German TV network ARD. The test focused on finding out how easy or difficult it is for a beginner to come to grips with each of these devices: That includes getting started, sending an e-mail, downloading a game, a book, and a movie from the corresponding online store, and opening a Web site.

The five key findings for Microsoft Surface are described below:

  1. Metro user interface and Windows desktop
  2. Features
  3. Getting started
  4. Design and apps
  5. Microsoft Store

On the next page: Interface, desktop, and tablet features

Borchers End

Professor Jan Borchers compared the Microsoft’s Surface, Samsung’s Galaxy Note, and Apple’s iPad in a usability test. (Photo: private)

1. Metro user interface and Windows desktop

Microsoft has not succeeded in breaking with tradition and has opted to stick with the familiar Windows desktop for the loyal faithful. For all those who want a little more “Apple feeling”, the Surface home screen uses the colored tiles that are part of Microsoft’s Metro user interface design language. “These aren’t really suitable for occasional users, though,” comments Borchers, “because they don’t want to create PowerPoint presentations; they want to access the Internet, write e-mails, and order goods. And, above all, they want all these things to be simple to do.”

2. Features

Surface is what you might call a “jack-of-all-trades” device. You can plug a keyboard into it and start typing right away. And you can plug in a mouse too. If you want to hook up an external keyboard to an Apple device, you need Bluetooth and access to a mobile network. With Surface, typing in a text is no problem at all – even if you’re sitting on a plane. All this leads Borchers to conclude that Microsoft was trying to “be everything to everyone” with this product. “If in doubt,” he says, “the engineers will build in the full spectrum of features.”

On the next page: Getting started

3. Getting started

While the “average” user will have Apple’s iPad up and running in ten minutes, the Microsoft tablet takes longer to set up – on average, three times longer.  The reason for this appears to lie in Surface’s multiuser environment. While Apple is content with a single-user concept, Microsoft follows a very different strategy in which multiple users can set up access to their own mail accounts. This “cloning” process during setup takes about 30 minutes. It does have one advantage though. “It puts an end to the annoyance caused when our kids “borrow” our tablets to quickly download music from iTunes or place an order with Amazon,” says Borchers. With Surface, that can’t happen because each user has his or her own “world”. But it does mean that you have to log in and out all the time − which is not really practical for the occasional, short periods of use that are typical for tablet owners.

One thing that particularly annoyed the testers was the fact that they received the message “An error has occurred” at the end of the installation, meaning that the entire installation process had to be repeated. At the second attempt, however, the user name was already taken. It wasn’t until the third attempt, which required a keyboard reset, that the installation actually worked. And it wouldn’t have worked at all without the keyboard. “That sort of thing really shouldn’t happen,” says Borchers.

On the next page: Tablet design and apps

4. Design and apps 

The tile logic is new, and users expect the color scheme to have some kind of meaning, but it doesn’t. The tiles are minimalistic in design, with white icons on a colored background. “Tiles of the same color are not linked to similar functions,” says usability expert Borchers. This probably won’t worry users too much, even if functional relationships might be useful, because you can shuffle and group the tiles – which differ in size for no apparent reason – in any way you like.

A more serious problem, however, is the introduction of a function that goes against neurophysiological rules, namely, that humans are better at using shapes and colors to remember where they have seen things before. In Windows 8, the app icons, for example, change constantly once you’ve called up content from them for the first time. The icon is no longer an icon: “A news app will always show the latest headlines; a Web browser will display the latest content — it’s not long before many of the icons contain numerous snippets of text and pictures that you can’t read and distinguish between because the icons are too small,” remarks Borchers. “Designed by techies for techies” is the expert’s assessment. “The designers have focused more on form than on function.”

Next page: Microsoft Store and summary

5. Microsoft Store

Obviously, the Microsoft Store does not yet have as much to offer as the Apple Store or Android stores. Even so, the offerings are not clearly differentiated for individual device categories. For example, you can use Surface to download X-BOX games from the Microsoft Store, but you can’t actually play them on the tablet. You can’t read e-books on the tablet either. And, during testing, the testers were unable to download a movie, even after three attempts.

The team of experts from Aachen University reached the following conclusion: “Overall, the Surface hardware gives the impression of being robust and thoughtfully designed. However, Microsoft’s lack of expertise in this product area is apparent in various areas, including a lack of visibility of interaction concepts. The two protective cases with built-in keyboards that are available for Surface, known as Type Cover and Touch Cover, are a major plus point for the product. The Type Cover, in particular, makes it very easy and convenient to type in a text.”

Incidentally, in a direct comparison of the three tablets, the winning device was the iPad, which obviously benefits from having already passed through several rounds of product improvements. In the nine tasks that Borchers’ team set for the tablets, the iPad came first six times and Surface “only” four times (with two shared first places). Surface only beat the iPod by virtue of its flight mode and the ease of typing in text.

Tags: , , , ,

Leave a Reply