Category Archives: insight

Understanding user behavior and preferences

SwissCHI event Apr 30, 09: “Prototyping of Rich Internet Applications”

Last Thursday I attended the monthly SwissCHI meeting. That evening's topic was "Prototyping of Rich Internet Applications" – Andreas Binggeli, Marc Blume and Yuan-Yuan Sun presented their Master's thesis (from their MASHCID studies). Interestingly, but not surprisingly, paper prototypes compared really well to more elaborate prototypes done in Axure RP Pro or realized using Ajax-y technology, and users didn't really pay much attention to the paper prototypes lower fidelity. (I wrote up a piece on which prototyping tool to use in which case earlier.)

After a brief introduction into what RIAs are (or rather, aren't: "RIA ≠ Ajax ≠ Web2.0"), Andreas, Marc and Yuan-Yuan presented their study. The following hypothesis were to be tested:

  • Between different forms of prototypes, there would only be differences in speed and visual appearance
  • There wouldn't be differences in user hesitation, difficulties, slowing-down, quality of interaction etc.

They'd be asking subjects to form an overall assessment of visual appearance, behavior, and speed.

The four prototype variants they were testing were:

  • hand-drawn paper prototype
  • Powerpoint-created paper prototype
  • Pseudo-HTML prototype created in Axure RP pro
  • Ajax prototype

These were their findings:

  • Both the hand-drawn and the PowerPoint-created paper prototypes were experienced as (too) slow; a lot of preparation on the researcher's side and high concentration were needed
  • Axure-powered prototypes used interaction elements that looked alright but didn't live up to their expectations
  • The Ajax-based prototype was quite slow in some parts, which lead to some immediate reactions not being perceived by the subjects, whereas some elements' affordance wasn't perceived properly
  • Interestingly, subjects didn't really see differences in the visual appearances of the prototypes
  • All methods can be used to prototype RIAs, but all have their very specific weaknesses
  • Interaction quality is compromised most severely by unexpected behavior, not so much by visual appearance or speed
  • The appropriate method should be chosen based on the phase the project is in, the Ajax pattern to be simulated, and the desired longevity of the prototype

In the end, if you take the effort into consideration, paper prototypes might still be the most appropriate method in most cases. This finding really doesn't come as a big surprise…

Business Week: Design vs. Design Thinking, and what’s “Design Thinking” anyway?

In his article, "Design Vs. Design Thinking", Business Week's Bruce Nussbaum offers some great insight into the current struggle between business schools, design schools, designers and business people around who actually defines "design thinking" and what "design thinking" actually is supposed to mean:

My own current thinking is that designers must play a critical role in the creation of this new field of design thinking. The whole core culture of design is essential to design thinking. In fact, I would argue that the rise of Web 2.0 and social networking reinforces the traditional design focus on empathy and integration – human factors, the user interface, culture. Web 2.0 technology is behind the boost to design in the corner office as businesses delve more deeply into the lives of their customers–who are demanding to be part of the process of creating and designing stuff. Social media reinforce their desire to participate.

But design thinking is such a new field that it's not clear whether design schools or business schools will develop the formal concepts and methodologies that turn it into a broad, deep and powerful tool of organizational change.

The fact is that design thinking (or whatever we wind up calling this new field) is being created at the borders of design, business, engineering and even marketing. And I don't know which institutions will take the lead in promoting it.

Read the full posting on Business Week

Hunting for the optimal prototyping tool

This is a summary of my experiences with different tools for and approaches to prototyping. Prototyping is, eventually, a central area of work for interaction designers, information architects and people with other roles in the software development process.

Why prototyping?

Why bother? The aim of prototyping is to simulate the behavior of digital artifacts before any real development effort has had to be invested. This is driven from the insight that changes get more expensive the later in the development process they are realized – and anyone who has worked in software development knows that changes are inevitable. Change requests surface whenever you walk your project team through a design (no matter on which level of development you are right then). Requirements for an optimal prototype can be derived from this insight: Prototypes musts be

  • fast to create (with little effort)
  • easy to change
  • close to what will actually be built later
  • good to communicate (i.e. in a file format that can at least be viewed by the project team members, better even annotated)

With interactive products à la Web2.0 a new approach to prototyping becomes necessary – thinking in static pages doesn't help anymore as pages or elements thereof can look totally different based on system state (e.g., user unknown / known, but not authenticated / authenticated) or on user action (user has finished an action successfully / an error has occurred). It makes sense to introduce modular design into the prototype already – it'll be the basis for the technological realization anyway. This way, the interaction designers have vast degrees of freedom to have modules react onto events in other modules or to have them ignore other events. Based on this, another requirements becomes clear: the tool must support a modular approach to prototyping.

Prototyping techniques

Based on the goal you want to achieve, you can differentiate horizontal and vertical prototypes (assuming you've got limited time). Horizontal prototypes illustrate a website or an application in its whole width, but offer little or no functionality once you start drilling down. Vertical prototypes in most cases show only very rough illustrations of wide areas of a site or an application, but in limited areas, they provide a very deep and detailed view into the site's or app's depths.

If you're about to create a prototype for a usability test of a specific function, in most cases you'll want to build a vertical prototype illustrating this function in depth – and all the surrounding areas only very sketchy. If you'd like to explore the navigational behavior across a whole site, you'd rather build a horizontal prototype with very limited deep functionality.

You can create electronic or paper prototypes. Paper prototypes have got a number of advantages – they're fast to create, can be changed right in interview sessions with test participants, and because they are intentionally not finished, they invite people to playfully interact with them. They can be extremely helpful in very early project phases, because usually that's when the last requirements are being negotiated. The big drawback of paper prototypes is that you'll have to create recurring modules as often as they occur. Also it's difficult to hand them on or to document them.

Electronic prototypes usually have a more polished appearance about them. This can be a disadvantage when it keeps test participants from voicing change requests – because they could think the solution is already near completion. In addition, the participants cannot directly interact with elements of the prototype. On the other hand, a modular approach can (more or less) be realized, and you can hand on designs to the project team. Documentation is a breeze too.

HTML is quite well suited for creating interactive prototypes – if you're savvy enough and / or got the right tools. A nice example is Adobe Golive. Golive allows you to work with objects and recurring elements that can be defined outside the page. It's relatively close to development and the target system, which can become a drawback as well – when the prototype is used as the starting point for developing the actual system. You should by no means fall into that trap; otherwise Alan Cooper's scar tissue will be created: Code that has been adapted over and over again, that is not elegant anymore, and that's by no means easy to maintain.

Of course you can create HTML prototypes without using tools such as Adobe Golive (or Dreamweaver) – it's just more effort. In that case you'd need an environment that can handle modules (and you find yourself deep in the actual development process), or you start creating ugly solutions based on frames within frames to let appear single pages as modules. Both approaches are suboptimal for accelerated prototyping.

I'll exclusively cover electronic prototyping tools in the following chapters.

Prototyping using electronic tools

Microsoft PowerPoint

Microsoft PowerPoint is probably the most widely used prototyping tool. As it is installed on nearly every office PC, it's got the unbeatable advantage that everyone can view a prototype created with it – and can perform edits. This leads to another question, the question of ownership. A prototype must belong to one person, and only this one person is allowed to carry out changes to the prototype. Even if this sounds like lots of work for this one person – this is the only way to ensure that the prototype is consistent within itself and reflects the current state of the discussion. (There may be exceptions to this rule if there are very strict design guide lines that are known to everyone involved in the project and if there is a top-down mechanism in place ensuring that people follow these rules.)

PowerPoint allows for fast putting together and a very rough initial arrangement of elements. Elements can be linked to simulate interactivity to a certain degree. Later, once you've been talking to engineering with a finished PowerPoint-based prototype and you've got a first glimpse onto a possible end result, you'll realize why PowerPoint is actually quite ill-suited for building prototypes: Measurements don't fit at all. You'll fit lots more onto an actual 1024×768 web page or screen than what can be displayed (and read and edited) on a PowerPoint slide. In addition, PowerPoint's scaling functions can tempt users to make elements fit on the screen that would otherwise never fit – which, unfortunately, results in  screens that cannot be built like that anymore.

Another really severe drawback of PowerPoint is that it cannot deal with modules. PowerPoint only works with copy&paste – and once you've distributed an element on 40 slides and need to go at it and change every single instance of it over and over again, you'll realize that something important has been forgotten and / or PowerPoint just isn't the tool of choice. To summarize: Not suited except for very rough idea visualization.


Denim is an innovative tool for early phases of the definition process. You can create pages and their links on different levels of detail, even simulate a certain interactivity. It uses gestures, if available also using a graphics tablet. Circular contextual menus speed up usage quite a bit. The tool, which is written entirely in Java, is available for Mac OS X, other *nix systems and Windows. Unfortunately it's got one major drawback: it's not really stable. In addition, it starts eating system resources like crazy once files become big, and the built-in HTML export functionality only works partially and produces code that, strangely, runs only on Internet Explorer. It's nearly impossible to export to PDF or just to print properly. So: promises to become an extraordinary, very innovative tool, but isn't ready yet.

Apple Interface Builder

Anyone who's got the Mac OS X developer tools (on original Mac OS X system disks or downloadable from can use Interface Builder to create really extraordinary simulations of Mac OS X user interfaces. Simulations? No – these are actual UIs, just without logic and data structure. Guides appear automatically when you start placing elements on the pane, ensuring that the placement is in sync with the latest version of the Apple Human Interface Guidelines. Amazing! Drawbacks: it's only well-suited for application design (and only support app design really well), runs only on Macs, and saves only to the proprietary .nib file format.

Microsoft Visio

Microsoft Visio is actually a tool for creating diagrams of all kinds. It's well suited for this task, and that's what I used to use it for quite a lot. It's also used a lot to create screen design concepts. It's got the big advantage over PowerPoint that the presentation is much closer to reality; you can actually get an impression of how much content to fit onto a screen. Working with modules is possible since Visio 2003 by placing files in files (see this very helpful article over at BoxesAndArrows). Stencils can be used for recurring elements, too. Unfortunately, Visio uses a proprietary file format; it can export into a multitude of different other file and image formats, but none of them is really good. (The same applied to OmniGraffle, by OmniGroup. Unfortunately, GomniGraffle cannot place OmniGraffle files. It's available for the Mac only; Visio is Windows-only.)

Adobe Photoshop

The queen of all image processing tools is used for prototyping purposes as well. As you can combine elements in layers (and layers in folders …), you can simulate a certain degree of modularity. The great possibilities in Photoshop tend to tempt designers to create a much higher-fidelity design of the prototype that what would actually be necessary (or desirable) at a given project stage. At the end there's a rather polished design that cannot be changed as quickly. High-polished designs, in addition, can impress participants in usability tests and keep them from voicing their opinions because everything looks so "finished" already. Photoshop can export to a number of file formats. A drawback lies in the huge data files, which can hinder data exchange. If you want to rather use jpgs or pngs – to reduce the file size –, you again need a tool like PowerPoint to ensure things are viewed in sequence (e.g., in a scenario), or you'll have to create and maintain a naming convention for file names. That's why I feel Photoshop is only partially suited for creating prototypes.

Adobe Illustrator

The BoxesAndArrows article shows that Adobe Illustrator and Visio are the only two graphics tools that support modular prototyping. You create elements in files you place (as PDFs, on Windows, native Illustrator files are supported as well) in other files. If you change anything in the elements, these changes are reflected nearly instantly in the main files as well. Finished versions can be exported to PDF; you can assume that the Adobe Reader is pre-installed on nearly all systems. If – in addition – you've got Adobe Acrobat, you can create coherent presentations and, if you've got enough time, even put in links to simulate a certain degree of interactivity. A drawback of Illustrator (even in the CS3 version) is that it still cannot deal with multi-page documents (even though Freehand had this feature for so many years ago – d'oh.). This can be smoothed out using the PlugIn MultiPage (by HotDoor). Thus, you can combine complete sets of view in one file, source out and maintain individually modules and optimize the resulting exports in Acrobat. Should you also have got InDesign, you can place the finished views (or the modules) in InDesign and create design comments directly there – supporting the life cycle from prototyping to design documentation. One big disadvantage of Illustrator is the steep learning curve you encounter when getting in touch with a complex vector graphics package like this.

Axure RP

Axure RP Pro promises to be a very interesting tool – including integration into Word and HTML export. The demo I've seen recently looked impressive, but it also comes with an impressive price tag.


Personally, in terms of prototypes, I'm currently most productive in Adobe Illustrator, possibly combining it with Acrobat: fast; close enough to the final result, but not too close; modular, that is it supports fast changes; exports files to a commonly readable format (PDF) that can even be annotated with the appropriate tools (e.g. Acrobat Elements). As licenses to Illustrator are normally not available with every office PC the question of ownership of the prototype is solved as well: It rests with a small number of people who stay in control of the prototype.

(This is the translation of a post for my now inactive German-language blog from June 2006 – I've updated some small things wherever necessary.)

Staying motivated

Beautiful article on how to stay motivated in the creative field:

Whether your chosen medium is pictures or language, food or formulas, everyone has the capacity to be creative in their work. But we can often lose our motivation to create, making it difficult to stay focused and excited on a project. So how does one keep their creative well from drying up?

Read more in Kevin Cornell's article on A List Apart.

“Designing Web Navigation” by James Kalbach

James Kalbach: Designing Web Navigation.Get it while it's hot: Jim Kalbach's book "Designing Web Navigation" (O'Reilley) has just been released. In Jim's own words (from his new blog, Experiencing Information):

Since web navigation design touches most other aspects of web site development in some way, the book necessarily paints a broad picture touching on many areas, including things like user research and visual design. But as much as possible the focus throughout remains clearly on creating an effective navigation system. I always try to bring it back home to web navigation whenever the conversation touches other areas.

I can wholeheartedly recommend it (I read some chapters in advance :-)).

On Eye Tracking – an overvalued technique

Recently our Hamburg User Experience Roundtable took place at the offices of the local market research and usability evaluation institute, SirValUse. Our hosts were happy to give an impressive and well-founded talk on one of the techniques they’re using – eye tracking (you can download the slides here – sorry, they are in German). As most of you will know, eye tracking is a technique used to track a subject’s focal point on a given visual stimulus.

It is assumed (and highly probable) that the focal area is also the area of (visual) attention and that properties of the stimulus in the focal areas are perceived, cognitively processed, and more or less consciously interpreted. When the focal areas are plotted over time, so called “heat maps” can be created that give insight on which areas on the stimulus seem to attract the users’ attention and thus are processed best. Now comes the interesting part: It is also assumed that the more attention an area gets, the higher the probability that the user sees, understands and acts upon a given property of the stimulus. So, for a screen design, if a given function is placed in a presumably hot area and that area stays hot during the re-test, everybody is happy.

This is something I don’t understand. For me, there is no direct connection between eye tracking (i.e., visual attention) and user action. Attention in itself is free of value, it is neither positive nor negative. The processing and the interpretation are the important things about stimuli. The fact that users see something doesn’t mean they will click on this thing, perform their tasks faster, or enjoy a better user experience. True, eye tracking can be used to see whether a given ad diverts the users from the content, or whether a certain link is in an area that’s likely to be scanned at all. But on the whole, visual attention just provides the data necessary for a user to decide on further activity. Thus, for me, eye tracking is a technique that can be used to maybe support a usability test or an expert review, but it’s not a usability evaluation method in itself – it’s great for research, and the result should be collected and made available in big books on human perception (just recently, Jakob Nielsen published a new eye tracking-based study in his Alertbox that is basically this kind of fundamental research). My colleague and friend Frank Ollermann, University of Osnabrueck, supports this point:

Basically this is about objectively assessing visual attention. No other technique can assess visual attention in such a timely accurate manner as error-free as eye tracking (provided the research design is sound). That’s what makes eye tracking so valuable for basic research, though less for usability evaluation – as it only provides indirect information on usability. Most information on usability can be assessed with other, less extensive techniques. For example for web design most of the time you’ll just find out whether a given element has been seen or not. (…) The question is whether this information could not have been gotten by easier, e.g. through an interview. In lots of cases the client insist on eye tracking because it’s ‘sexy’ and because you can impress other clients or the organization executing the project. So, I’d say: eye tracking is an indispensable technique, but it’s not, however, indispensable in the area of usability evaluation (personal communication, May 9, 2006).

The funny thing is that lots of researchers and practitioners know this fact. But few people say so. There is big money in eye tracking, and the heat maps are great for senior management presentations. I think that’s the biggest reason eye tracking is the latest hype in the usability field.

Usability Testing in Dallas, TX

It’s the third time this year that I’m in the US to conduct usability tests during the ASUG forums. Back in April I was in Atlanta, GA[GP:Atlanta]; in September I spent some time in Phoenix, AZ[GP:Phoenix], and now I’m here in Dallas, TX[GP:Dallas]. Time and again I feel it’s extremely fascinating to get into a conversation with our (SAP’s) customers and not only to discuss our new solutions but also to ground my own perception of the status quo at the customers. Lots of our customers still use comparatively ancient versions of our ERP software. The products we’re testing aren’t just the next generation: they constitute a quantum leap in functionality and user experience. On the one hand this is scary; on the other this means our test participants’ experience has not yet been influenced through other, earlier products.