Revealing unawareness in usability related decision-making

Nowadays, many users experience usability issues with their electronic products. It does not work as they expect or otherwise irritates the user, so he becomes dissatisfied about the product and may even complain about it. These numbers of complaints to companies and usability issues are high and rising. Reasons for these increasing numbers are the highly complex electronic products that are being developed, the global economy in which they are created and produced, and the wide variety of users that uses the product.

It is highly challenging to develop these increasingly complex products, with their even more complex interactions and for the wide variety of users. So design for usability is becoming ever more important. Many usability techniques are available to create these products, however the pressure from the market results in limited time to develop usable products. This makes that feedback from the market is not available on time, that usability tests are skipped, and hasty decisions are made. This all, may lead to ‘incorrect’ decisions and consequently to usability issues. These are some of the aspects that complicate creating products and make that usability issues still occur, despite the available usability techniques. In design practice decision-making was investigated to find out what makes usability related decision-making ‘go wrong’.

The relevance of this research is to improve usability related decision-making in design projects, thereby improving the product usability. A benefit of improved product usability is the competitive advantage for companies. Nowadays products are becoming increasingly complex, with ever more complex interactions and interfaces -   usability design is becoming ever more critical.

This PhD research project aims to improve the usability related decision-making process in design practice. It explores the critical factors that influence the quality of usability decision-making.

 

 

Method and Results

The research started with a literature study to clarify terms such as design and decision-making. Designing products is an iterative process to create products of which problem solving and finding creative solutions are part of. Within this process the designer encounters different kinds of problems; structured problems and ill-structured problems. Each of these problems requires different approaches; Rational Problem Solving for structured problems and Reflective Practice for ill-structured problems. Both approaches are necessary in the process of creating products.

In design theory only limited literature on decision-making is available, therefore it was required to study decision-making in fields beyond design. Naturalistic Decision Making (NDM) literature appeared to be an interesting perspective on decision-making, possibly relevant to design. NDM researchers observed decision-making in the ‘real world’, outside the laboratories. They realised that Rational Problem Solving is not always possible in the ‘real world’ as decision-makers have also to cope with ill-structured problems and other influencing factors and elements.

NDM researchers identified two factors that influence decision-making; the ‘real world context’ and ‘uncertainty’. These factors influence the decision task, making it more difficult to make decisions, and affecting the quality of decisions possibly resulting in ‘incorrect’ decisions. The factor ‘real world context’ is characterised by various elements. The influencing factor ‘uncertainty’ is defined by the types what the decision-maker can be uncertain of and by the sources that induce uncertainty. Comparing these influencing factors from NDM theory with design theory showed a similarity between the elements that characterise the context and the aspects of uncertainty. This suggested that these influencing factors could also be relevant to decision-making in design. Therefore the NDM perspective was used to investigate usability related decision-making in design practice.

Knowing what influences decision-making indicates how decision-making could be improved in order to reduce the number of usability issues. A first explorative study (Study 1) at a Dutch design agency was conducted to identify possible influencing factors on usability related decision-making. The results of eight retrospective interviews revealed three influencing factors on usability related decision-making:

  • design context
  • uncertainty
  • unawareness

The third influencing factor, which was not mentioned in literature, might be a critical factor as it could lead to unexpected surprises such as usability issues. Further investigation of this factor was required to verify whether unawareness actually is an influencing factor on usability related decision-making in design practice and whether it is critical or not.

A second study (Study 2) was conducted at a multinational product development company. At that time unawareness was still an undefined term. In order to identify unawareness the consequences of this influencing factor were investigated; unforeseen usability issues. This was done by conducting retrospective interviews with 14 key team members, which revealed various unforeseen usability issues. Tracing them back, it could be concluded that unawareness plays a role during decision-making in design practice and that it is a critical influencing factor as various unforeseen usability issues resulted from it. It was decided to further investigate this factor, being so critical and unknown.

The third study (Study 3) was a retrospective study at a multinational development company to obtain a better understanding of the influencing factor ‘unawareness’. The results were based on a document analysis of 2.056 project documents. This study provided detailed examples of unawareness during decision-making that resulted in unforeseen usability issues. A description of unawareness was made based on these various examples of unawareness in design practice. Unawareness is described – similar to uncertainty – by its types and sources.

Three types of unawareness were identified to describe what the decision-maker can be unaware of:

  • unawareness about information
  • unawareness about the consequences
  • unawareness about decisions

Three sources were identified that contribute to unawareness:

  • inadequate consideration
  • inadequate overview
  • fixation

In design practice, it is this influencing factor ‘unawareness’ that contributes to usability related decision-making going ‘wrong’. This in spite of the many available usability techniques, these techniques do not address the sources of unawareness. Unawareness during the decision-making process results in decisions of poor quality, leading to ‘incorrect’ decisions and usability issues. Therefore unawareness in the design process needs to be reduced to improve the quality of usability related decisions. To do so, it is necessary to acknowledge this influencing factor, recognise the sources of the factor and address the sources that induce the unawareness.

Innovation

This PhD research revealed the influencing factor ‘unawareness’. A critical influencing factor on usability related decision-making in design practice. The types and sources of this factor are based on a detailed analysis of a large case study. Usability issues can be reduced by addressing the sources of ‘unawareness’.

Benefits

With the explicit knowledge about the factors that influence usability related decision-making, companies and designers will be able to improve decision-making and thereby improving product usability.

Valorisation

Besides a thesis and academic papers, the results of this research were disseminated by presentations, and workshops.

Leaflet

During the symposium on the 10th November 2011 Christelle Harkema executed two workshops on ‘Improving usability related decision-making’. As a reminder of this workshop all participants received a leaflet with a short description of the workshop and a visualisation of the influencing factors on usability related decisions. This leaflet can be found on the Results page.