In Part 1, we spoke about why we are trying to address in-car HMI and what we are aiming to achieve with this five-part thought piece.
This section deals with the current state of HMI and the organisational, cognitive and regulatory issues within. We believe that in order to be able to offer innovative new approaches and ideas, we first need to look back and understand the issues. How has the manufacturing industry coped with advances in digital technology? How has interaction between humans and cars changed? What cognitive difficulties has this thrown up? And finally, how have regulatory agencies responded?
Organisational & legacy issues
The velocity of growth in digital technology and its adoption has not been matched by automotive manufacturers. Although expert in the design and build of highly functional and emotional hardware, automotive manufacturers often ignore the critical interactions and opportunities between car hardware and HMI.
This is generally due to home-grown complexities — long cycles of engineering and manufacture continually put them behind the curve of computational evolution (Moore’s Law does not apply to large scale manufacturing and materials sciences).
Another roadblock is the siloed team structure within manufacturing industries, where engineering is often isolated from interior UI and interaction design teams.
When working with organisations that have this structure, we’ve experienced a narrowing approach to problem solving and opportunities. As a result, cutting-edge technology is not used to its best advantage and the basic theories underlying cognition and human–machine interaction are not fully applied.
Even when teams do work together, it is often too late to share disruptive product and experience ideas. This is evidenced by the ‘stick a touchscreen on it’ phenomenon, seen on many appliances and in-car interfaces.
The motor industry also tends to rely on a ‘couture’ methodology for car design, whereby very high level concepts are brought to life, with the aim of wowing motor show attendees. In order to get some semblance of the concept into an actual production vehicle, designers have to ‘over-egg’ the idea in anticipation of its dilution.
And today we are adding the digital user interface. The two large monitors in the new S-Class, each measuring 12.3 inches, currently represent a benchmark.
This approach, although effective, does not suit concepts that require a high degree of user interaction with new technology. This is because in order to get these experiences right, you need to develop the user experience with real users, in near-real situations. We believe that a methodology is needed that allows concept, user experience and user interface ideas to be developed in a collaborative and effective manner. We will cover this in detail later in this series.
Cognition & muscle memory issues
The nature of driving a car has not changed since the Austin 7 of the early 1920’s. The archetypical automobile, with its specific driving position, steering and control scheme, gauges and even its reliance on the internal combustion engine has not really taken any major evolutionary steps in the last 80 years.
This dogged consistency has helped the automotive industry over the years — a user needs only to learn to drive one car in order to drive any car.
What has been changing significantly is the integration of electronics and, more recently computers, into the HMI (e.g. GPS, telematics, ADAS, and infotainment systems). These elements have introduced new layers of complexity to interactivity, completely changing cognitive models and expectations.
Consider the interior of a Mercedes SL from the 1970s compared with a present day Mercedes SL.
In a typical interaction design fashion, we can break down the analysis of the past and present HMI by looking at:
- Controls: tools or devices which offer control of in-car functions (e.g. a knob on a music player).
- Affordance: the nature of manipulation a control offers, while performing an action (e.g. a knob can be turned about on an axis).
- Feedback: the change or reaction brought about by the controller (e.g. the change in volume when the knob is turned).
- Mapping: developing a ‘feel’ for controls — the ability to understand what a control does and where it is located (an accomplished pianist can play the instrument blind, by ‘feel’ or muscle memory).
- Learnability: the ability to understand the way a control behaves over time (e.g. turn the knob clockwise to increase volume).
- Modes: the number of ways a tool or device can be used and repurposed by switching to a new function (e.g. the same knob can also be used to control brightness).
In the car interiors of yesteryear, the spatial arrangement of the dials and knobs allowed for a mental map of the HMI to be developed over time and in turn built into muscle memory.
Since there were fewer electrical complexities, mechanical elements such as knobs, switches and sliders provided both control and direct mechanical feedback — a tangible user interface. There were far fewer modes, because there was direct communication between the elements on the car and the controls.
Learnability was always a factor, but a sparse set of controls meant an inherent simplicity.
With modern HMI it’s a completely different story. There are many more controllable elements within cars, numbering in the hundreds — navigation systems, telematics, ride control and infotainment systems to name but a few.
Nowadays, there is a mix of both tangible and graphical user interfaces (GUI), with the GUI behaving as the primary source of feedback with multiple modes for different in-car functions and systems. Elements within the GUI can be controlled either indirectly via mouse-like devices or more recently, by manipulating elements via touch.
This short clip shows how the control knob indirectly controls modes and elements within the GUI of the Mercedes SL 2013. (Watch from 0:59 seconds)Embedded content: https://www.youtube.com/watch?v=--t4xuywkVI
Here is what you see: a screen and a controller…
This adds a fivefold complexity in cognition:
- Shifting between modes: (and learning where they are) does not allow for a single mental map to be built up over time. By way of an example, see the video for how to move between sat-nav and seat position control.
- Ease of Mapping or Learnability of the GUI: Where are the controls and what are their functions? Can they be learnt easily?
- Affordances: how does moving a circular knob relate to movement between modes on the screen? Can we bring back the direct control seen in the cars of yesteryear?
- Feedback: at present there is a reliance on visual feedback in the GUI which can be very distracting. Complimenting visual feedback with aural and/or haptic feedback might be the answer. (Haptic technologies are tactile feedback systems that take advantage of the sensation of touch.)
- Consistency and muscle memory: if a person changes car models or even manufacturers, they have to relearn some of the basic controls from scratch.
With the advent of these multi-modal HMIs, people are now faced with an unprecedented level of complexity, as well as the added pressures of congested driving. This plays a huge role in decision making due to the limited nature of data storage and access in the human mind.
Limitations of cognition & ‘The Working Memory Model’
To understand the limitations of cognition, we first need to look into how people consciously engage with the world through the construction of long-term and short-term memories and their retrieval over time.
The ‘Working Memory Model’ put forward by Baddeley & Hitch in 2000 (first in 1974) describes the interplay between a ‘crystallized’ long-term memory system, a ‘fluid’ short-term memory system and a limited capacity ‘episodic buffer’. The crystallised long-term memory system comprises language, knowledge of shapes and forms, and muscle memory, whereas fluid memory tends more to the visual and auditory.
(For further information, see these related papers and talks.)
One can consider the act of driving to be crystallised, since it uses long-term muscle memory, whereas the information consumed around that task of driving — Who’s walking in front of the car? Where am I driving to? What am I listening to? — can be considered to be part of a conscious experience and in the episodic buffer at any particular time.
The episodic buffer has a limited capacity for taking in visual, auditory and motor cues, so a HMI which does not allow for the easy construction of skill or muscle memory can easily overtax it. This in turn leads to bad decision-making and frustration on the part of the driver due to cognitive overload.
Over the years human–computer interaction studies have postulated several key methods for tackling the problem of cognitive overload. One such method is the concept of ‘Chunking’ as proposed by George Miller in 1956, whereby an individual can remember or process only seven chunks of information, in their correct serial-order, in his working memory. This essentially means that grouped items are easier to recall, because grouping assists phonological and visuospatial memory.
The concept of chunking also seems to apply to visual forms — similarly grouped shapes or patterns are also easier to remember. This is known as the Gestalt principle of proximity. (Further evidence for this concept is suggested in studies here.)
We will speak about employing these methods in actual HMI design in later sections of this thought series.
Frustration, confusion & safety issues
The complexities of in-car HMI, combined with the limitations of human memory may result in cognitive overload for drivers.
As a result, minimising distraction and reducing driver error has been the focus of law and policy makers worldwide. Efforts are being made to curb the use of distracting devices (eg mobile interfaces) which in turn also influences the design of HMI.
For instance, the European statement of principles on in-vehicle HMI issued by the Commission of the European Communities states:
- The system should be designed to support the driver and should not give rise to potentially hazardous behaviour by the driver or other road users.
- The system should be designed in such a way that the allocation of driver attention to the system displays or controls remain compatible with the attentional demand of the driving situation.
- The system should be designed so as not to distract or visually entertain the driver.
Similarly, in the USA the National Highway Traffic Safety Administration guidelines state that:
- The driver’s eyes should usually be looking at the road ahead.
- The driver should be able to keep at least one hand on the steering wheel while performing a secondary task (both driving-related and non-driving related).
- The distraction induced by any secondary task performed while driving should not exceed that associated with a baseline reference task (manual radio tuning).
- Any task performed by a driver should be interruptible at any time.
- The driver, not the system / device, should control the pace of task interactions.
- Displays should be easy for the driver to see and content presented should be easily discernible.
So, it is fair to say that many of the issues we currently face in HMI design are borne of the legacy of the space and the automotive industry infrastructure. The siloed nature of design teams means that while new functions may be introduced, organisational structure might not allow for the sensitive incorporation of these features.
This leads on to learnability issues, not to mention frustration and the obvious safety concerns as put forward by regulatory agencies. Though manufacturers are definitely aware of these issues, it is our view that many of them could be mitigated early and rapidly, if a “one-team” collaborative approach was adopted, where they can design, build and test ideas with real users to learn and iterate accordingly.
__text in bold__In Part 3 we look at some of the methods employed by manufacturers to try and tackle these issues, and the changing patterns in this field.