In the comments of some articles on natural user interfaces you can read that NUIs are just graphical user interfaces without any differences.
For a some simple definitions of a GUI, like the one from Wikipedia:
- “A graphical user interface is a software component that allows the users of a computer to interact with the machine via graphical icons.”
this might apply, because visual interfaces are graphic in nature.
As a graphical interface, however, constitutes more than pictorial symbols, a more IT-specific definition is needed:
- “A Graphical User Interface (GUI) refers to the graphical user interface, which make the use of computer easier …. Graphical User Interfaces are clearly structured and include several functional areas, such as the menu bar, toolbar, toolbar, status bar, task bar and scroll bars. These bars contain information, commands or graphical icons that can be accessed by clicking them with the mouse …. “
With this definition of ITWissen.info you can already see the first differences between the GUI and NUI. Although many natural surfaces also have menues, or appropriate alternatives to a menu, but are usually dispensed with other elements.
According to Prof. Wolfgang Hensler (NUI Expert from Germany), this is due to the fact that most display devices with natural input capabilities are relatively small and have no space for displaying all options, on the other, NUI are limited to the essential items, so that the user can quickly summarizes the relevant options.
Content, such as menus and settings only emerge on the screen when they are relevant to the user.
For this purpose, each object, whether word or symbol, has its own context-dependent functions which you can call with one click. Instead of selecting a line (in writing program) with the mouse and press the button in the toolbar for “bold”, the user marks the part and the possible options are displayed automatically.
Thus, the user must not bother with nested menus or unknown keyboard shortcut.
Another difference to graphical user interfaces are the intuitive input methods of NUI interfaces.
Where users previously had to use keys, buttons or a computer mouse, now language, touch, or body movements are used to control a device. Thus, the user can interact directly with the elements on the screen without an additional device, such as the mouse. Thereby a seamless interaction between humans and computers has been created by which you can handle virtual objects in a realistic manner.
Therefore a picture in a photo application on a multi-touch device can be handled as if it woud be directly on the table, except that there are more options available with the digital technology than just to turn it.
The differences between the GUI and NUI do not end with the display and control of objects. Besides these obvious issues there are program-technical and social aspects that are not immediately recognizable to each user.
Technical aspects of programming
Besides the fact that touch-devices work with multiple touch points rather than mouse clicks, various aspects in the development of NUI applications must be considered.
For one, the mouse-cursor is missing in natural systems. Thus, hover or mouseover effects are no longer possible. These events, which are triggered when the user moves his mouse cursor over an object in GUI applications, usually are used to display further information of an image in an infobox.
The absence of the mouse, however, has other effects on the user interface. Touch surfaces that have to be controlled just by touch, have controls that must be tailored to the size of a fingertip. The pixel-level control, as is possible with a mouse pointer, is no longer possible on natural user interfaces. Experts advise that NUI buttons for applications with touch input are accordingly construed to this fact.
For small screens, such as a smartphone, a minimum size of 5 mm x 5 mm is recommended for a button, for larger displays objects with at least 1cm x 1cm should be used.
- „Specifically, this means that the size of the interaction elements must not meet the cursor, but the finger.“,
says Prof. Wolfgang Hensler.
People who use equipment with natural user interfaces behave, from a social point of view, different from people who use conventional computer with a graphical user interface.
This is partly due to the multi-user capability. While on normal computers, only one person can work actively, with NUI devices a co-operative work on a device is possible.
Microsoft’s chief designer August de los Reyes said:
- “When you plug in your music player, or are in a meeting where people pull out their laptops, suddenly there is a barrier between people.
When people gather around a Surface table, this barrier disappears.
The technology steps into the background, and the relationship of the users around the technology is being strengthened”
But you can not only make such statements inrelation to the work environment. In the public sector are examples where barriers between people who use a multi-touch wall, for example, dissolve.
James Hammond, principal analyst at Forrester Research Inc. in Cambridge, described it in a blog entry his experiences that he got on a business trip at the airport.
When he was on his way from a lounge at the American Airlines flight, he came across a multi-touch wall and watched the visitors, who were playing around on her.
- “I’m sure you know people that are afraid of computers and while maybe it’s not so much of an issue with the Millennial generation it was interesting for me to watch folks walk right up and start playing with the screen, with no instructions or familiarity with the interface at all. This really speaks to Mike’s second criteria, the natural user interface was highly usable. It provided some basic cues that we’re used to like selectable windows (rectangles of course, because that’s how we interact with the Internet, right?). In no time at all regular folks we’re playing with the screen, bringing up multiple data feeds and moving them around – and I’m pretty sure they weren’t IT professionals.
Because the screen was massive multi-touch more than one user could interact with it at a time. As two or three people we’re working at their own little section, I noticed that several times they began looking at what the other folks we’re doing, and even talking about their experience. In effect, some real-time social computing was taking place, not just the faceless kind with the Internet in the middle.”
The intuitive and natural use of the multi-touch wall helped to encourage people to try out new equipment they were not familiar with. In addition, you can read in the text that the control is easy to learn even for computer-inexperienced user and does not require any briefing.
Moreover, in this case, the social interaction with strangers would not have occurred if it the multi-touch wall would have been a normal information terminal with a keyboard and mouse.
This difference in social behavior clearly shows the positive aspect that a device with a natural user interface has opposed to conventional computers with graphical user interface.
Upcoming blog entries
Making The Game – Part 9: Benefits of natural input options using practical examples
Making The Game – Part 10: Game Design