What is Gesture-Based Computing?

It is already common to interact with a new class of devices entirely by using natural gestures. The Microsoft Surface, the iPhone and iPod Touch, the Nintendo Wii, and other gesture-based systems accept input in the form of taps, swipes, and other ways of touching, hand and arm motions, or body movement. These are the first in a growing array of alternative input devices that allow computers to recognize and interpret natural physical gestures as a means of control. We are seeing a gradual shift towards interfaces that adapt to — or are built for — humans and human movements. Gesture-based computing allows users to engage in virtual activities with motion and movement similar to what they would use in the real world, manipulating content intuitively. The idea that natural, comfortable motions can be used to control computers is opening the way to a host of input devices that look and feel very different from the keyboard and mouse — and that enable our devices to infer meaning from the movements and gestures we make.

INSTRUCTIONS: Enter your responses to the questions below. This is most easily done by moving your cursor to the end of the last item and pressing RETURN to create a new bullet point. Please include URLs whenever you can (full URLs will automatically be turned into hyperlinks; please type them out rather than using the linking tools in the toolbar).

Please "sign" your contributions by marking with the code of 4 tildes (~) in a row so that we can follow up with you if we need additional information or leads to examples- this produces a signature when the page is updated, like this: - alan alan Jan 27, 2010

(1) How might this technology be relevant to the educational sector you know best?

  • Of all of the topics considered for this year's Horizon report, this one is probably the most 'under the radar' in its potential impact -- largely because everyone is already quite familiar with it, and it is increasingly entrenched in the lives of many via the devices listed above (and also as a result of multi-touch now enabled in the Nexus One, and soon to be turned on in other Android devices; new gesture-based interactions expected in videogaming through things like Project Natal; the upcoming Apple iPad; applications written to take advantage of the touch capabilities built into Windows 7, etc.). We think we know this one ... but there are great implications for education when you remove the and liberate gesture-based interactions from the artificial constraints imposed the tyranny of point-and-click paradigm via mouse/finger interaction. - michael.trucano michael.trucano Feb 8, 2010
  • Gesture based computing is by its nature intuitive and engaging. It allows interaction by young learners, and also it may engage learners for whom the traditional forms of input and output via keyboard and mouse cause difficulty.- Gavin Gavin Feb 8, 2010

(2) What themes are missing from the above description that you think are important?

  • The potential impact of this interaction modality on learning opportunities for illiterate and low-literate learners is important to mention. - michael.trucano michael.trucano Feb 8, 2010
  • One area that it seems to me is not well understood, is how the context in which we learn affects how and what we learn. Gesture based computing can allow many different contexts for learning, not simply sitting quietly, but much more activity with learning.- Gavin Gavin Feb 8, 2010
  • A further area not mentioned is that gesture based computer may particularly help some learners who for a variety of reasons may have difficulty with traditional input and output devices.- Gavin Gavin Feb 8, 2010

(3) What do you see as the potential impact of this technology on teaching, learning, or creative expression?

  • One immediate potential implication is on the use of ICT devices to support a variety of activities in early childhood development (ECD). Young children may not know how to read or spell or type (well), but everything knows how to grab and pull and point and scratch, etc. - michael.trucano michael.trucano Feb 8, 2010
  • Agree with Mike, and this may stretch to activity among older learners who do not necessary share language, ability to use a keyboard or some of the assumed behaviours that we adopt once we become experienced computer users.- Gavin Gavin Feb 8, 2010

(4) Do you have or know of a project working in this area?

  • Interactive Whiteboard makers, Surface development, Dance Mats, Wii games are being used but more digging needed to find current projects.- Gavin Gavin Feb 8, 2010
Please share information about related projects in our Horizon K-12 Project form.