O

Sensory Engagement & the Haptic

Gesture & the Screen

Jessica Helfand

Jessica Helfand received both her BA in graphic design and architectural theory and her MFA in graphic design from Yale University. She has taught for fifteen years in the graduate program in graphic design in the School of Art, where she is currently Senior Critic and a Lecturer in Yale College. In the fall of 2012, she will be a visiting artist at Wesleyan University.

She is a member of Alliance Graphique Internationale (AGI) and a recent laureate of the Art Director's Hall of Fame. In 2010, she and William Drenttel were named the first Henry Wolf Residents in Design at the American Academy in Rome. Helfand was appointed by the Postmaster General to the U.S. Citizens Stamp Advisory Committee in 2006, where she chaired the Design Subcommittee until 2012.

Previously Adjunct Professor at New York University's graduate program in Interactive Telecommunications, Helfand is the author of several books, including Screen: Essays on Graphic Design, New Media and Visual Culture (2001) and Reinventing the Wheel (2002), both published by Princeton Architectural Press. She has also written Paul Rand: American Modernist (Winterhouse, 1998). She has lectured at the AIGA National Biennial Conference, Cooper-Hewitt National Design Museum, Walker Art Center, Columbia University School of Journalism, the Annenberg School of Public Policy, and the Netherlands Design Institute, among others.

Paola Antonelli Digital By Design’ forward

Designers have also matured beyond the first moments of irresistible and immoderate enthusiasm for the new mediums, and learned to control their touch and to wear technology, instead of letting technology wear them.

DIGITAL BY DESIGN (2008)

Foreword
by Paola Antonelli

Whether they use the skins and shells of objects as an interface or animate them from within, artists and designers are set on a path that will transform the world into an information parkour and enrich our lives with emotion and motion, direction, depth and freedom. It is a revolution that started several years ago, but technology – a wide word whose submenu for artists’ consumption could read information, digital, nano-, bio-, and any other prefix that inspires a trip to the lab – is today available to a wider range of thinkers and doers, and sophisticated enough to be modulated with the lightness and precision of a laser (thirty years ago a hatchet would have done the trick). Artists and designers have also matured beyond the first moments of irrepressible and immoderate enthusiasm for the new mediums, and learned to control their touch and to wear technology, instead of letting technology wear them. To experience the works in this book – a highly distilled sample of today’s best design and art production that uses technology in creative and unexpected ways – is to know how wonderful our present age can be.

Indeed, there is a touch of wide-eyed wonderment in every project, along with a friendly competitive feeling and the desire to share in the joy of discovery that is typical of communities of ninja-geek tinkerers who share unrequited passions. The great John Seely Brown, often considered one of the historical enablers of this technology transfer, calls it ‘thinkering’, to emphasize the high intellectual and innovative benefits that come from it. Indeed, in the hands of these brilliant artists, technology becomes a way to rediscover human nature, to push our most traditional buttons and also new ones we did not know we had, to make humans tick in a novel way. It sets us on the course of creation, collaboration, communication and construction that has become an impellent necessity and the only way to real future progress.

Paola Antonelli is Senior Curator in the Department of Architecture and Design at the Museum of Modern Art. In 2004 she was recognised as one of the top one hundred most powerful people in the artworld by Art Review.Paola has curated several architecture and design exhibitions in Italy, France, and Japan. She has been a Contributing Editor for Domus magazine (1987-91) and the Design Editor of Abitare (1992-94). She has also contributed articles to several publications, among them Metropolis, the Harvard Design Review, I.D. magazine, Paper, Metropolitan Home, Harper's Bazaar and Nest.

Gesture and the Interface

I believe we will look back on 2010 as the year we expanded beyond the mouse and keyboard and started incorporating more natural forms of interaction such as touch, speech, gestures, handwriting, and vision--what computer scientists call the "NUI" or natural user interface.

--Steve Ballmer, CEO Microsoft Ballmer, S. "CES 2010: A Transforming Trend -- The Natural User Interface."

http://www.huffingtonpost.com/steve-ballmer/ces-2010-a-transforming-t_b_416598.html

Mahir M. Yavuz & Javier Lloret 2009 ‘Newshaper’

Newshaper is a tabletop interactive system which deals with re-contextualization of daily news items gathered from live news feeds. It allows users to create their own relational networks and 'emotional maps' based on actual live news streamed from internet. Due to the dynamic nature of live data streams system constantly creates different content for users and due to manipulations done by the users it generates different visual results of the same content. In a more general sense, users of Newshaper system shape their own perception about the daily news and share their vision with other users. System allows them to save their designed structure.


Most of the time, mass media sources deliver the daily news in a context. This context creates a meta-meaning that usually effects the simple meaning of the news. By tagging, by categorizing, by geo-locating, by making it popular or unpopular mass media is shaping the perception of the daily news items in a different perspective. Newshaper allows their users to break this pre-defined structures which have been delivered by the source. After users shape their own news scape, system also allows to see the comparison between pre-defined structure and user defined structure in a visual way.


By using Newshaper users might define their feelings about news and importance of each item. The users can save their mood and the news map they created as a proof of that particular moment. Average of all sessions recorded by users creates the overall mood of the day in order to display the common thoughts and feelings about the actual news provided by media sources.

Body Paint by Mehmet Akten

“Body Paint” by Mehmet Akten is an interactive installation and performance allowing users to paint on a virtual canvas with their body, interpreting gestures and dance into evolving compositions.

Custom software analyzes live feed from infra-red cameras in real-time, and converts shape and motion into colors, drips and brush-strokes. The software was written in C++ using the open-source toolkit openFrameworks and computer vision library openCV.

The installation is designed to work with any number of people and is scalable to cover small or large areas. The interaction is very simple – movement creates paint. Hidden in the simplicity, are many layers of subtle details. Different aspects of the motion – size, speed, acceleration, curvature, distance all have an effect on the outcome – strokes, splashes, drips, spirals – and is left up to the users to play and discover.

While the installation is suitable for a single user, when multiple users are present a new dynamic emerges between people. A user-to-user interaction is born when the audience start playing with each other through the installation, throwing virtual paint at each other, trying to splash their friends, working collaboratively to create shared artwork, or mischievously trying to vandalize others’ work.

Reactive Environments

Artworks have left the screen, to manifest the virtual sphere in palpable spatial experiences’ Joachim Sauter ‘A Touch of Code’ p.5

Jason Bruges Studio 2010

Commissioned for Milan 2010, Jason Bruges Studio’s latest creation is a captivating artwork featuring Lumiblade OLEDs.
Mimosa is an interactive artwork displaying behavior that mimics responsive plant systems.The piece was inspired by the Mimosa family of plants, which change kinetically to suit their environmental conditions.
The studio has used the slim form of individual OLEDs to create delicate light petals, forming flowers, which open and close in response to visitors.

http://www.jasonbruges.com/projects/international-projects/mimosa

Jessica Helfand ‘Dematerialisation of Screen Space’ 2001

In interactive environments, the promenade - and its implicit digressions - are as important as the destinations’ p122 (Graphic Design Theory - readings from the Field)

Rain Room’ Random International 2012

http://www.bbc.co.uk/news/uk-england-london-19873953

Lev ManovichThe Language of New Media2001

The spectator is no longer chained, immobilised, anesthetized by the apparatus that serves her ready-made images; now she has to work, to speak, in order to see’ p.109
United Visual Artists ‘High Arctic’ 2011

http://www.uva.co.uk/work/high-arctic#/10

Kenya HaraDesigning Design 2007

Quality of information offered when senses have been mobilised
‘Haptic’ senses
P126 (Graphic Design Theory - readings from the Field)
Robb Godshaw ‘Cryoscope’

http://www.kickstarter.com/projects/robbgodshaw/cryoscope-a-touch-based-weather-communicating-devi

Active Objects

Thus far, 21st-century culture is centered on interaction: “I communicate, therefore I am” is the defining affirmation of contemporary existence, and objects and systems that were once charged only with formal elegance and functional soundness are now also expected to have personalities.

http://www.moma.org/interactives/exhibitions/2011/talktome/

interactions » archives » XVII.1 - January / February, 2010

The hand ax reminds us that design has always been about interaction, and interaction has always been tangible. What’s new is that physical interaction is becoming computationally mediated-or conversely, that computational media are becoming physically embodied.
Tangible Interaction = Form + ComputingMark Baskinger, Mark Gross
Tom Jenkins 2004


http://www.tom-jenkins.net/projects/audioshaker.htm

Human Computer Balance

Household 2010 ‘Human Error’ http://www.house-hold.org/projects/human_error.php
Katharina Holzl 2011 ‘Music Box – Business Card’