Google Glass

Hayes led the interaction research team for Google Glass and was part of the core team from the time team members started making custom hardware through shipping our v1 and v2 hardware. He focused on developing new experiences which are uniquely enabled by the form factor. Wink to Photo and notification glance emphasized hands-free control. Entertainment (music and earbuds), novel input methods (eye gestures, head movement, voice and more), and Mini Games illustrate the Interaction Research, design strategy, technology development, and system prototyping Hayes’ team conducted.

Story Visit

Story Visit is a website where children and long-distance loved ones can read story books together. Story Visit combines video conferencing and connected story books: when a grown-up turns the page, the child’s page turns along with it. If the child points to something on his screen, the grown-up can see that on her screen too. Now families can read stories together even when they can’t be in the same place.

Our research showed that conversations with 3 year olds increased from several minutes (in a typical video call) to 15-20 minutes when they used Story Visit. Reading books together really is more fun!

Story Time for the 21st Century

Hayes Raffle, Rafael “Tico” Ballagas, Glenda Revelle, et al., in Collaboration with Sesame Workshop

Family Story Play

Family Story Play supports grandparents to read books together with their grandchildren over the Internet. Story Play improves communication across generations and over a distance, and to supports parents and grandparents in fostering the literacy development of young children.

The interface encourages active child participation in the book reading experience by combining a paper book, a sensor-enhanced frame, video conferencing technology, and video content of a Sesame Street Muppet (Elmo). These features improve child engagement in long-distance communication. Story Play also encourages dialogic reading styles that are linked with literacy development.

Family Story Play leverages a synergy between young children’s education—in this case, a rich shared reading experience—and communication with long-distance family. Our goal is to create new kinds of toys and tools that support creativity, learning, and family togetherness.

Katie and her dad read, and then mimic Grover in the story while the grandparent reads to them over video chat.

Sound of Touch


Hayes Raffle
& David Merrill, with Roberto Aimi
Steel, wood, custom electronics, audio processing software running on a PC, and various materials

The Sound of Touch
brushing-beige-wide2.jpg
Sound Of Touch

Sound of Touch is a semi-acoustic instrument for real-time capture and sensitive physical stimulation of sound samples using digital convolution. A hand-held wand can be used to record sound and then playback the recording by brushing, scraping, striking or otherwise physically manipulating the wand against physical objects. During playback, the recorded sound is continuously filtered by the acoustic interaction of the wand and the material being touched.

{website | papers | press | exhibitions}

Freestanding Texture kits invite visitors to sculpt sound by exploring the visual, tactile and acoustic ranges of familiar materials. Their designs are inspired by Mondrian’s mid-career abstract paintings of blocks and lines, and in the texture kits, familiar materials are decontextualized as repurposeable media content.

While an acoustic instrument’s resonance is typically determined by the materials from which it is built, digital audio tools are usually divorced from the world of physical acoustics. With the Sound of Touch, resonant materials can be chosen during the performance itself, allowing performers to shape the acoustics of digital sounds by leveraging their intuitions for the acoustics of physical objects. The Sound of Touch permits real-time exploitation of the sonic properties of a physical environment, to achieve a rich and expressive control of digital sound that is not typically possible in electronic sound synthesis and control systems.

Future of Play

As a researcher At the MIT media lab I have been pursuing my vision to create interactive toys that people bring to life through play and learning. My expertise is in designing products that navigate the boundary between the physical and digital worlds, building on over ten years inventing and bringing innovative designs to market. My work involves conceiving new ideas for interactive media, and helping manage prototyping, testing and marketing of those ideas. My goal is to reinvent children’s interactive technologies to reflect children’s passions, creativity and encourage collaboration.

Topobo moose

Topobo is a 3D constructive assembly system with kinetic memory, the ability to record and playback physical motion. You can build a dog with Topobo, wiggle its body around with your hands and teach it to walk. The dog will then repeat your motions repeatedly. The same way kids can learn about static structures playing with blocks, they can learn about dynamic structures playing with Topobo. In this video interview with the Science Channel I explain the system in more detail.

Topobo integrates all of the things I love about design – a clean and emotionally engaging concept, strong aesthetics, clear interaction and enough flexibility for the user to put their own personality into the product. Furthermore, designing Topobo has been an amazing collaboration, and I have been able to learn skills from product designers, graphic designers, electrical and computer engineers, educators and kids. We have tested Topobo in classrooms with kids ages 5-14 throughout the design cycle. These interactions were enlightening for us, giving us insight how others made meaning of the system, seeing people build and animate creations we’d never imagined, and getting feedback about how to improve Topobo in future iterations. To our delight, both 2nd and 8th graders told us they thought Topobo was designed for them!

Our work led to four peer-reviewed conference papers, an ID Magazine Design Distinction, a Prix Ars Electronica Honorable Mention for interactive art, and numerous international shows in art, design and education. The system has also had a mature design cycle. In 2004 I received a Microsoft iCampus grant to conduct longitudinal research studies, and pursued a complete two-year redesign of the system including production manufacturing in Hong Kong. My experience conducting design work at the Chinese factory has given me a deeper understanding of the full design and manufacturing cycle of interactive media products.
Kids playing with Topobo

Jabberstamp - girls drawing+recording

Jabberstamp is the the first tool that allows children to synthesize their drawings and voices. To use Jabberstamp, children create drawings, collages or paintings on normal paper. They press a special rubber stamp onto the page to record sounds into their drawings. When children touch the marks of the stamp with a small trumpet, they can hear the sounds playback, retelling the stories they have created.

Children ages 4+ use Jabberstamp to embed names, narratives, characters’ voices and environmental sound effects in their original drawings. Children’s compositions help them communicate their stories with peers and adults, and allow them to record and situate stories in personally meaningful contexts before they have mastered writing.

I originally stumbled upon toy invention through an artist internship that helped shape my undergraduate sculpture studies at Yale. One summer I helped artist Michael Joaquin Grey conceive and design a new building toy called ZOOB, a haptic modeling system that allows people to reflexively learn about biological structures like bones and molecules. I later helped Michael start up a design and manufacturing company to bring ZOOB to market.  ZOOB was a great success—we did over $3M in our first year, and won a number of awards including Dr. Toy Best Toy, Toy of the Year and ID Magazine Design Distinction for consumer products. Over the next several years I directed the company’s internal design department, overseeing all of the company’s print and product design.
zoob-lineup.jpg

After ZOOB, my independent art and design work led me to the MIT Media Lab, where I joined the Tangible Media Group to develop my technical and conceptual skills developing electronic media. My early work at MIT was driven by an ambition to create new product designs that engage people’s kinesthetic sense and love of play to support communication, entertainment and learning.

Super Cilia Skin
Super Cilia Skin is a literal membrane separating a computer from its environment. Like our skin, it is haptic I/O membrane that can sense and simulate movement and wind flow. Our intention is to have it be universally applied to sheath any surface.  As a display, it can mimic another person’s gesture over a distance via a form of tangible telepresence. A hand-sized interface covered with Super Cilia Skin would produce subtle changes in surface texture that feel much like a telepresent "butterfly kiss."  

Our extensive design studies spanned dozens of physical and electronic prototypes and led to a technical paper in CHI and a full journal paper in "Textile: The Journal of Cloth and Culture" (Berg 2004). The investigation focused on the haptics and scale: as a product, Super Cilia Skin is a "touch telephone." As wallpaper or carpet, we considered the potential for actuated, telepresent interior design. At the tectonic scale, sheathing tall buildings, the interface assumes dual roles as billboard size display or as wind-driven electromagnetic power generators.

The You’re In Control system uses computation to enhance the act of urination. Sensors in the back of a urinal detect the position of impact of a stream of urine, enabling the user to play interactive games on a screen mounted above the urinal.

In order to allow men and women to publicly interact with YIC, we built a customized game controller that can be strapped around the waist and pressurized to create a stream of water. Players can toggle between the "Flying Hamsters" and "You’re A Nation" games by flushing the urinal. It really is a pissing contest!

You’re In Control (Urine Control) interactive gaming system

About fifty years ago my grandfather invented a polymer clay called Sculpey and reinvented how children can create and express themselves. His life as an inventor and entrepreneur inspires me to imagine how electronic toys can be more than flashy gadgets or media conduits, and actually amplify children’s creativity, expression and playful learning.

+Hayes Raffle

{ contact | cv | portfolio }

 

Fuzzmail

Fuzzmail logo
Fuzzmail screenshot

With email, why does a love letter have to look the same as a business letter? We invented Fuzzmail to give people’s typed communications more emotional breadth. Fuzzmail records the act of writing and lets the author send it as an email. Dynamic changes, typoes, pauses and writeovers are captured and communicated. Fuzzmail was created to provide a more emotionally expressive alternative to email, so that an emailed love letter does not have to look the same as a business letter.

You can check out one of my favorites about family and caffeine here.

Fuzzmail allows you to create more expressive text communications by incorporating dramatic timing and rhythm into the content of the written message. The act of writing becomes more like singing or acting. A fuzzmail may be a carefully scripted performance or a spontaneous creation.

Give Fuzzmail a try at http://www.fuzzmail.org.

* Winner of the 2007 Web 2.0 Awards Honorable Mention, “Fun Stuff.”

Super Cilia Skin

Super Cilia Skin
Super Cilia Skin hand front
Super Cilia Skin hand

Natura maxime miranda in minimis
(Nature is greatest in little things)

Super Cilia Skin is a literal membrane separating a computer from its environment. Like our skin, it is haptic I/O membrane that can sense and simulate movement and wind flow. Our intention is to have it be universally applied to sheath any surface.  As a display, it can mimic another person’s gesture over a distance via a form of tangible telepresence.  A hand-sized interface covered with Super Cilia Skin would produce subtle changes in surface texture that feel much like a telepresent “butterfly kiss.”

A small object surrounded with Super Cilia Skin could propel itself across the floor, or stop to create
visually expressive changes in surface. Conversely, a Super Cilia Skin surface could propel objects across it using mechanical gestures like the movements of a centipede’s feet.

{website | video}

Publications

Included in Inform Material, Thema. 2008.

Included in Brownell, Blaine (Ed). Transmaterial 2: A Catalog of Materials That Redefine Our Physical Environment. Princeton, NJ: Princeton University Press (2008).

Raffle, H., and Tichenor, J. Super Cilia Skin: Interfaz de Texturas, Pasajes Construcción: Materiales. Number 31, September 2007, pp. 8-9 and front cover. Madrid: América Ibérica.

Raffle, H., Tichenor, J., and Ishii, H. Super Cilia Skin: A Textural Interface, Textile: The Journal of Cloth and Culture. London: Berg Publishers. 2004.

Raffle, H., Joachim, M., and Tichenor, J. Super Cilia Skin: An Interactive Membrane, Proceedings of CHI 2003.

ZOOB

zoob-lineup.jpg

ZOOB® is the first construction toy based on things that grow, from DNA to dinosaurs. Hayes worked with inventor Michael Grey to conceive and design the initial system. As employee #6 at Primordial, LLC, Hayes helped oversee all aspects of production design, including helping build and manage a design team to create product designs, packaging designs, and videos. The team won an ID Magazine award in the 45th Annual Design Review in Consumer Products and the toy has been popular for over twenty years.