Raise Both Hands and Say, Yeah: Multitouch Technology Is Here
Rocky Mountain Internet Users Group
Minutes of the 14 July 2009 meeting, "Raise Both Hands and Say, Yeah:
Multitouch Technology Is Here"
About 25 people attended tonight's meeting. Josh Zapin facilitated and
Jeremy Kohler recorded the minutes.
Microstaff (www.microstaff.com) provides refreshments, Copy Diva
(www.copydiva.com) provides the audio-visual equipment, NCAR
(www.ncar.ucar.edu) provides the facility, and ONEWARE (www.oneware.com)
sponsors these minutes.
Thanks to Joe Spinello for recording the podcast.
Look for a posting on Craig's List and RMIUG for a job opening in a software
consulting/cable software company. Looking for a mainstream Java developer,
web services, agile programming.
There's a good java user group that meets at CU.
INTRODUCTION (Josh Zapin)
Engineers have always been trying to find ways to interact with the human
body. Your car's steering wheel is an example of an elegant interface. Your
VCR's programming interface is probably an example of a bad one. For
computers, we started with punch cards, then came the keyboard and mouse.
The original Palm had a rudimentary touch screen. I think multitouch is very
interesting, and today you see it on the iPhone. Now you can control
applications with several body parts. The iPhone features pinching and
expanding gestures to zoom, etc. Multitouch is a very natural way to
interact with a computer. Apple laptops now have multitouch right on the
trackpad as a standard feature. Multitouch adds a new dimension to the way
we do things, and it could open up a whole new world in the way we interact
ABOUT THE SPEAKER
Laura Nichols (email@example.com) is a Senior Technology Lead at Crispin
Porter & Bogusky, a Boulder-based advertising agency whose clients include
Microsoft, Volkswagen, Old Navy, Best Buy, and American Express. Laura
spends most of her time there working on the Microsoft team providing
leadership for the development team, programming, and new technology
research. She specializes in .NET development and has developed a passion
for multitouch technology. Previously, Laura was a Technical Lead at Texture
Media, a digital agency that prided itself on building brands online, where
she led the development team for clients such as Midas, Celestial
Seasonings, and National Cinemedia.
Natural User Interface Group (www.nuigroup.com) Perceptive Pixel
(www.perceptivepixel.com), Jeff Han's company 22MILES (www.22miles.com),
applications for sports Microsoft Surface (www.microsoft.com/surface)
TacTable (www.tactable.com) Instructables (www.instructables.com), info for
do-it-yourself Multi-Touch G2 (www.multi-touch-screen.net), software from PQ
WHAT IS MULTITOUCH?
Multitouch is a set of interaction techniques that allow you to control
graphical interfaces with your hands, using several fingers at once. It's a
system that recognizes more than one input at a time. Examples include the
iPhone, Blackberry Storm, HTC phone, Microsoft Surface, and the HP
TouchSmart computer. There's so much new stuff coming out now that it's hard
to keep up.
Companies that specialize in multitouch include Perceptive Pixel, which was
founded by Jeff Han, a pioneer of multitouch.
Another company, called Schematic, is an interactive digital agency. It made
a 12 x 5 foot multitouch wall that was displayed at the Cannes Film
Festival. The wall recognized people based on the RFID chips in their
nametags, which allowed the wall to give people very personalized
individualized information based on their registration information. This was
entirely developed in Flash.
The Multitouch makes the Cell. These are just boxes that can be scaled into
larger forms by putting the boxes together. It's an interconnecting set of
display cells, arranged in any size and shape. Nobody else is doing this.
Ralph Lauren used its store window to project its web site 24/7. People
could walk up to the window on the street and place orders. It was developed
in-house at Ralph Lauren. A projector puts the web site on the window, and a
touch-type film that was applied to the window senses the touches.
HISTORY OF MULTITOUCH
The iPhone made multitouch mainstream, but it really started in the late
1960's at the University of Toronto and at the University of Illinois. The
Plato IV computer-assisted education system was built in 1972--researchers
used it to study how touch computers could assist in education. Then in 1982
the Flexible Machine Interface at the University of Toronto became the first
multitouch interface. In 1983, Myron Krueger developed gestures and
demonstrated them using a light table. He was way ahead of his time. In 1984
Bell Labs created the first multitouch screen (Myron projected onto a light
table, so he wasn't touching a screen directly). In 1992 IBM and Bell South
made Simon, the first smart phone. In 1998 Fingerworks made some multitouch
tablets (and Apple eventually acquired it to develop the iPhone).
In 2005 Jeff Han came up with some absolutely amazing stuff. He projects an
image onto to a mirror, which bounces to a big screen, and a camera records
the image. He's working on pinching gestures, a virtual typewriter, and is
perfecting pressure sensitivity. You can zoom in and rotate maps, play
games, etc. He has an application that pulls in data from MRI and CT scans,
and then lets doctors drill into the brain and look at the image through
multitouch. Obviously this can be very practical.
Then came the iPhone and, in 2007, Microsoft Surface. It can identify
objects place on a table, like cameras and cell phones. So you can use it to
gather stuff onto a device: Put your phone on the surface, it automatically
syncs up and lets you exchange files.
Everyone is making multitouch flat panels, and they're getting competitive
and cheaper. There are also multitouch windows. For floor screens, you
project onto a mat and you can dance on it. Could be like the Wii Fit (a
pressure-sensitive platform you stand on to go skiing, etc) but with more
One manufacturer made a screen with real water that runs down its
surface--you touch the water and it refracts the sparks you create, which
makes for a neat demonstration.
How about World of Warcraft? Yup, you touch the screen to move your
character, or hit it to attack someone else's character! (You'd need a
really durable screen for that.)
At the Chicago airport, travelers can use multitouch on public displays to
get maps, weather, etc. that's pertinent to their trip.
REACHING THE MASS MARKET
So you want to develop an application? Ask yourself: Does it work with the
screen you want to use? Is the hardware durable enough to last the expected
lifetime? Is multitouch actually useful and appropriate for my application?
Don't use touch just for the sake of the technology. Start by focusing on
the experience instead.
Making a mobile product? It better be durable because users are going to be
dropping it, etc. And if you're doing desktop-level graphics, you'll need a
Don't use multitouch for writing: No one is going to write a novel that way.
Keep in mind that multitouch is for consuming and manipulating data, not
Why multitouch applications work: In 1999, the Museum of Modern Art created
the Un-Private House Table. It was a touch table with a hockey puck. It
recognized the data in the puck and let you look at floor plans specified in
Multitouch is a good medium for sharing: You can push data across the
multitouch table to someone else.
Multitouch is about exploring. Make exploring worth it so you get surprised
and things happen. Design it so you can change stuff and observe the
effects. People should be able to use it to figure out how things work.
Make it simple. It's new, so if it isn't simple, people won't adopt it. A
multitouch table might let you order drinks and play games, but that's it.
The Ibar is an entire bar that is a touch screen, possibly the largest
surface of its kind. It knows when your glass is empty and you need a
refill. A lot of this is just for fun.
A restaurant in Europe lets you order entirely by touch screen: You don't
see a waiter until the food arrives.
Keep it minimal: What you leave out is as important as what you put in.
Consider the Liberty Science Center: It made a cave painting wall where kids
could make handprints, fingerpaint, to simulate cave painting. Nothing more
Size matters: A giant screen isn't always best. The Detroit institute of Art
used a small one to display the Book of the Dead and let people flip through
it to study it--making it huge wouldn't have made sense for that purpose.
Have a clear goal, and make it practical. The Sprint Digital Lounge in the
Sprint store allowed visitors to compare phones on a touch screen without
having to walk all around the store trying to remember everything. You get
all the information in one place. So Sprint had a practical use and a clear
TOUCH TECHNOLOGY TYPES
Resistive: Uses two metallic layers such that pushing makes the layers
touch. It's an old technology that doesn't support multitouch well. For
example, you can't pinch with it. But it's good for single-touch
applications. Unfortunately the layers develop cracks and fail after a lot
Capacitive: This is glass coated with a transparent conductor. Human touch
affects the electric fields and changes capacitance. But it only works with
fingers, not objects. The iPhone uses it. It's cheap and easy to implement.
Projected: Embedded wires register where you touch, and it lets you use
objects like a stylus. It's not mainstream yet. But it's expensive and not
workable for large screens because of latency issues--you can't go much
above 42 inches.
Acoustic: Transducers listen to the sound of the touch. This works on on any
surface, so it very versatile. Works even when scratched.
Optical: A projector, a camera, infrared lights, and some acrylic. This is
good for large screens. Infrared light is used for detection: The light
passes through the acrylic panel from the edges and when you touch, you
create a dark spot. The camera senses infrared and sees the spot. The
software analyzing the camera image then registers a touch. Instead of
infrared inside the acrylic, can use lasers that skim along the top of the
surface, so you don't even have to quite touch the surface. Infrared-based
systems have trouble dealing with direct sunlight.
You'll find resistive, capacitive, and projected technologies in notebooks
Keyboard challenges: Keyboards don't work well on a touch screen, so people
are working on adding different types of feedback (which improve typing
accuracy). You can get visual, audio, or tactile feedback. So when you touch
it vibrates, for example, and people make fewer mistakes because you can
feel where you are.
Do-it-yourself is not that hard. You need acrylic and a camera that has had
its infrared filter removed and replaced with a visible filter. Use vellum
to eliminate glare. The acrylic must have polished edges so the infrared
light can get through--you can have it fire polished or sit around for an
eternity sanding it.
Tangible User Interface Object is a protocol for decoding detected gestures
from your camera.
The Natural User Interface Group is a great resource for this, and it's very
Advertising and games are great for getting the multitouch medium out there,
but real-world applications are the important thing. Remember this is for
data consumption, not creation. Hardware is key: It can fail and kill your
great application. And don't use it for technology's sake: Only adopt
multitouch if it's useful.
Now here is the future according to Microsoft: It sees multitouch in file
sharing, medical imaging, sports, ebooks, shopping, credit cards, in-store
navigation and product assistance, engineering, science, and just about
every aspect of everyday life (there's a Microsoft video demonstrating these
Q: Can I use a "pretend" finger on my iPhone when it's really cold out and I
don't want to take off my glove?
A: Nope, capacitive technology requires a real finger.
Comment from audience: I hear that Apple working on a glove, and there's a
third-party stylus out there.
Q: Is multitouch patented?
A: Apple is trying to patent it parts of it, but the nature of the idea is
such that the whole concept probably isn't patentable. People have patents
on the software of course, but no patents yet on multitouch itself.
Q: Android phones have some pivot hardware, but it's not working well enough
yet for general release. I sure hope Apple doesn't patent that and prevent
this stuff from coming out.
A: I don't think Apple can stop people from doing this stuff. A lot of
companies are out there developing devices and screens.
Q: What about open source software?
A: I like the stuff you can get through the Natural User Interface Group.
You can make it in .NET and WPF (Windows Presentation Foundation). The code
is developed in C++ with a WPF API.
Q: Does tracking more fingers make things difficult?
A: No, performance does not degrade, even with dozens of fingers.
Q: What about adaptive technologies for people who don't have fingers?
A: I haven't seen much of that yet. Obviously, multitouch is not yet
accessible to amputees and blind people. I think there will be way to
support these groups as multitouch becomes more mainstream. There would have
to be special prosthetics that can interact with the technology.
Q: How much computer power do you need?
A: Lots. You need a lot of video memory, and a dual CPU helps. The Community
Core Vision software is a hog.
Q: What's the lowest-price platform out there to develop for?
A: Well there's the HP TouchSmart computer which goes for $1500. That's not
cheap. Then again, my first implementation that I built in the garage was
essentially a cardboard box with a web cam taped to it--that didn't really
Q: So what are you doing with your current garage project?
A: I'm developing applications in my spare time and using the interface for
prototyping. My clients want to see a prototype to demonstrate ideas before
making much of an investment. And it's versatile: It can run on Mac,
Windows, or Linux.
Q: What kind of camera is best?
A: The best deal is the PlayStation 3 Eye camera (you want at least 60
frames per second, and that's the more affordable camera that can do that).
A web cam will work too, but you have to rip out the infrared filter and
replace it with a visible light filter. Some cameras have the infrared
filter painted onto the lens, so make sure you do your research.
RMIUG, http://www.rmiug.org appreciates the sponsorship of MicroStaff,
http://www.microstaff.com , ONEWARE, http://www.ONEWARE.com and Copy Diva,
To unsubscribe from this list, send an email to
firstname.lastname@example.org or visit:
Yahoo! Groups Links
<*> To visit your group on the web, go to:
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
(Yahoo! ID required)
<*> To change settings via email:
<*> To unsubscribe from this group, send an email to:
<*> Your use of Yahoo! Groups is subject to: