Hrvoje Benko
Hrvoje Benko Microsoft Research
Researcher About Me
Natural Interaction Research - Microsoft Research Research
Bldg. 99, One Microsoft Way, Redmond WA 98052-6399, USA (map) Publications
+1-425-707-2731 Press
benko (at) microsoft . com Professional Activities
How to say my full name? Pronunciation of Hrvoje Benkowav (But I go with "Benko" for simplicity!)  
Short bio: biographyhtml    
Headshot:jpg (low-res 0.2MB)  jpg (high-res 4.7MB)  
About me

I am a researcher in Natural Interaction Research group at Microsoft Research. My research is on novel surface computing technologies and their impact on human-computer interaction. Prior to working at Microsoft, I received my PhD at Columbia University, working on augmented reality projects that combine immersive experiences with interactive tabletops.

My projects explore the space where the flat digital world of surface computing meets the curved, physical, 3D space we live in. My current research interests include:

  • new input form factors
  • multi-touch and freehand gestural input
  • augmented reality

In 2010, I've worked with a group at Microsoft Hardware to turn one of my research projects into a product. We created Microsoft Touch Mouse which enables the user to use touch gestures on top of the mouse to control their Windows experience (blog post) (product page) (press coverage). Check out my YouTube promotional video (link). Here are some sweet product shots:

Microsoft Touch Mouse Microsoft Touch Mouse Microsoft Touch Mouse

Research Highlights
 
Spatial Augmented Reality
I am fascinated with the possibility of using cameras and projectors to embue the space with interactivity. Such projector+camera combinations can enable both data visualization and interactivity without requiring the user to wear any additional gear. I've explored these concepts with collaborators in several projects.

Check out MirageTable, LightSpace, Pinch-the-Sky Dome, DepthTouch projects. 
 
Get Microsoft Silverlight Get Microsoft Silverlight
MirageBlocks (2011) LightSpace (2010)
   
Multitouch Input Devices
Current computer input devices are well optimized input solutions, but they often offer very limited bandwith for interacting with computers. For example, very few devices really take advantage of the full dexterity of the user's hands. My projects have investigated extending the interaction vocabulary of several standard input devices (e.g., mice, pens, and spartphone touchscreens) through multi-touch sensing embedded on the surface of the device. Not only are such multi-touch devices capable of both standard operation and multi-touch gestures, but they can also implicitly sense a lot of contextual information about their use (e.g., whether the user is right or left handed). My collaborators and I, took the idea of a multitouch mouse from a research concept and made it into a product (Microsoft Touch Mouse).

Check out MultiTouch Pen, Mouse 2.0, Interaction Models for MultiTouch Mouse, EMG+Surface projects.    
 
Get Microsoft Silverlight Get Microsoft Silverlight
Microsoft Touch Mouse (2011) Mouse 2.0 (2009)
   
Touch and Gestures on New Form Factors
Most of our computing devices in use today are flat and rectangular and the data we present on them seems to follow a similar trend. However, the world we live in is full of objects which have non-flat shape. I have been designing multitouch interactive devices with high curvature (e.g., spheres, cylinders, and domes) and exploring interaction capabilities of such non-flat, curved and potentially deformable and bendable displays. There are many inherent obstacles in enabling both display and sensing capabilities on highly curved displays, but such devices often offer many unique valuable opportunities.

Check out Sphere, Pinch-the-Sky Dome, Mouse 2.0, and MultiTouch Pen projects.
 
Get Microsoft Silverlight Get Microsoft Silverlight
Pinch the Sky Dome (2009) Sphere (2008)
 
Books
2011 Virtualna Okruzenja Pandžić, I. S., Pejša, T., Matković, K., Benko, H., Čereković, A., Matijašević, M. (2011). "Virtualna Okruženja: Interaktivna 3D grafika i njene primjene". (In Croatian, translated title: Virtual Environments: Interactive 3D graphics and applications). ISBN: 978-953-197-606-0. Element d.o.o., Zagreb, September 2010.   
       
       
Peer Reviewed Conference Papers and Technotes
       
2012 MirageTable Benko, H., Jota, R. and Wilson, A. D. (2012). "MirageTable: Freehand Interaction on a Projected Augmented Reality Tabletop". In Proc. of ACM CHI 2012. pdf (0.8 MB)
wmv (4MB)
wmv (28MB)
     
LightGuide Sodhi, R., Benko, H., and Wilson, A. D. (2012). "LightGuide: Projected Visualizations for Hand Movement Guidance". In Proc. of ACM CHI 2012. (best paper honorable mention) pdf (0.5 MB)
wmv (27MB)
2011 MultiTouch Pen + Accelerometer Sun, M., Cao, X., Song, H., Izadi, S., Benko, H.,  Guimbretiere, F., Ren, X., and Hinckley, K. (2011). "Enhancing Naturalness of Pen-and-Tablet Drawing through Context Sensing". In Proceedings of ACM Interactive Tabletops and Surfaces (ITS '11). p. 83-86. pdf (0.5 MB)
wmv (32MB)
OmniTouch Harrison, C., Benko, H., Wilson, A. D. (2011). "OmniTouch: Wearable Multitouch Interaction Everywhere". In Proceedings of ACM UIST 2011. p. 441-450. pdf (3.9MB)
wmv (35MB)
     
PocketTouch Saponas, T.S., Harrison, C., Benko, H. (2011) "PocketTouch: Through-Fabric Capacitive Touch Input". In Proceedings of ACM UIST 2011. p. 303-308. pdf (0.8MB)
wmv (40MB)
     
Multi-Touch Pen Song, H., Benko, H., Guimbretiere, F., Izadi, S., Cao, X., and Hinckley, K. (2011). "Grips and Gestures on a Multi-Touch Pen". In Proceedings of ACM CHI 2011. p. 1323-1332. pdf (0.5MB)
wmv (13MB)
     
Rock and Rails Wigdor, D., Benko, H., Pella, J., Lombardo, J., and Williams, S. (2011). "Rock & Rails: Extending Multi-touch Interactions with Shape Gestures to Enable Precise Spatial Manipulations". In Proceedings of ACM CHI 2011. p. 1581-1590. pdf (1.2MB)
wmv (56MB)
       
2010 Pinch-the-Sky Dome Benko, H. and Wilson, A. D. (2010). "Multi-Point Interactions with Immersive Omnidirectional Visualizations in a Dome". In Proceedings of ACM Interactive Tabletops and Surfaces (ITS '10). p. 19-28. pdf (1MB)
wmv (32MB)
     
Pen + Touch = New Tools Hinckley, K., Yatani, K., Pahud, M., Coddington, N., Rodenhouse, J., Wilson, A., Benko, H., and Buxton, B. (2010). "Pen + Touch = New Tools". In Proceedings of ACM UIST '10. pp. 27-36. pdf (1.2MB)
wmv (119MB)
LightSpace Wilson, A. and Benko, H. (2010). "Combining Multiple Depth Cameras and Projectors for Interactions On, Above, and Between Surfaces". In Proceedings of ACM UIST '10. pp. 273-282. pdf (0.8MB)
wmv (39MB)
     
MT Mouse Models Benko, H., Izadi, S., Wilson, A. D., Cao, X., Rosenfeld, D., and Hinckley, K. (2010). "Design and Evaluation of Interaction Models for Multi-touch Mice". In Proceedings of Graphics Interface 2010. p. 253-260. pdf (1.4MB)
wmv (27MB)
Manual Dexterity Hinckley, K., Yatani, K., Pahud, M., Coddington, N., Rodenhouse, J., Wilson, A., Benko, H., and Buxton, B. (2010). "Manual Deskterity: An Exploration of Simultaneous Pen + Touch Direct Input". ACM CHI 2010 Extended Abstracts (alt.chi). pdf (0.5MB)
wmv (24MB)
     
Pictionaire Hartmann, B., Morris, M., Benko, H., and Wilson, A. D.  (2010). "Pictionaire: Supporting Collaborative Design Work by Integrating Physical and Digital Artifacts". In Proceedings of ACM CSCW '10. p. 421-424. pdf (4MB)
mov (30MB)
Three's Company Tang, A., Pahud, M., Inkpen, K., Benko, H., Tang, J., and Buxton, B. (2010). "Three's Company: Understanding Communication Channels in Three-way Distributed Collaboration". In Proceedings of ACM CSCW '10. p. 271-280. pdf (0.8MB)
       
2009 EMG + Surface Benko, H. Saponas, T.S., Morris, D., and Tan, D. (2009). "Enhancing Input On and Above the Interactive Surface with Muscle Sensing". In Proceedings of ACM Interactive Tabletops and Surfaces (ITS '09). p. 93-100. pdf (0.7MB)
wmv (29MB)
pptx (48MB)
     
ShadowGuides Freeman, D., Benko, H., Morris, M. R., and Wigdor, D. (2009). "ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures". In Proceedings of ACM Interactive Tabletops and Surfaces (ITS '09). p. 165-172. pdf (0.6MB)
wmv (45MB)
     
Dome Benko, H. (2009). "Beyond Flat Surface Computing: Challenges of Depth-Aware and Curved Interfaces". In Proceedings of ACM MultiMedia '09. pdf (0.5MB)
pptx (176MB)  
Mouse 2.0 Villar, N., Izadi, S., Rosenfeld, D., Benko, H., Helmes, J., Westhues, J., Hodges, S.,  Butler, A., Ofek, E., Cao, X., and Chen, B. (2009). "Mouse 2.0: Multi-touch Meets the Mouse". In Proceedings of ACM User Interface Software and Technology (UIST '09). p. 33-42. (best paper award) pdf (0.8MB)
wmv (45MB)
Mice and Keyboards on the tabletop Hartmann, B., Morris, M. R., Benko, H., and Wilson, A. D. (2009). "Augmenting Interactive Tables with Mice & Keyboards". In Proceedings of ACM User Interface Software and Technology (UIST '09). p. 149-152. pdf (1.2MB)
wmv (51MB)
     
Ripples Wigdor, D., Williams, S., Cronin, M., White, K., Levy, R., Mazeev, M., and  Benko, H. (2009). "Ripples: Utilizing Per-Contact Visualizations to Improve User Interaction with Touch Displays". In Proceedings of ACM User Interface Software and Technology (UIST '09). p. 3-12. pdf (1.2MB)
wmv (42MB)
     
Separability of Manipulations Nacenta, M., Baudisch, P, Benko, H., and Wilson, A. D. (2009). "Separability of Spatial Manipulations in Multi-touch Interfaces". In Proceedings of Graphics Interface '09. Kelowna, BC, Canada, May 2009. p. 175-182. pdf (0.7MB)
wmv (26MB)
       
2008 Sphere Benko, H., Wilson, A. D., and Balakrishnan, R. (2008). "Sphere: Multi-Touch Interactions on a Spherical Display". In Proceedings of ACM User Interface Software and Technology (UIST '08). Monterey, CA. p. 77-86. pdf (1MB)
wmv (30MB)
pptx (9MB)
   
2007 Pointer Warping Benko, H. and Feiner, S. (2007). "Pointer Warping in Heterogeneous Multi-Monitor Environments". In Proceedings of Graphics Interface '07. Montreal, Canada. May 28-30. p. 111-117. pdf (5.5MB)
     
Balloon Selection Benko, H. and Feiner, S. (2007). "Balloon Selection: A Multi-Finger Technique for Accurate Low-Fatigue 3D Selections." In Proceedings of IEEE Symposium on 3D User Interfaces. Charlotte, North Carolina, USA. March 10-11. p. 79-86. pdf (0.8MB)
mov (31MB)
       
2006 Precise dual finger selection techniques Benko, H., Wilson, A. D., and Baudisch, P. (2006). "Precise Selection Techniques for Multi-Touch Screens." In Proceedings of ACM CHI 2006 (CHI'06: Human Factors in Computing Systems). Montreal, Canada. April 24-27. p. 1263-1272. pdf (1MB)
mov (21MB)
       
2005 Multi-monitor mouse Benko, H., Feiner, S. (2005). "Multi-Monitor Mouse." ACM CHI 2005 (CHI'05: Human Factors in Computing Systems) Extended Abstracts. Portland, Oregon, USA. April 2-7. p. 1208-1211. pdf (1.4MB)
mov (7MB)
     
Cross-Dimensional Gestures Benko, H., Ishak, E.W., Feiner, S. (2005). "Cross-Dimensional Gestural Interaction Techniques for Hybrid Immersive Environments." In Proceedings of IEEE Virtual Reality (VR '05). Bonn, Germany. March 10-12. p. 209-116. pdf (2.5MB)
mov (32MB)
       
2004 VITA Benko, H., Ishak, E.W., Feiner, S. (2004). "Collaborative Mixed Reality Visualization of an Archaeological Excavation." In Proceedings of  The International Symposium on Mixed and Augmented Reality (ISMAR '04). November 2-5. p. 132-140. pdf (1.3MB)
mov (38MB)
Allen, P., Feiner, S. Troccoli, A., Benko, H., Ishak, E., Smith, B. (2004). "Seeing into the Past: Creating a 3D Modeling Pipeline for Archaeological Visualization." In Proceedings of  International Symposium on 3D Data Processing Visualization and Transmission (3DPVT '04). p. 751-758. pdf (1.2MB)
       
2003 MAVEN Kaiser, E., Olwal, A., McGee, D., Benko, H., Corradini, A., Li, X., Cohen, P., Feiner, S. (2003). "Mutual Dissambiguation of 3D Multimodal Interaction in Augmented and Virtual Reality." In Proceedings of The Fifth International Conference on Multimodal Interfaces (ICMI '03). Vancouver, BC. Canada. November 5-7. p. 12-19. pdf (2MB)
divx (21MB)
     
SenseShapes Olwal, A., Benko, H., Feiner, S. (2003). "SenseShapes: Using Statistical Geometry for Object Selection in a Multimodal Augmented Reality System". In Proceedings of  The Second International Symposium on Mixed and Augmented Reality (ISMAR '03). Tokyo, Japan. October 7-10. p. 300-301. pdf (0.3MB)
divx (32MB)
jpg (1MB)
       
Book Chapters
2010 The Tale of Touch Input Benko, H. and Wigdor, D. Imprecision, Inaccuracy, and Frustration: The Tale of Touch Input. In "Tabletops - Horizontal Interactive Displays". Editor: Christian Müller-Tomfelde. Springer HCI Series, Springer-Verlag London Ltd., April 2010.  pdf link
       
Journal Articles
       
2004 Mirage Thorisson, K., Benko, H., Abramov, D., Arnold, A., Maskey, S., Vaseekaran, A. (2004). "Constructionist Design Methodology for Interactive Intelligences". AI Magazine. Vol. 25, No. 4. p. 77-90. pdf (1MB)
mov (26MB)
       
2001 eIreland Benko, H. (2001). "e-Ireland: Europe's New Internet Hub". Perspectives on Business and Economics, Volume 19. Martindale Center for the Study of Private Enterprise, Lehigh University, Bethlehem, PA, USA. p. 39-48. pdf (0.3MB)
       
Technical Reports
       
2009 DepthTouch Benko, H., Wilson, A. (2009). "DepthTouch: Using Depth-Sensing Camera to Enable Freehand Interactions On and Above the Interactive Surface". Microsoft Research Technical Report MSR-TR-2009-23. March, 2009. pdf (0.3MB)
wmv (30MB)
     
Benko, H., Morris, M. R., Brush, A.J.B., Wilson, A.D. (2009). "Insights on Interactive Tabletops: A Survey of Researchers and Developers". Microsoft Research Technical Report MSR-TR-2009-22. March, 2009. pdf (0.3MB)
 
2004 Cross-Dimensional Gestures Benko, H., Ishak, E.W., Feiner, S. (2004). "Cross-Dimensional Gestural Interaction Techniques for Hybrid Immersive Environments". Columbia University, New York. Technical Report. CUCS-029-04, August 2004. pdf (2.7MB)
       
       
Demos/Posters
       
2011 SketchSpace Holman, D. and Benko, H. (2011). "SketchSpace: Designing Interactive
Behaviors with Passive Materials". CHI 2011 Extended Abstracts, May, 2011. 		pdf (0.7MB)
wmv (14.7MB)
StereoBlocks Jota, R. and Benko, H. (2011). "Constructing Virtual 3D Models with Physical Building Blocks". CHI 2011 Extended Abstracts, May, 2011. pdf (0.3MB)
wmv (22MB)
       
2010 Pinch-the-Sky Dome Benko, H., Wilson, A. (2010). "Pinch-the-Sky Dome: Freehand Multi-Point Interactions with Immersive Omni-Directional Data". CHI 2010 Extended Abstracts, April, 2010. pdf (0.3MB)
wmv (60MB)
       
2008 DepthTouch Benko, H., Wilson, A. (2008). "DepthTouch: Using Depth-Sensing Camera to Enable Freehand Interactions On and Above the Interactive Surface". IEEE Workshop on Tabletops and Interactive Surfaces '08. Amsterdam, the Netherlands, October 1-3, 2008. pdf (1.2MB)
wmv (30MB)
       
2004 VITA Benko, H., Ishak, E.W., Feiner, S. (2004). "VITA: Visual Interaction Tool for Archaeology". The International Symposium on Mixed and Augmented Reality (ISMAR 2004). November 2-5, 2004. pdf (0.2MB)
     
VITA Benko, H., Ishak, E.W., Feiner, S. (2004). "VITA: Visual Interaction Tool for Archaeology (Demo)". In Proc. The ACM SIGMM Effective Telepresence Workshop (ACM SIGMM ETP 2004), October 15, 2004. p. 48-49. pdf (0.2MB)
     
Monte Polizzo Allen, P., Feiner, S., Meskell, L., Ross, K., Troccoli, A., Smith, B., Benko, H., Ishak, E., and Conlon, J. (2004). "Digitally Modeling, Visualizing and Preserving Archaeological Sites". In Proc. Joint Conference on Digital Libraries 2004 (JCDL 2004). Tuscon, AZ. June 7-11. pp. 389. pdf (0.3MB)
   
2002 Feiner, S. Bell, B., Benko, H., Blasko, G., Güven, S., Hallaway, D., Höllerer, T., and Lok, S. (2002). "Mobile Augmented Reality Systems." Living with the Genie: Governing Scientific and Technological Transformation in the 21st Century. New York, NY. March 5-7.  
       
       
Workshop Papers
       
2011 Steinicke, F., Benko, H., Daiber, F., Keefe, D.F., de la Rivière, J.-B. (2011) "Touching the 3rd dimension (T3D)". (workshop). CHI Extended Abstracts 2011. p. 161-164. pdf (1.4MB)
       
2009 Sphere Benko, H. and Wilson, A. (2009). "Design Challenges of Interactive Spherical User Interfaces". Programming Reality Workshop at CHI'09. April, 2009. pdf (0.2MB)
       
2005 Cross-Dimensional Gestures Ishak, E., Benko, H., and Feiner, S. (2005). "Development and Evaluation of Mixed Reality Interaction Techniques". Workshop on 3D User Interfaces (3DUI at IEEE VR'05). Bonn, Germany. March 10-12, 2005. pdf (0.3MB)
       
2003 VITA Benko, H., Ishak, E., and Feiner, S. (2003). "Collaborative Visualization of an Archaeological Excavation". Workshop on Collaborative Virtual Reality and Visualization (CVRV 2003). Lake Tahoe, CA. October 26-28, 2003. pdf (1.2MB)
       
       
Ph.D. Thesis
       
2007 PhD Dissertation Benko, H. (2007). "User Interaction in Hybrid Multi-Display Environments". Ph.D. Dissertation. Department of Computer Science, Columbia University, New York, NY. May 2007. pdf (12.4MB)
       
       
Invited Talks (incomplete list)
2011 Benko, H. (2011). "High-fidelity Augmented Reality Interactions". Microsoft Research Latin American Faculty Summit. May 19, 2011. pptx (189MB)
     
Benko, H. (2011). "Multi-touch Interactions on Small Input Devices". DUB Research Talk. University of Washington. Februady 2, 2011. pptx (70MB)
       
2009 Benko, H. (2009). "Alternative Form Factors of Surface Computing". Microsoft Research Faculty Summit. July 15, 2009. pptx (6MB)
     
Wilson, A. D. and Benko, H. (2009). "Riffing on Surface". Department of Computer Science, University of California Santa Barbara. March 6, 2009. pptx (6MB)
       
2008 Benko, H. (2008). "Non-Flat Surface Computing". Microsoft Research Cambridge Lab. September 15, 2008. pptx (19MB)
       
2005 VITA Benko, H. (2005). "Collaborative Mixed Reality Visualization of an Archaeological Excavation". MIT Media Lab. January 12, 2005. ppt (13MB)
       
2004 VITA Benko, H., Troccoli, A. (2004). "Creating Telepresence: 3D Modeling and Visualization of an Archaeological Excavation". Ph.D. Seminar, Computer Science, Columbia University. November 23, 2004. ppt (36MB)