R. Gal, Y. Wexler, H. Hoppe, E. Ofek & D. Cohen-Or

Seamless Montage for Texturing Models

Accepted to EuroGraphics 2010

N. Villar, S. Izadi, H. Benko, J. Helmes, D. Rosenfeld, E. Ofek, J. Westhues, A. Butler, X. Cao, B. Chen &  Steve Hodges

Mouse 2.0: Multi-Touch Meets the Mouse

UIST 2009, Victoria, Canada

Best paper award

In this paper we explore the possibilities for augmenting the standard computer mouse with multi-touch capabilities so that it can sense the position of the user’s fingers and thereby complement traditional pointer-based desktop interactions with touch and gestures. We present five different multi-touch mouse implementations, each of which explores a different touch sensing strategy, which leads to differing form-factors and hence interaction possibilities. In addition to the detailed description of hardware and software implementations of our prototypes, we discuss the relative strengths, limitations and affordances of these different input devices as informed by the results of a preliminary user study.

B. Chen, B. Neubert, E. Ofek, O. Deussen & M. F. Cohen

Integrated Videos and Maps for Driving Directions

UIST 2009, Victoria, Canada

While onboard navigation systems are gaining in importance, maps are still the medium of choice for laying out a route to a destination and for way finding. However, even with a map, one is almost always more comfortable navigating a route the second time due to the visual memory of the route. To

make the first time navigating a route feel more familiar, we present a system that integrates a map with a video automatically constructed from panoramic imagery captured at close intervals along the route. The routing information is used to create a variable speed video depicting the route. During playback of the video, the frame and field of view are dynamically modulated to highlight salient features along the route and connect them back to the map. A user interface is demonstrated to allow exploration of the combined map, video, and textual driving directions. We discuss the construction of the hybrid map and video interface. Finally, we report the results of a study that provides evidence of the effectiveness of such a system for route following.

T. Yamaguchi, Wilburn , Z. Cao & E. Ofek
Video-based modeling of dynamic hair

Pacific-Rim Symposium on Image & Video Technology 2009, Tokyo, Japan

P. Mishra, E. Ofek & G. Kimchi
Validation of Vector Data using Oblique Images
ACM SIGGIS 2008, CA USA

Oblique images are aerial photographs taken at oblique angles to the earth’s surface. Projections of vector and other geospatial data in these images depend on camera parameters, positions of the entities, surface terrain, and visibility. This paper presents a robust and scalable algorithm to detect inconsistencies in vector data using oblique images. The algorithm uses image descriptors to encode the local appearance of a geospatial entity in images. These image descriptors combine color, pixel-intensity gradients, texture, and steerable filter responses. A Support Vector Machine classifier is trained to detect image descriptors that are not consistent with underlying vector data, digital elevation maps, building models, and camera parameters. In this paper, we train the classifier on visible road segments and non-road data. Thereafter, the trained classifier detects inconsistencies in vectors, which include both occluded and misaligned road segments. The consistent road segments validate our vector, DEM, and 3-D model data for those areas while inconsistent segments point out errors. We further show that a search for descriptors that are consistent with visible road segments in the neighborhood of a misaligned road yields the desired road alignment that is consistent with pixels in the image.

J. Xiao, T. Fang. P. Tan, Z. Peng, E. Ofek & L. Quan
Image Based Facade Modeling
ACM SIGGRAPH Asia 2008, Singapure

We propose in this paper a semi-automatic image-based approach that uses images captured along the streets, and relies on structure from motion to automatically recover the camera positions and point clouds as the initial stage for the modeling. We start a building facade as a flat rectangular plane or a developable surface, and the texture image of the flat facade is composited from the multiple visible images with handling of occluding objects. A facade is then decomposed and structured into a Directed Acyclic Graph of rectilinear elementary patches. The decomposition is carried out top-down by a recursive subdivision, and followed by a bottom-up merging with the detection of the architectural bilateral symmetry and repetitive patterns. Each subdivided patch of the flat facade is augmented with the depth that is optimized from the 3D points. Our system also allows the user to easily provide feedbacks in the 2D image space for the proposed decomposition and augmentation. Finally, our approach is demonstrated on a large number of facades from a variety of street-side images.

H. Wang, H. Hoppe, Y. Wexler and E. Ofek

Factoring Repeated Content Within and Among Images (11M)

ACM SIGGRAPH 2008, Los Angeles USA

We reduce transmission bandwidth and memory space for images by factoring their repeated content. A transform map and a condensed epitome are created such that all image blocks can be reconstructed from transformed epitome patches. The transforms may include affine deformation and color scaling to account for perspective and tonal variations across the image. The factored representation allows efficient random-access through a simple indirection, and can therefore be used for real-time texture mapping without expansion in memory. Our scheme is orthogonal to traditional image compression, in the sense that the epitome is amenable to further compression such as DXT. Moreover it allows a new mode of progressivity, whereby generic features appear before unique detail. Factoring is also effective across a collection of images, particularly in the context of image-based rendering. Eliminating redundant content lets us include textures that are several times as large in the same memory space.

B. Epshtein, E. Ofek, Y. Wexler and P. Zhang

Hierarchical photo organization using geo-relevance

ACM GIS 2007, Seattle WA, USA

We present a novel framework for organizing large collections of images in a hierarchical way, based on scene semantics. Rather than score images directly, we use them to score the scene in order to identify typical views and important locations which we term Geo-Relevance. This is done by relating each image with its viewing frustum which can be readily computed for huge collections of images nowadays. The frustum contains much more information than only camera position that has been used so far. For example, it distinguishes between a photo of the Eiffel Tower and a photo of a garbage bin taken from the exact same place. The proposed framework enables a summarized display of the information and facilitates efficient browsing.

N.Li, N. Moraveji, H. Kimura and E. Ofek

Improving the Experience of Controlling Avatars in

Camera-Based Games Using Physical Input

The 14^th ACM International Conference on Multimedia, Santa Barbara, CA, USA 2006

This paper investigates two methods of improving the user experience of camera-based interaction. First, problems that arise when avatars are designed to mimic a user’s physical actions are presented. Second, a solution is proposed: adding a layer of separation between user and avatar while retaining intuitive user control. Two methods are proposed for this separation: spatially and temporally. Implementations of these methods are then presented in the context of a simple game and evaluate their effect on performance and satisfaction. Results of a human subject experiment are presented, showing that reducing the amount of user control can maintain, and even improve, user satisfaction if the design of such a reduction is appropriate. This is followed by a discussion of how the findings inform camera-based game design.

H. Jiang, E. Ofek, N. Moraveji and S. Yuanchun

Direct Pointer: Direct Manipulation for Large Display Interaction using Handheld Cameras

SIG CHI  2006, Montreal, Canada

This paper describes the design and evaluation of a technique, Direct Pointer, that enables users to interact intuitively with large displays using cameras equipped on handheld devices, such as mobile phones and personal digital assistant (PDA). In contrast to many existing interaction methods that attempt to address the same problem, ours offers direct manipulation of the pointer position with continuous visual feedback. The primary advantage of this technique is that it only requires equipment that is readily available: an electronic display, a handheld digital camera, and a connection between the two. No special visual markers in the display content are needed, nor are fixed cameras pointing at the display. We evaluated the performance of Direct Pointer as an interaction product, showing that it performs as well as comparable techniques that require more sophisticated equipment.

Y. Matsushita, E. Ofek, X. Tang and H. Shum

Full Frame Video Stabilization with Motion Inpainting

IEEE PAMI 2006

Video stabilization is an important video enhancement technology which aims at removing annoying shaky motion from videos. We propose a practical and robust approach of video stabilization that produces full-frame stabilized videos with good visual quality. While most previous methods end up with producing low resolution stabilized videos, our completion method can produce full-frame videos by naturally filling in missing image parts by locally aligning image data of neighboring frames. To achieve this, motion inpainting is proposed to enforce spatial and temporal consistency of the completion in both static and dynamic image areas. In addition, image quality in the stabilized video is enhanced with a new practical deblurring algorithm. Instead of estimating point spread functions, our method transfers and interpolates sharper image pixels of neighboring frames to increase the sharpness of the frame. The proposed video completion and deblurring methods enabled us to develop a complete video stabilizer which can naturally keep the original image quality in the stabilized videos. The effectiveness of our method is confirmed by extensive experiments over a wide variety of videos.

Y. Wei, E. Ofek, L. Quan and H. Shum

Modeling Hair from Multi Views

SIGGRAPH  2005, Los Angeles, CA.

Y. Matsushita, E. Ofek, X. Tang and H. Shum

Video Completion with Motion Inpainting for Video Stabilization

CVPR  2005, San Diego, CA.

Video stabilization is an important video enhancement technology which aims at removing annoying shaky motion from videos. We propose a practical and robust approach of video stabilization that produces full-frame stabilized videos with good visual quality. While most previous methods end up with producing low resolution stabilized videos, our completion method can produce full-frame videos by naturally filling in missing image parts by locally aligning image data of neighboring frames. To achieve this, motion inpainting is proposed to enforce spatial and temporal consistency of the completion in both static and dynamic image areas. In addition, image quality in the stabilized video is enhanced with a new practical deblurring algorithm. Instead of estimating point spread functions, our method transfers and interpolates sharper image pixels of neighboring frames to increase the sharpness of the frame. The proposed video completion and deblurring methods enabled us to develop a complete video stabilizer which can naturally keep the original image quality in the stabilized videos. The effectiveness of our method is confirmed by extensive experiments over a wide variety of videos.

B. Chen, E. Ofek, H. Shum and M. Levoy

Interactive Deformation of Light Fields

SIGGRAPH I3D  2005, Washington D.C.

X. Cao, E. Ofek, and D. Vronay

Evaluation of Alterantive Presentation Control Techniques. (PDF, 420K)

SIG CHI 2005, Protland, OR.

This paper describes the design and evaluation of a technique, Direct Pointer, that enables users to interact intuitively with large displays using cameras equipped on handheld devices, such as mobile phones and personal digital assistant (PDA). In contrast to many existing interaction methods that attempt to address the same problem, ours offers direct manipulation of the pointer position with continuous visual feedback. The primary advantage of this technique is that it only requires equipments that are readily available: a large electronic display, a handheld camera, and a connection between the two. No special visual markers in the display content are needed, nor are fixed cameras pointing at the display. We evaluated the performance of Direct Pointer as an interaction product, showing that it performs as well as comparable techniques that require more sophisticated equipment.

R. Gvili, A. Kaplan, E. Ofek, G. Yahav

Depth Key 

Stereoscopic Displays and Applications: The Engineering Reality of Virtual Reality 2003  (Proceedings of SPIE/IS&T Volume 5006), San Jose, CA.

A. Redert, M. Op de Beeck, C. Fehn, W. IJsselsteijn, M. Pollefeys, L. J. Van Gool, E. Ofek, I. Sexton and P. Surman

ATTEST: Advanced Three-dimensional Television System Technologies.

3DPVT 2002, Padova, Italy.

E. Ofek and A. Rappoport

Interactive reflections on curved objects

ACM SIGGRAPH 1998

Global view-dependent illumination phenomena, in particular reflections,
greatly enhance the realism of computer-generated imagery.
Current interactive rendering methods do not provide satisfactory
support for reflections on curved objects.
In this paper we present a novel method for interactive computation
of reflections on curved objects. We transform potentially reflected
scene objects according to reflectors, to generate virtual objects.
These are rendered by the graphics system as ordinary objects, creating
a reflection image that is blended with the primary image. Virtual
objects are created by tessellating scene objects and computing
a virtual vertex for each resulting scene vertex. Virtual vertices are
computed using a novel space subdivision, the reflection subdivision.
For general polygonal mesh reflectors, we present an associated
approximate acceleration scheme, the explosion map. For specific
types of objects (e.g., linear extrusions of planar curves) the
reflection subdivision can be reduced to a 2-D one that is utilized
more accurately and efficiently.

E. Ofek, E. Shilat, A. Rappoport, and M. Werman.

Multi-resolution Textures from Image Sequences (PDF, 515K)

IEEE Computer Graphics and Applications 1997