Microsoft continues to innovate in the NUI space, continuously providing new modalities for interacting with our computing systems. We are leveraging our expertise in NUI and developing a radically new way for people to create information visualizations and interact with their data through simple interactive sketches. We aim to make manipulating and analyzing data as easy as creating napkin sketches, and much more powerful.
We assert that removing some of the interaction friction present in traditional desktop interfaces enables users to more effectively explore, understand, and communicate data. We choose sketch as the grounding modality because we believe that sketching itself promotes thinking, insight, and inspiration. Such a system will enable levels of interactive analysis not previously possible, individually or collaboratively, which is critical as our data sources continue to get bigger and more complex.
Even though communication of insights is one of the main goals of data exploration,
presenting visual information to other people is underexplored.
To create a new and engaging form of storytelling with data,
we explore how to leverage and extend the narrative storytelling attributes of whiteboard animation with interactive information visualization techniques.
SketchStory is an interactive whiteboard system for storytelling with data through real-time sketching.
It facilitates the creation of personalized and expressive data charts quickly and easily,
where the presenter sketches example icon and SketchStory automatically completes the chart by synthesizing from example sketches.
Furthermore, SketchInsight allows the presenter to interact with the data charts.
SketchStory Video for InfoVis 2013 [.wmv (43MB)]
Current interfaces for common information visualizations such as bar graphs, line graphs, and scatterplots usually make
use of the WIMP (Windows, Icons, Menus and a Pointer) interface paradigm with its frequently discussed problems of multiple
levels of indirection via cascading menus, dialog boxes, and control panels. Recent advances in interface capabilities such as the
availability of pen and touch interaction challenge us to re-think this and investigate more direct access to both the visualizations
and the data they portray. We conducted a Wizard of Oz study to explore applying pen and touch interaction to the creation of
information visualization interfaces on interactive whiteboards without implementing a plethora of recognizers. Our wizard acted as
a robust and flexible pen and touch recognizer, giving participants maximum freedom in how they interacted with the system. Based
on our qualitative analysis of the interactions our participants used, we found several insights about pen and touch interactions in the
context of learnability and the interplay between pen and touch gestures.
When faced with the task of understanding complex data, it
is common for people to work on whiteboards, where they
can collaborate with others, brainstorm lists of important
questions, and sketch simple visualizations. However, these
sketched visualizations seldom contain real data. We address
this gap by extending these sketched whiteboard visualizations
with the actual data to be analyzed. Guided by
an iterative design process, we developed a better understanding
of the challenges involved in bringing sketchbased
interaction to data analysis. In this work we contribute
insights into the design challenges of sketch-based
charting, and we present SketchVis, a system that leverages
hand-drawn input for exploring data through simple charts.
SketchVis Video for ITS 2011 [.wmv (1.9MB)]
SketchStory: Telling More Engaging Stories with Data through Freeform Sketching
IEEE TCVG (InfoVis 2013), Vol. 19, No. 2, pp. 2416-2425, 2013
Data Visualization on Interactive Surfaces: A Research Agenda from the DEXIS Workshop
IEEE CG&A, Vol. 33, No. 2, pp. 16-24, 2013 (Visualization Viewpoints)
Beyond Mouse and Keyboard: Expanding Design Considerations for Information Visualization Interactions
IEEE TVCG (InfoVis 2012), Vol. 18, No. 12, pp. 2689-2698, 2012
Understanding Pen and Touch Interaction for Data Exploration on Interactive Whiteboards
IEEE TVCG (InfoVis 2012), Vol. 18, No. 12, pp. 2779-2788, 2012
Data Analysis on Interactive Whiteboards through Sketch-based Interaction
Proceedings of ITS 2011, pp. 154-157, 2011