My research concerns the development of intelligent machines - to “teach” computers to (1) understand the behaviour and intent of humans, and (2) to correctly interpret (“Perceive” or “See”) objects and scenes depicted in colour/depth images or videos. I work in the areas of Computer Vision (RGB and 3D - KINECT), Machine Learning, Discrete Optimization, Behavioural Game Theory and Human-Computer Interaction. My current research interests include 3D Reconstruction and Rendering, Probabilistic Programming, Interpretable and Verifiable Knowledge Representations from Deep Models. In terms of applications, I am interested in Conversation agents for Task completion, Machine learning systems for Healthcare and 3D rendering and interaction for augmented and virtual reality.
(Feb 2016) I am starting a new AI team in Microsoft Research called Cognition. We are Hiring!
(May 2015) I have moved to Microsoft Redmond where I will be acting as the Machine Learning advisor to the Chief Research Officer, Rick Rashid.
Some Recent Projects
Academic and Industrial Duties (All Academic Duties)
Recent News (All Updates)
- May 2016, New work on Hand pose estimation will appear in SIGGRAPH 2016.
- May 2016, Our paper on real time full body reconstruction (Fusion 4D) will appear in SIGGRAPH 2016.
- March 2016, We have developed a super efficient algorithm (Global Patch Collider) for correspondece estimation that will appear in CVPR 2016.
- March 2016, Paper on Layered Scene Decomposition via the Occlusion-CRF will be presented at CVPR 2016.
- February 2016, Our new Visual stortelling dataset and its accompanying paper will be released at NAACL 2016.
- February 2016, Cloze Test - our proposal for evaluating the ability to cature narrative structure - to be released in our NAACL 2016 paper.
- January 2016, Moving to the Microsoft headquaters in Redmond to start a new AI team.
- December 2015, Paper on Non-greedy training of decision trees appears in NIPS 2015.
- December 2015, Paper on interpretable autoencoders - Deep Convolutional Inverse Graphics Network is presented at NIPS 2015.
- September 2015, Paper on Hierarchical Sampling Optimization for Estimating Human Hand Pose will appear at ICCV 2015.
- August 2015, MobileFusion has received a lot of press converage (see articles in (WIRED, Mashable, and even the Register)
- August 2015, Our paper Mobile Fusion that describes a method for generating 3D reconstruction of objects using a standard smartphone has been accepted to appear in ISMAR 2015.
- July 2015, Our paper on Maximum Flows by Incremental Breadth-First Search (with Andrew Goldberg, Sagi Hed, Robert Tarjan, Renato Werneck and Haim Kaplan) will appear at the European Symposium on Algorithms (ESA)
- July 2015, Interactive Scene Understanding (Semantic Paint) is covered widely in the Press (BBC, Engadget, and even the Daily Mail).
- July 2015, Our paper on learning to decipher heaps for software verificiation wins the Best Paper Award at the Constructive ML workshop at ICML 2015.
- June 2015, Our paper on "Interactive Scene Understanding" - a collaboration with University of Oxford and Stanford will be presented at SIGGRAPH 2015. See video here.
- June 2015, The "Picture Probablitic Programming" paper receives the Best Paper Honorable Mention Award at CVPR 2015.
- May 2015, Paper on learning Perturbation Models with Multidimensional Parametric Min-cuts has been accepted to appear in UAI 2015.
- May 2015, Paper on Information Gathering in Networks via Active Exploration has been accepted to appear in IJCAI 2015.
- May 2015, Video of the Hand Pose Estimation work is now available here.
- April 2015, Picture, a new programming language for vision problems, has been accepted to appear in CVPR 2015 and has received a lot of press coverage (MIT News, The Register, Scientific Computing, iProgrammer).
- April 2015, Paper on Real-time Hand Pose Estimation is appearing in CHI 2015.
- March 2015, Our paper on fast hashing has been accepted to appear in CVPR 2015.
- February 2015, Paper on Crowdsourcing Language Understanding in the Wild is accepted to appear in WWW 2015. See blog posts here and here.
- January 2015, Paper on Consensus Message Passing is accepted to appear in AISTATS 2015
- December 2014, Joined the Advisory board for the NEMOG (New Economic Models for Digital Games) project.
- November 2014, Paper on Learning with Multiple Annotation-specific Loss Functions is accepted to appear in EMMCVPR 2015.
- October 2014, Paper and Video describing our work on automatic layout of virtual objects for augmented reality released at ISMAR 2014.
- October 2014, Paper on Just-in-Time Inference is accepted to appear in NIPS 2014.
- September 2014, Paper on Real-time Face Reconstruction from a Single Depth Image is accepted to appear in 3DV 2014.
- August 2014, Our paper on Exploration of Group Viewing Patterns Paper is the Runner-up for the Best Paper Award at TVX 2014.
- July 2014, 3 papers (contour completion, learning with perceptual loss functions, Non-parametric Higher-order MRFs) are accepted to appear in ECCV 2014.
- June 2014, Our demo on FilterForest for Image Denoising is the Runner-up for the Best Demo Award at CVPR 2014.
- May 2014, Paper on automatic layout of virtual objects for augmented reality is accepted to appear at ISMAR 2014.
- April 2014, Paper on community priors for crowdsourcing is the Runner-up for the Best Paper Award at WWW 2014.
- March 2014, Paper on Depth from IR illumination fall-off is accepted to appear in SIGGRAPH 2014.
- February 2014, Our edited volume Tractability: Practical Approaches to Hard Problems is released by Cambridge University Press.
- February 2014, 3 Papers (Learning Portfolios for camera relocatlization, Personalized gesture recognition, Filter Forests for Image Labelling) are accepted to appear in CVPR (2 for oral presentation).
- January 2014, Paper on encoraging diversity in multiple-output prediction is accepted for oral presentation at AISTATS 2014.
- January 2014, Paper on community priors for crowdsourcing is accepted to appear in WWW 2014.
- December 2013, Our paper on the effect of principles on power of agents in strategic games is accepted to appear in AAMAS 2014.
- December 2013, Paper on User Behaviour Adaptation Under Interface Change accepted to appear in IUI 2014.
- November 2013, Our Infer.Net based model for crowdsourcing Won the CrowdScale challenge at HCOMP 2013.
- October 2013, Gave talk at the Human Behaviour Understanding workshop at ACM multimedia.
- September 2013, Paper on Decision DAGs accepted to appear in NIPS 2013.
- September 2013, Invited to join the editorial board of CVIU.
- August 2013, Paper on Sementic labelling of Voxel Spaces accepted to appear in ICCV 2013.
- July 2013, Paper on text detection and recognition accept to appear at ICDAR 2013.
- June 2013, Tutorial on Solving real world problems with RGBD sensors at CVPR 2013.
- June 2013, Gave a talk at the London School of Economics (LSE) on the value of Big Data.
- April 2013, Papers on online algorithms for diverse recommendations and computation of coalition structures in coalition games are accepted to appear in AAAI 2013.
- March 2013, Four papers (two for oral presentation) accepted to appear in CVPR 2013.
- February 2013, Invited to join the editorial board of IJCV.
- January 2013, Paper on faster training of structural SVMs is accepted to appear in AISTATS 2012 ( video ).
- October 2012, PAMI paper describing the pose estimation system for KINECT has been accepted.
- September 2012, Two papers (1) structured output prediction with multiple choices (2) context driven random forests are accepted to appear in NIPS 2012.
- August 2012, The accepted ECCV and DAGM papers are now available online.
- July 2012, Jamie Shotton and I have written a new book chapter on our recent work on Human Pose Estimation for the KINECT.
- June 2012, Five papers (two for oral presentation) accepted to appear in the European Conference on Computer Vision (ECCV).
- May 2012, The MSR KINECT gesture dataset collected for our CHI 2012 paper is now available.
- April 2012, 3 Papers on Behavioural Game Theory and Personality-Online Behaviour patterns are accepted to appear in ACM WebSci 2012 .
- March 2012, Papers accepted to CHI 2012, AISTATS 2012, Eurographics 2012 and PAMI/IJCV are now available online.
- January 2012, Carsten Rother and I are teaching the Advanced Computer Vision course in the Engineering department of the University of Cambridge.
- October 2011, Our paper on simulataneous localization and 3D mapping wins the Best Paper Award at ISMAR 2011.
- October 2011, Interviews on the role of Behavioural game theory appear in Forbes and The Economic Times.
- September 2011, Our first Facebook app (MSR Project Waterloo) for conducting behavioural game theory experiments is now online!
- September 2011, The KinectFusion system has been made public. (Project Page) (video) (See UIST and ISMAR papers for detail)
- June 2011, Our book on Advances in Markov Random Fields for Vision and Image Processing has been published by MIT press.
- June 2011, 4 papers on adpative algorithms for MAP inference appear in AISTATS 2011, ICML 2011 and CVPR 2011.
- December 2011, Our paper on evaluation and learning of interactive Segmentation systems wins the Best Paper Award at ICVGIP 2010.
- October 2010, Our paper on inference with co-occurence potential wins the Best Paper Award at ECCV 2010.
Books and Edited Volumes