The RoomAlive Toolkit is an open source SDK that enables developers to calibrate a network of multiple Kinect sensors and video projectors. The toolkit also provides a simple projection mapping sample that can be used as a basis to develop new immersive augmented reality experiences similar to those of the IllumiRoom and RoomAlive research projects.
Code Hunt is a serious educational game. The Code Hunt community is interested in all aspects of research and development around the game, including analysis of the data and development of the platform.
Code Hunt is a serious gaming platform for coding contests and practicing programming skills. It is based on the symbolic white box execution engine, Pex. Code Hunt is unique as an online coding platform in that each puzzle is presented with test cases only, no specification. Players have to first work out the pattern and then code the answer. Code Hunt has been used by over 100,000 players as of February 2015.
We publish a subset of the data from the paper "Discriminative Ferns Ensemble for Hand Pose Recognition".
Stroke Recovery with Kinect is an interactive rehabilitation system that helps stroke patients improve their upper-limb motor functioning in the comfort of their own home. By using the Microsoft Kinect sensor’s gesture recognition technology, the system recognizes and interprets the user’s movements, assesses their rehabilitation progress, and adjusts the level of difficulty for subsequent therapy sessions.
Pex4Fun is a browser-based teaching and learning environment targeting teachers and students for introductory to advanced programming or software engineering courses. At the core of the platform is an automated grading engine based on symbolic execution. In Pex4Fun, teachers can create virtual classrooms, customize existing courses, and publish new learning material including learning games.
IllumiRoom is a proof-of-concept system from Microsoft Research. It augments the area surrounding a television screen with projected visualizations to enhance the traditional living room entertainment experience.
Building a 6-dof tracker using the Kinect projector.
Participating phones are positioned in a stack, each phone held by a different player. The top phone's screen displays its camera's viewpoint, overlaid with a maze incorporating an AR tag, and a ball guided by accelerometers. When the ball is guided to the end of the maze it 'falls' through a hole, at which point the second phone can 'catch' it, but only if it is positioned correctly. At this point the phones swap positions to continue the game.
We exploit the falloff of visual acuity away from the gaze direction in the human visual system for dynamic 3D rendering. Through user studies, we have honed our system parameters and demonstrated the effectiveness of the system. We have also shown the system to bring significant performance increases, or equivalent reductions in hardware and power requirements, in typical 3D rendering applications on existing hardware. Finally, the method is easily integrated into existing 3D applications.
Seeking to offer Kinect-like experiences using only mobile devices, leading to new mobile gaming experiences. Key technical achievements include development of fast, accurate and robust phone-to-phone distance measurement system, and a 3D phone-to-phone measurement techniques. Several innovative games, including SwordFight, have been prototyped.
Rainbow Tangle is a game on a sequence of permutations. The purpose of the game is to create as straight routes of the permutation particles as possible.
A selection of systems that weave language-learning experiences into the fabric of our everyday lives by leveraging our natural abilities and interests. These systems include a Windows Phone 7 game for learning Mandarin sounds, a Microsoft Kinect game for learning spatial language, and web flashcards that integrate skill-based microgames. Each represents part of a vision in which personalized learning services orchestrate connected learning experiences across devices, activities, and contexts.
We use data from Xbox games to investigate the impact of social play, retention of players, and usage of game features. All this can help to inform engineering decisions during game development.
Just Press Play is a collaboration with Rochester Institute of Technology (RIT) to wrap gameful and ludic experiences around the traditional pedagogical processes of a modern university. Starting in October of 2011, 750 students at RIT’s school of Interactive Games and Media received some cryptic instructions to log onto a website, create a profile, talk to a professor, and pick up a keychain.
We investigate new types of gaming apps on mobile phones: High-speed, locational, phone-to-phone games. The goal is to develop games such as a real-time, face-to-face SwordFight game without any external infrastructure. Two players Alice and Bob are located in the same room, and use their phones as if they were light saber swords. Alice and Bob wave their phones; the orientation and location of one phone relative to the other simulates sword striking and blocking.
Sketch Match is powered by MindFinder research work. The app supports users to search for similar images by sketching. It also provides a sketch matching game for entertainment and learning.
Supporting collaborations between major research institutions and Microsoft Research Connections, this program intends to build the foundations for a unified game layer for education. By instrumenting our everyday experiences, we can transform them into gameful experiences and, by doing so, provide the inputs for entirely new capabilities such as eportfolios, adaptive learning, and project-based learning. We look forward to inviting you to the game!
We describe the new symbolic differentiation feature in HLSL. We provide details of the compiler implementation, along with information relevant to a shader writer wanting to use the feature.
We think differently about games. Instead of disconnected experiences that are difficult to assess, we empower the educator to teach in new ways. Technological innovations drive our research toward building a new platform for education. Instrumentation of the digital and analog worlds of educators, students, administrations, and institutions enables new kinds of longitudinal studies. We engage with the community and advocate within Microsoft to transform the education experience.
MirageBlocks demo showcases the use of 3D stereo projection combined with Kinect camera to capture, display, and interact with 3D objects. Any physical object can instantly be virtualized and then interacted with with bare hands using a realistic physical simulation.
Kinect for Xbox 360 and Windows makes you the controller by fusing 3D imaging hardware with markerless human-motion capture software. Our group investigates such software. Mixing computer vision, graphics, and machine learning techniques, we look at how to build algorithms that can learn to recognize human poses quickly and reliably.
Microsoft Research has launched The Path of Go, an Xbox Live Arcade game based on ‘Go’, the ancient Chinese board game.
Orleans is a software framework for building client + cloud applications. Orleans encourages use of simple concurrency patterns that are easy to understand and implement correctly, building on an actor-like model with declarative specification of persistence, replication, and consistency and using lightweight transactions to support the development of reliable and scalable client + cloud software.
The complexity of game of Go is greater than that of Chess; the most advanced Go computer players reach at best the level of a human amateur. We believe that machine learning can radically improve computer Go.