Share on Facebook Tweet on Twitter Share on LinkedIn Share by email
Pex4Fun

Pex4Fun is a browser-based teaching and learning environment targeting teachers and students for introductory to advanced programming or software engineering courses. At the core of the platform is an automated grading engine based on symbolic execution. In Pex4Fun, teachers can create virtual classrooms, customize existing courses, and publish new learning material including learning games.

Pex for fun

Try it out on the web

Go to www.pex4fun.com, and click Learn to start tutorials.

Main Publication to cite

Nikolai Tillmann, Jonathan De Halleux, Tao Xie, Sumit Gulwani, and Judith Bishop, Teaching and Learning Programming and Software Engineering via Interactive Gaming, in Proc. 35th International Conference on Software Engineering (ICSE 2013), Software Engineering Education (SEE), May 2013

Massive Open Online Courses (MOOCs) have recently gained high popularity among various universities and even in global societies. A critical factor for their success in teaching and learning effectiveness is assignment grading. Traditional ways of assignment grading are not scalable and do not give timely or interactive feedback to students. To address these issues, we present an interactive-gaming-based teaching and learning platform called Pex4Fun. Pex4Fun is a browser-based teaching and learning environment targeting teachers and students for introductory to advanced programming or software engineering courses. At the core of the platform is an automated grading engine based on symbolic execution. In Pex4Fun, teachers can create virtual classrooms, customize existing courses, and publish new learning material including learning games. Pex4Fun was released to the public in June 2010 and since then the number of attempts made by users to solve games has reached over one million. Our work on Pex4Fun illustrates that a sophisticated software engineering technique – automated test generation – can be successfully used to underpin automatic grading in an online programming system that can scale to hundreds of thousands of users.

More Publications by Pex4Fun team

    2014

    • Nikolai Tillmann, Judith Bishop, Nigel Horspool, Daniel Perelman, and Tao Xie, Code Hunt - Searching for Secret Code for Fun, in Proceedings of the 7th International Workshop on Search-Based Software Testing , June 2014

      Code Hunt - Searching for Secret Code for Fun

    • Nikolai Tillmann, Peli de Halleux, Tao Xie, and Judith Bishop, Code Hunt: Gamifying Teaching and Learning of Computer Science at Scale, in Proceedings of the 1st ACM Conference on Learning at Scale, , 2014

      Code Hunt (http://www.codehunt.com/) is an educational coding game (that runs in a browser) for teaching and learning computer science at scale. The game consists of a series of worlds and levels, which get increasingly challenging. In each level, the player has to discover a secret code fragment and write code for it. The game has sounds and a leaderboard to keep the player engaged. Code Hunt targets teachers and students from introductory to advanced programming or software engineering courses. In addition, Code Hunt can be used by seasoned developers to hone their programming skills or by companies to evaluate job candidates. At the core of the game experience is an automated program analysis and grading engine based on dynamic symbolic execution. The engine detects any behavioral differences between the player’s code and the secret code fragment. The game works in any modern browser, and currently supports C# or Java programs. Code Hunt is a dramatic evolution of our earlier Pex4Fun web plat- form, from which we have gathered considerable experience (including over 1.4 million programs submitted by users).

    2013

    • Nikolai Tillmann, Jonathan de Halleux, Tao Xie, and Judith Bishop, Pex4Fun: A Web-Based Environment for Educational Gaming via Automated Test Generation, in Proc. 28th IEEE/ACM International Conference on Automated Software Engineering (ASE 2013), Tool Demonstrations, November 2013

      Pex4Fun (http://www.pex4fun.com/) is a web-based educational gaming environment for teaching and learning programming and software engineering. Pex4Fun can be used to teach and learn programming and software engineering at many levels, from high school all the way through graduate courses. With Pex4Fun, a student edits code in any browser – with Intellisense – and Pex4Fun executes it and analyzes it in the cloud. Pex4Fun connects teachers, curriculum authors, and students in a unique social experience, tracking and streaming progress updates in real time. In particular, Pex4Fun finds interesting and unexpected input values (with Pex, an advanced test-generation tool) that help students understand what their code is actually doing. The real fun starts with coding duels where a student writes code to implement a teacher’s secret specification (in the form of sample-solution code not visible to the student). Pex4Fun finds any discrepancies in behavior between the student’s code and the secret specification. Such discrepancies are given as feedback to the student to guide how to fix the student’s code to match the behavior of the secret specification. This tool demonstration shows how Pex4Fun can be used in teaching and learning, such as solving coding duels, exploring course materials in feature courses, creating and teaching a course, creating and publishing coding duels, and learning advanced topics behind Pex4Fun.

    • Thomas Ball, Peli de Halleux, Daan Leijen, and Nikhil Swamy, Increasing Human-Tool Interaction via the Web, in 11th ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, ACM, June 2013

      Software tools researchers can accelerate their ability to learn by exposing tools to users via web technologies, allowing them to observe and test the interactions between humans and tools. At Microsoft Research, we have developed a web service (http: //www.rise4fun.com/) for such a purpose that is available for community use.

    • Tao Xie, Nikolai Tillmann, and Jonathan de Halleux, Educational Software Engineering: Where Software Engineering, Education, and Gaming Meet, in Proc. 3rd International Workshop on Games and Software Engineering (GAS 2013), May 2013

      We define and advocate the subfield of educational software engineering (i.e., software engineering for education), which develops software engineering technologies (e.g., software testing and analysis, software analytics) for general educational tasks, going beyond educational tasks for software engineering. In this subfield, gaming technologies often play an important role together with software engineering technologies. We expect that researchers in educational software engineering would be among key players in the education domain and in the coming age of Massive Open Online Courses (MOOCs). Educational software engineering can and will contribute significant solutions to address various critical challenges in education especially MOOCs such as automatic grading, intelligent tutoring, problem generation, and plagiarism detection. In this position paper, we define educational software engineering and illustrate Pex for Fun (in short as Pex4Fun), one of our recent examples on leveraging software engineering and gaming technologies to address educational tasks on teaching and learning programming and software engineering skills.

    2012

    2011

    • Nikolai Tillmann, Jonathan de Halleux, and Tao Xie, Pex for Fun: Engineering an Automated Testing Tool for Serious Games in Computer Science, no. MSR-TR-2011-41, March 2011

      Although there are various emerging serious games developed for education and training purposes, there exist few serious games for practitioners or students to improve their programming or problem-solving skills in the computer science domain. To provide an open platform for creating serious games in learning computer science, we have developed a web‐based serious gaming environment called Pex for Fun, in short Pex4Fun (http://www.pexforfun.com/), for learning critical computer science skills such as problem solving skills and abstraction skills. It works on any web‐enabled device, even a smart phone. It comes with an auto‐completing code editor, providing a user with instant feedback similar to the code editor in Microsoft Visual Studio. It is a cloud application with the data in the cloud, enabling a user to use it anywhere where an Internet connection is available. New learners of programming can play games there created by us to master basic programming concepts such as arrays, loops, and exception handling. In fact, any user can create new games for others to play.

    • Judith Bishop, Jonathan de Halleux, Nikolai Tillmann, R. Nigel Horspool, Don Syme, and Tao Xie, Browser-based software for technology transfer, in SAICSIT Conf., 2011
    • Nikolai Tillmann, Jonathan de Halleux, and Tao Xie, Pex4Fun: Teaching and learning computer science via social gaming, in CSEE&T, 2011

    2010

    • Tao Xie, Jonathan de Halleux, Nikolai Tillmann, and Wolfram Schulte, Teaching and Training Developer-Testing Techniques and Tool Support , in of the 25th Annual ACM Conference on Systems, Programming, Languages, and Applications: Software for Humanity (SPLASH 2010), Educators' and Trainers' Symposium, Reno/Tahoe Nevada, Association for Computing Machinery, Inc., October 2010

    Publications by the community

    Rishabh Singh, Sumit Gulwani, Armando Solar-Lezama.
    Automated Feedback Generation for Introductory Programming Assignments
    To appear in PLDI 2013

    Kiran Lakhotia
    En Garde: Winning Coding Duels Through Genetic Programming
    6th International Workshop on Search-Based Software Testing (SBST 2013)

    Schweizer, Daniel.
    Overapproximating the Cost of Loops
    Eidgenössische Technische Hochschule Zürich, Department of Computer Science, Chair of Programming Methodology (2013).

    Tao Xie.
    Cooperative Testing and Analysis: Human-Tool, Tool-Tool, and Human-Human Cooperations to Get Work Done. [PDF][BibTeX]
    In Proceedings of the 12th International Working Conference on Source Code Analysis and Manipulation (SCAM 2012), Keynote Paper, Trento, Italy, Pages 1-3, September 2012.

    Ning Chen, Sunghun Kim.
    Puzzle-based automatic testing: bringing humans into the loop by solving puzzles
    In Proceedings of the ASE 2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering

    Community

    Are you interested in using Pex4fun for your studies? Or do you have some research ideas that involve mining the >1,000,000 program submissions? Do you have a published paper we can add to our community list? Don't hesitate to write us at pexdata@microsoft.com.

    Roadmap

    Stay tuned for a new front- and back-end. The new back-end of the Pex4Fun engine will introduce a REST-based cloud API that allows you to automate program analysis tasks. The new front-end will make Pex4Fun even more fun!

     

    Watch video!

    Core Team
    Nikolai Tillmann
    Nikolai Tillmann

    Peli de Halleux
    Peli de Halleux

    Judith Bishop
    Judith Bishop

    External Collaborators


    Tao Xie, UIUC
    Associate Professor


    Nigel Horspool, UVic
    Professor