Pex4Fun is a browser-based teaching and learning environment targeting teachers and students for introductory to advanced programming or software engineering courses. At the core of the platform is an automated grading engine based on symbolic execution. In Pex4Fun, teachers can create virtual classrooms, customize existing courses, and publish new learning material including learning games.
Massive Open Online Courses (MOOCs) have recently gained high popularity among various universities and even in global societies. A critical factor for their success in teaching and learning effectiveness is assignment grading. Traditional ways of assignment grading are not scalable and do not give timely or interactive feedback to students. To address these issues, we present an interactive-gaming-based teaching and learning platform called Pex4Fun. Pex4Fun is a browser-based teaching and learning environment targeting teachers and students for introductory to advanced programming or software engineering courses. At the core of the platform is an automated grading engine based on symbolic execution. In Pex4Fun, teachers can create virtual classrooms, customize existing courses, and publish new learning material including learning games. Pex4Fun was released to the public in June 2010 and since then the number of attempts made by users to solve games has reached over one million. Our work on Pex4Fun illustrates that a sophisticated software engineering technique – automated test generation – can be successfully used to underpin automatic grading in an online programming system that can scale to hundreds of thousands of users.
- Nikolai Tillmann, Peli de Halleux, Tao Xie, and Judith Bishop, Code Hunt: Gamifying Teaching and Learning of Computer Science at Scale, in Proceedings of the 1st ACM Conference on Learning at Scale, , March 2014.
Code Hunt (http://www.codehunt.com/) is an educational coding game (that runs in a browser) for teaching and learning computer science at scale. The game consists of a series of worlds and levels, which get increasingly challenging. In each level, the player has to discover a secret code fragment and write code for it. The game has sounds and a leaderboard to keep the player engaged. Code Hunt targets teachers and students from introductory to advanced programming or software engineering courses. In addition, Code Hunt can be used by seasoned developers to hone their programming skills or by companies to evaluate job candidates. At the core of the game experience is an automated program analysis and grading engine based on dynamic symbolic execution. The engine detects any behavioral differences between the player’s code and the secret code fragment. The game works in any modern browser, and currently supports C# or Java programs. Code Hunt is a dramatic evolution of our earlier Pex4Fun web plat- form, from which we have gathered considerable experience (including over 1.4 million programs submitted by users).
- Nikolai Tillmann, Jonathan de Halleux, Tao Xie, and Judith Bishop, Pex4Fun: A Web-Based Environment for Educational Gaming via Automated Test Generation, in Proc. 28th IEEE/ACM International Conference on Automated Software Engineering (ASE 2013), Tool Demonstrations, November 2013.
Pex4Fun (http://www.pex4fun.com/) is a web-based educational gaming environment for teaching and learning programming and software engineering. Pex4Fun can be used to teach and learn programming and software engineering at many levels, from high school all the way through graduate courses. With Pex4Fun, a student edits code in any browser – with Intellisense – and Pex4Fun executes it and analyzes it in the cloud. Pex4Fun connects teachers, curriculum authors, and students in a unique social experience, tracking and streaming progress updates in real time. In particular, Pex4Fun ﬁnds interesting and unexpected input values (with Pex, an advanced test-generation tool) that help students understand what their code is actually doing. The real fun starts with coding duels where a student writes code to implement a teacher’s secret speciﬁcation (in the form of sample-solution code not visible to the student). Pex4Fun ﬁnds any discrepancies in behavior between the student’s code and the secret speciﬁcation. Such discrepancies are given as feedback to the student to guide how to ﬁx the student’s code to match the behavior of the secret speciﬁcation. This tool demonstration shows how Pex4Fun can be used in teaching and learning, such as solving coding duels, exploring course materials in feature courses, creating and teaching a course, creating and publishing coding duels, and learning advanced topics behind Pex4Fun.
- Thomas Ball, Peli de Halleux, Daan Leijen, and Nikhil Swamy, Increasing Human-Tool Interaction via the Web, in 11th ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, ACM, June 2013.
Software tools researchers can accelerate their ability to learn by exposing tools to users via web technologies, allowing them to observe and test the interactions between humans and tools. At Microsoft Research, we have developed a web service (http: //www.rise4fun.com/) for such a purpose that is available for community use.
- Tao Xie, Nikolai Tillmann, and Jonathan de Halleux, Educational Software Engineering: Where Software Engineering, Education, and Gaming Meet, in Proc. 3rd International Workshop on Games and Software Engineering (GAS 2013), May 2013.
We deﬁne and advocate the subﬁeld of educational software engineering (i.e., software engineering for education), which develops software engineering technologies (e.g., software testing and analysis, software analytics) for general educational tasks, going beyond educational tasks for software engineering. In this subﬁeld, gaming technologies often play an important role together with software engineering technologies. We expect that researchers in educational software engineering would be among key players in the education domain and in the coming age of Massive Open Online Courses (MOOCs). Educational software engineering can and will contribute signiﬁcant solutions to address various critical challenges in education especially MOOCs such as automatic grading, intelligent tutoring, problem generation, and plagiarism detection. In this position paper, we deﬁne educational software engineering and illustrate Pex for Fun (in short as Pex4Fun), one of our recent examples on leveraging software engineering and gaming technologies to address educational tasks on teaching and learning programming and software engineering skills.
- Nikolai Tillmann, Jonathan de Halleux, Tao Xie, and Judith Bishop, Pex4Fun: Teaching and Learning Computer Science via Social Gaming, in Proc. 24th IEEE-CS Conference on Software Engineering Education and Training (CSEE&T 2012), Tutorial, April 2012.
- Proceedings of the 43rd ACM technical symposium on Computer science education, SIGCSE 2012, Raleigh, NC, USA, February 29 - March 3, 2012, in SIGCSE, ACM, 2012.
- Nikolai Tillmann, Jonathan de Halleux, Tao Xie, and Judith Bishop, Teaching and learning computing via social gaming with Pex4Fun (abstract only), in SIGCSE, ACM, 2012.
- Nikolai Tillmann, Jonathan de Halleux, and Tao Xie, Pex for Fun: Engineering an Automated Testing Tool for Serious Games in Computer Science, no. MSR-TR-2011-41, March 2011.
Although there are various emerging serious games developed for education and training purposes, there exist few serious games for practitioners or students to improve their programming or problem-solving skills in the computer science domain. To provide an open platform for creating serious games in learning computer science, we have developed a web‐based serious gaming environment called Pex for Fun, in short Pex4Fun (http://www.pexforfun.com/), for learning critical computer science skills such as problem solving skills and abstraction skills. It works on any web‐enabled device, even a smart phone. It comes with an auto‐completing code editor, providing a user with instant feedback similar to the code editor in Microsoft Visual Studio. It is a cloud application with the data in the cloud, enabling a user to use it anywhere where an Internet connection is available. New learners of programming can play games there created by us to master basic programming concepts such as arrays, loops, and exception handling. In fact, any user can create new games for others to play.
- Judith Bishop, Jonathan de Halleux, Nikolai Tillmann, R. Nigel Horspool, Don Syme, and Tao Xie, Browser-based software for technology transfer, in SAICSIT Conf., 2011.
- Nikolai Tillmann, Jonathan de Halleux, and Tao Xie, Pex4Fun: Teaching and learning computer science via social gaming, in CSEE&T, 2011.
- Tao Xie, Jonathan de Halleux, Nikolai Tillmann, and Wolfram Schulte, Teaching and Training Developer-Testing Techniques and Tool Support , in of the 25th Annual ACM Conference on Systems, Programming, Languages, and Applications: Software for Humanity (SPLASH 2010), Educators' and Trainers' Symposium, Reno/Tahoe Nevada, Association for Computing Machinery, Inc., October 2010.
Publications by the community
Rishabh Singh, Sumit Gulwani, Armando Solar-Lezama.
Automated Feedback Generation for Introductory Programming Assignments
To appear in PLDI 2013
En Garde: Winning Coding Duels Through Genetic Programming
6th International Workshop on Search-Based Software Testing (SBST 2013)
Overapproximating the Cost of Loops
Eidgenössische Technische Hochschule Zürich, Department of Computer Science, Chair of Programming Methodology (2013).
Cooperative Testing and Analysis: Human-Tool, Tool-Tool, and Human-Human Cooperations to Get Work Done. [PDF][BibTeX]
In Proceedings of the 12th International Working Conference on Source Code Analysis and Manipulation (SCAM 2012), Keynote Paper, Trento, Italy, Pages 1-3, September 2012.
Ning Chen, Sunghun Kim.
Puzzle-based automatic testing: bringing humans into the loop by solving puzzles
In Proceedings of the ASE 2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering
Are you interested in using Pex4fun for your studies? Or do you have some research ideas that involve mining the >1,000,000 program submissions? Do you have a published paper we can add to our community list? Don't hesitate to write us at firstname.lastname@example.org.
Stay tuned for a new front- and back-end. The new back-end of the Pex4Fun engine will introduce a REST-based cloud API that allows you to automate program analysis tasks. The new front-end will make Pex4Fun even more fun!