Magic Bullet: A Dual-Purpose Computer Game

"Play it to have fun, and help with research"

(Try it out, simply click and play!)

Jeff Yan and Su-Yang Yu

INTRODUCTION

Magic Bullet is an online game that we have developed to streamline the robustness evaluation of CAPTCHAs, as a spin-off from our previous project that identified critical vulnerabilties in multiple widely deployed CAPTCHAs including those used by Microsoft and Yahoo.

CAPTCHA has been widely deployed by commercial web sites as a security technology for purposes such as anti-spam. A common approach to evaluating the robustness of CAPTCHA is the use of machine learning techniques. Critical to such machine learning approaches is the acquisition of an adequate set of labeled samples, on which the learning techniques are trained. However, such sample labeling is difficult for computers, since the strength of CAPTCHAs stems exactly from the difficulty computers have in recognizing either distorted texts or image contents. Therefore, until now, researchers have to complete the labeling process manually, which is tedious and expensive.

Magic Bullet for the first time turns such sample labeling into a fun experience, and that has achieved a labeling accuracy of as high as 98% in a pilot study. MB serves two purposes simultaneously: 1) like any other games, people play MB just for fun, and 2) people's game play contributes to solving a real problem that has practical utilities but which no known computer algorithm can yet solve, although they might not realize this at all.

GAME DESIGN

Inspired by the EPS game (aka the Google image labeler), Magic Bullet is a competitive online game, where at least two teams play against each other, with each team having at least two players. The game theme is collaborative shooting, and players in the same team must cooperate to win the game.

In a typical setting (as we have implemented so far), MB is a 2 vs. 2 team based game. Each of the four players will have the same game interface, a screen with two targets - one is for their own team and the other for their opponents (see Figure 1). During each game round, a randomly chosen character image - typically a segmented CAPTCHA character - is displayed in a position an equal distance away from the two targets.

Figure 1. A screenshot of the Magic Bullet game

Key game rules are:

You and three other players will be randomly assigned to two pairs, each being a team.

You do not know who you are playing with/against; no in-game communication between players is provided.

The character image will become a bullet moving slowly towards the target of the team that first enters any key.

The team that first agrees on the character image wins the current round of the game - the bullet will zoom across the gaming area to hit their target, and score points for this team.

The movement of the bullet can drastically change as if by magic - hence the game name "Magic Bullet".

People play the game to beat friends, win points and have fun. However, their game play also tells researchers which image is which character - the assocation of the correct characters and the images are "labeled samples", which can contribute to the development of smarter computer programs and improvement of computer security.

The design and implementation details, together with an initial evaluation, of MB are reported in our academic paper, which will appear at IJCAI'09, a leading AI conference in Pasadena, CA in the summer, 2009.

APPLICATIONS

Magic Bullet was initially designed to label segmented CAPTCHA characters. However, it can also be used:

To aid the development of better algorithms for handwriting recognition. When developing handwriting recognition algorithms, it is routine to segment cursive texts and then manually label character samples. MB can streamline such a process.

As a training tool for improving people's keyboard typing skills, since fast but accurate keyboard typing is key to winning the game.

As a novel CAPTCHA (probably not particularly strong though). A website can ask people to play this game for a while to win enough points before they are considered humans and allowed to submit a valid comment or other web form.
To do binary labeling, where only two types of objects are required to be labeled respectively. We have extended MB to provide a generic solution to binary labeling. One application of this extension is to streamline the robustness evaluation of image CAPTCHAs such as Microsoft's Assira. Of course, this extension (e.g. our "Cat or Dog" game), can be a novel CAPTCHA itself, which is stronger than the MB-game CAPTCHA.

TRY IT OUT!

Magic Bullet is a web game. You don't have to install anything, just click and play at here. Any browser on any OS will do.

CONCLUSIONS

Magic Bullet is a simple but novel extension to the ESP game. It provides the first effective solution to some labeling problems that the famous ESP game fails to address. Being both collaborative and competitive in nature, Magic Bullet also appears to define a novel genre of human computation games.