Home    About    Team    Projects    Publications    Awards    News   

ASAA - A gesture based game for image tagging

Multimedia content is being spread throughout the web at a pace that has never been seen before. Videos and images are filling our computers, blogs and internet communities. It is necessary to organize and index all of this content for better search and visual usability. One way to do this is to assign text to that metadata by means of keywords - taggs. The use of automatic tagging systems is a solution for this problem, but these systems are still behind when it comes to semantic significance. Other approach is done manually by humans - we have good and useful computation as well as visual association skills. Nevertheless, this constitutes a dilemma, because this is a process that is slowed by the lack of human motivation and entertainment.
This project tries to engage a deeper understanding of that motivation and also to propose a way for people to have fun while tagging images. There is relevant work done in this area (see related work in paper), so the objective is to take a step towards different case studies and scenarios, mostly where people have idle time to spend. This scenarios include Leisure activities for people who want to have fun tagging photos, specially in the places where these photos can be taken. Using idle time at airports, bus stops, or waiting rooms is also a scenario to explore, as well as Educational purposes for children that could use this application to develop image/text matching skills while having fun. At last (but not least), using this application for Rehabilitation purposes, for people who have problems like aphasia.

Prior to the computational prototype, a paper prototyping was made and tested with a series of students (described in the paper). It is important to understand issues regarding usability, interactive design and acceptance by the users. The next phase consists in a series of usability tests, using the lab and also some public places, where people freely interact with the game, using random photos, or even their own, to see different motivations and results. We are also testing some other version of the game where people do not interact with gestures but with a mouse and keyboard, to understand if they prefer playing gesture based games or simply games.

Computational Protoype
The ASAA interface

This work has been submitted and accepted for ACM - CHI2008 Work In Progress and was presented at Florence April, 2008

ASAA poster presentation

This work is part of Rui Jesus PhD and Duarte Gonçalves MSc theses supervised by Professor Nuno Correia

ASAA demo video


"This paper proposes and discusses the ASAA (Application for Semi-Automatic Annotation) interface, a new computer game for image tagging. The application is composed by a 3D game interface, a game engine that uses a system for automatic image annotation and gestural input to play the game..."

See the full paper at ACM portal

Current Projects 
Personal Memories 
Previous Projects