Sharing memories and experiences is a major activity of human beings that has been practiced during thousands of years, enabling the exchange of knowledge and information across generations and cultures. Pictures and videos are rich vehicles to transport this information. Recent technological advances of the digital devices have been changing the concepts of capture, share and store of visual information (e.g., camera-phones with integrated GPS and considerable storage capacity). People can take photos or make small video clips of everything, everywhere and, through the World Wide Web share this information with friends. Consequently, a vast amount of personal digital information is being stored. To manage this information in a fruitful way, an efficiently retrieval system must be used. In this project a multimedia information retrieval will be developed which will use audio and visual features extracted from image/video data and metadata obtained at capture time to find information in personal collections. The low-level features will be used to train probabilistic models for specific semantic concepts to retrieve information in personal memories. The metadata will be captured automatically using the camera parameters and other sensors incorporated in the device. User interfaces will be designed in order to make easy the search and navigation in the database both for desktop/laptop PC's including a web component but also for small displays in mobile devices. The system will be evaluated using personal collections of several persons and in tourism related such as a visit to a museum, a historical site or others points of interest.
PublicationsJesus, R., Gonçalves, D., Abrantes, A., Correia, N., Playing Games as a Way to Improve Automatic Image Annotation, Proceedings of IEEE International Workshop on Semantic Learning Applications in Multimedia (SLAM08), in conjuntion with CVPR08 (2008).
Jesus, R., Dias, R., Frias, R., Abrantes, A, Correia, N., Memoria Mobile: Sharing Pictures of a Point of Interest, Proceedings of the Working Conference on Advanced Visual interfaces (AVI '08). ACM, New York, NY, (2008).
Gonçalves, D., Jesus, R., and Correia, N. 2008. A gesture based game for image tagging. In CHI '08 Extended Abstracts on Human Factors in Computing Systems (Florence, Italy, April 05 - 10, 2008). CHI '08. ACM, New York, NY, 2685-2690.
Jesus, R., Dias, R., Frias, R., Abrantes, A., Correia, N., Sharing Personal Experiences while Navigating in Physical Spaces. In 5th Workshop on Multimedia Information Retrieval, proceedings of the 30th international ACM Information Retrieval Conf (SIGIR,Ricardo Dias, Rui Jesus, Rute Frias, Nuno Correia, Mobile Interface of the MEMORIA Project. In Proceedings of the 30th international ACM Information Retrieval Conf (SIGIR,
Rui M. Jesus, Ricardo Dias, Rute Frias, Nuno Correia. Geographic image retrieval in mobile guides. Proceedings of the 4th ACM workshop on Geographical information retrieval, Lisbon, Portugal, November 9, 2007.Jesus, R., Santos, E., Frias, R., Correia, N. An Interface to Explore Personal Memories. 15th Portuguese Computer Graphics Group Conference, Porto
Jesus, R., Abrantes, A., Correia, N., Photo Retrieval from Personal Memories using Generic Concepts. Advances in Multimedia Information Processing - PCM 2006, Springer LNCS, 2006. 4261: p. 633-640.
Jesus, R., Martins, T., Frias, R., Abrantes, A., Correia, N., PhotoNav: a System to Capture, Share and Access Personal Memories. In Memories for life Colloquium,