Vision-Based Interaction within a Multimodal Framework

No Thumbnail Available
Date
2022
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
Our contribution is to the field of video-based interaction techniques and is integrated in the home environment of the EMBASSI project. This project addresses innovative methods of man-machine interaction achieved through the development of intelligent assistance and anthropomorphic user interfaces. Within this project, multimodal techniques represent a basic requirement, especially considering those related to the integration of modalities. We are using a stereoscopic approach to allow the natural selection of d evices via pointing gestures. The pointing hand is segmented from the video images and the 3D position and orientation of the forefinger is calculated. This modality has a subsequent integration with that of speech, in the context of a multimodal interaction infrastructure. In a first phase, we use semantic fusion with amodal input, considering the modalities in a so-called late fusion state.
Description

        
@inproceedings{
10.2312:pt.20011318
, booktitle = {
10º Encontro Português de Computação Gráfica
}, editor = {
Joaquim Madeira
and
Jorge Salvador Marques
and
Miguel Salles Dias
and
Joaquim A. Jorge
}, title = {{
Vision-Based Interaction within a Multimodal Framework
}}, author = {
Sá, Vítor
and
Malerczyk, Cornelius
and
Schnaider, Michael
}, year = {
2022
}, publisher = {
The Eurographics Association
}, ISBN = {
978-3-03868-193-9
}, DOI = {
10.2312/pt.20011318
} }
Citation