João Freitas
From IEETA
| Subject | Silent Speech for Human-Computer Interface in European Portuguese - MAPi |  |
|---|---|---|
| Advisor | António J. S. Teixeira, Miguel Sales Dias | |
| Group | Signal Processing Laboratory | |
| Transverse activity | Transverse Activity on Innovative Biomedical Technologies | |
| Status | PhD student | |
| Starts | 2010/10/01 | |
| Ends | 2013/09/30 | |
| Country | Portugal | |
| Projects | ||
| Past projects | ||
| Events | Interspeech 2012 | |
| Proposals | ||
| Personal page |
Objectives
This thesis aims at investigating and developing a successful and natural SSI HCI modality which targets all users (universal HCI) including elderly, for European Portuguese (EP). This SSI will be then used as a HCI modality to interact with computing systems and smartphones. The research will be specifically interested in studying the benefit brought by this novel HCI modality for EP elderly speakers, for whom speaking might require a substantial effort. This research will adopt the following approaches:
a. Multi-sensor Analysis
An SSI can be implemented using several types of sensors or a combination of them in order to achieve better results. For this thesis we will preferably adopt the less invasive approaches and sensors that are able to work both in silent and noisy environments such as, video, NAM microphones, sEMG or Ultra-Sound, following recommendations found in the literature, regarding multi-sensor devices. Further investigation will also be conducted on feature extraction, data acquisition and silent speech processing (silent speech recognition with Hidden Markov Models or other machine learning techniques), of data collected from these sensors, as well as on combining techniques through multi-sensor devices and data fusion in order to complement and overcome the inherent shortcomings of some devices without decreasing the usability.
b. European Portuguese Adoption
The existing SSI research projects do not contemplate any languages other than English and Japanese. With this work we will address the challenges of developing an SSI for European Portuguese, the first approach for this language in the Portuguese and international academia. One of the areas of research to address is the problem of recognizing nasal sounds.
c. Targeting Universal Interface and Elderly Speakers
After determining the different possibilities for each type of SSI, a hybrid and minimally invasive solution will be envisioned, specified, developed and tested, including existing hardware components and new software solutions, targeting a universal interface including elderly people. The specific limitations and requirements imposed by an elderly speaker need to be stipulated based on a pre-defined user profile in order to provide an efficient use of the interface.
d. User Requirements and Usability Evaluation
During the full span of the project duration, close contact with end-users, including elderly, will be sought, starting from user requirements capture to the adoption of a full usability evaluation methodology, which will collect feedback and draw conclusions based on real subjects while interacting (using SSI) with computing systems and smartphones, respectively, in real case indoor home scenarios and in mobility environments.
Publications
Chapters in books
- João Freitas, António J. S. Teixeira, Miguel Sales Dias, Carlos A C Bastos. Towards a Multimodal Silent Speech Interface for European Portuguese. Speech Technologies, INTECH, vol. Book 1, 2011
Articles in proceedings of international conferences
- João Freitas, Miguel Sales Dias, António J. S. Teixeira. Towards a Silent Speech Interface for Portuguese: Surface Electromyography and the nasality challenge. International Conference on Bio-inspired Systems and Signal Processing (BIOSIGNALS 2012), Vilamoura, Portugal, February 2012
- Ana Júdice, João Freitas, Daniela Braga, António Calado, Miguel Sales Dias, António J. S. Teixeira, Catarina Oliveira. Elderly Speech Collection for Speech Recognition Based on Crowd Sourcing. Int. Conf. on Software Development for Enhancing Accessibility and Fighting Info-exclusion (DSAI), SAE, Littlemore Park, Armstrong Road, Oxford, OX4 4FY Oxford, UK, 2010