Multimodal Interaction

objective(s)

At the end of this course students should be empowered to:

• Be able to use and combine various modalities, in particular speech, gestures and touch;
• Understand current challenges and solutions in their design;
• Design and implement multimodal interfaces for applications.

content(s)

1. Fundamentals:

     1.Modalities, Terminology

      2.Architectures     

      3.Brief State-of-the-art
      4.Usability, AMITUDE
 2.Modalities:
      1.Speech (Recognition and Synthesis, Understanding, Generation)
      2.Touch
      3.Gestures
      4.Others
3.Fusion e fission
4.Dialogue
5.Tools and Resources:
      1.Agents
      2.Service Oriented Architectures (SOA)
      3.W3C languages (ex: EMMA)
      4.HephaisTK
      5.RavenClaw, Olympus
6.Case studies (ex: AT&T  How May I Help You,  SIRI, Living Usability Lab Telerehabilitation)
7.Mini Projects