Note: Your custom schedule will not be saved unless you create a new account or login to an existing account.
| Paper: | MLSP-P3.8 |
| Session: | Speech and Audio Processing |
| Time: | Wednesday, May 19, 15:30 - 17:30 |
| Presentation: |
Poster |
| Topic: |
Machine Learning for Signal Processing: Speech and Audio Processing Applications |
| Title: |
AUDIO-VISUAL GRAPHICAL MODELS FOR SPEECH PROCESSING |
| Authors: |
John Hershey; University of California, San Diego | | |
| | Hagai Attias; Microsoft Research | | |
| | Nebojša Jojic; Microsoft Research | | |
| | Trausti Kristjansson; Microsoft Research | | |
| Abstract: |
Perceiving sounds in a noisy environment is a challenging problem. Visual lip-reading can provide relevant information but is also challenging because lips are moving and a tracker must deal with a variety of conditions. Typically audio-visual systems have been assembled from individually engineered modules. We propose to fuse audio and video in a probabilistic generative model that implements cross-model self-supervised learning, enabling adaptation to audio-visual data. The video model features a Gaussian mixture model embedded in a linear subspace of a sprite which translates in he video. The system can learn to detect and enhance speech in noise given only a short sequence of audio-visual data. We show some results for speech enhancement, and discuss extensions to the model that are under investigation. |
| |
| Back | |