Title: Speech-Based Real-Time Presentation Tracking Using Semantic Matching

Speaker: Reza Asadi, PhD candidate, College of Computer and Information Science, Northeastern University

Location: Northeastern University, 440 Huntington Avenue, West Village H, 3rd Floor, Room #366, Boston, Massachusetts 02115
Abstract
Many commercial and research products have been developed to provide support for oral presentations. However, little work has been conducted to provide real-time tracking of a speaker’s presentation relative to their supporting media. Given the content of presentation slides and speaking notes, a presentation tracking system can track the content coverage by the speaker. This can help speakers ensure that they cover their planned content, and enable various real-time presentation support technologies, such as automatic content recommendation during presentation delivery. Presentation tracking is, however, a complex task due to the inaccuracy of current speech recognition systems and the fact that speakers rarely follow their presentation notes exactly.

In this dissertation, I present a novel framework for real-time tracking of presentations at the sub-slide level, as well as global presentation tracking through a slide deck using automatic speech recognition and semantic text retrieval techniques. To evaluate the effectiveness of presentation tracking in presentation assistance applications, I integrated the tracking framework into three different systems:

IntelliPrompter, a speech-based note display system that dynamically adjusts the note display interface to highlight the most likely next topic to present.
Quester, a system that enables fast access to relevant presentation content during a question answering session and supports nonlinear presentations led by the speaker.
RoboCOP, a robot that acts as a public speaking coach to provide spoken feedback during presentation rehearsals.

Results of the evaluation studies showed that utilizing the presentation tracking during rehearsal and delivery of presentations can result in significant improvements in presenters' experience and performance during presentations.
About the Speaker
Reza Asadi is a PhD student in Computer Science program at Northeastern University. His research is on using automatic speech recognition and natural language processing to provide automatic assistance during rehearsal and delivery of oral presentations. He received a MSc degree in Computer Science from Northeastern and a BSc degree in Mechanical Engineering from Sharif University of Technology.
Committee

Harriet Fell
Timothy Bickmore
Lu Wang
Darren Edge (Microsoft ResearchCambridge)