Human communication weaves speech, language, and nonverbal behaviour into a continuous exchange of meaning and emotion. This keynote traces the way from sensing emotion to building multimodal systems that assess human states and traits, with applications from mental health assessment to emotionally responsive artificial companions, and the vision of multimodal communication intelligence that helps improve lives.

Speaker tracking is the task of separating and following multiple speakers over time, addressing overlapped speech, talker identity, and time-varying speaker count. The proposed segment-online framework performs speaker separation, voice activity detection, and a two-stage sequential organization strategy (overlap-based stitching and memory-based verification), achieving state-of-the-art tracking on LibriCSS and AMI.

As synthetic voices become nearly indistinguishable from human speech, no single technique can reliably prevent their misuse. This talk presents complementary passive and proactive defenses—from deepfake detection toward word localization, and from audio watermarking toward self-recovery watermarking—and shares ideas on what the speech community can do together.