Carol Espy-Wilson
University of Maryland, MD·Engineering
How a person speaks carries information about their language, their dialect and their health. Speech depends on the precise movement and timing of the lips, jaw, tongue, glottis (the vocal folds) and soft palate.
- Fields
Engineering
Computer Science
Biomedical Engineering
- Topics
speech inversion
articulatory phonetics
speech acoustics
- Grants
- 2 known grants, through Apr 2027
What they work on
- Extend a speech inversion tool, which already estimates tongue and lip movements from audio, to recover nasality and voicing by training neural networks on acoustic features.
- Collect a corpus of American English speech with simultaneous audio, nasal versus oral airflow, electroglottography and articulatory movement recordings to serve as ground truth for training and validation.
- Test whether the trained speech inversion system generalizes to speakers of Canadian French and Russian, whose articulatory habits differ from those of English speakers.
- Develop an API that extracts voice biomarkers from speech audio, using articulatory dynamics, acoustic features and deep learning, with near real-time cloud processing.
- Assess the translation potential of the voice-biomarker tool for medical devices and health applications through first-hand investigation of the healthcare industry ecosystem.
More about the research
Measuring those movements has usually required specialized equipment. These two projects aim to recover articulation and other health-related features from ordinary audio recordings using signal processing and machine learning. One builds a tool that infers articulator movement from sound, an approach called speech inversion. The other assesses how voice biomarkers, vocal features linked to health conditions, could be built into medical devices and health apps.
Why it matters
Healthcare lacks accessible, non-invasive and continuous ways to monitor chronic and emerging conditions. Analyzing voice recordings could support earlier detection and personalized treatment of neurological, respiratory and mental health conditions, for both in-person and remote patients. A tool that reads articulation from audio alone could help clinicians detect speech-timing disruptions in dysarthria and track depression and schizophrenia. It could also improve speech recognition systems and help field researchers document endangered languages.
Known grants
From NIH and NSF research grants. Labs often have other funding we can't see.
Closest labs
Jonathan Amith
Gettysburg College, PA·Computer Science
Build POWSM, a foundation speech model that transcribes any language as sequences of consonants and vowels, and prompt it with vector representations of sound correspondences so it can recognize low-resource varieties as transformed versions of well-resourced ones.
David Harwath
University of Texas at Austin, TX·Computer Science·Early-career
Develop attention-based neural network segmentation layers that discover units in raw speech audio, applied hierarchically to recover structure at multiple levels such as phones, words and phrases.
Andres Valdez
Pennsylvania State University, PA·Biomedical Engineering
Assess the translation potential of a voice-based pre-screening tool for RSV, influenza and COVID-19 through I-Corps interviews and first-hand study of the healthcare industry.
The school
University of Maryland: 49 labs on PI Pages, $1.5B a year on research, 602 research doctorates a year.
Reach out
A start for your email. Fill in the brackets.