Research

Generative models make it easy to alter or re-synthesize speech, which raises the question of whether a received recording can be trusted. This ongoing project studies watermarking for speech verification. The sender embeds an identifier into the audio, together with a dynamic, signal-dependent watermark that binds the mark to the content. The receiver decodes the watermark and checks it against a reference identifier, and the audio is rejected on a mismatch, for example after tampering by a generative model.

Work in progress.

Advances in speech synthesis and voice conversion create real risks to personal privacy, which calls for robust speaker anonymization. Existing systems modify hand-crafted features or speaker embeddings and often struggle to produce enough identity variance across generated voices. NouveauVoice is a pseudo-speaker generation framework based on a hierarchical deep variational autoencoder (NVAE), integrated as a standalone plug-in on top of FACodec and CosyVoice2. Evaluated under a VoicePrivacy-style protocol, it exceeds 38% equal error rate against an automatic speaker verification attacker while balancing speaker diversity, intelligibility and emotional expressiveness.

Neural audio codecs have significantly impacted speech and audio tasks, but high-fidelity reconstruction matters less when the consumer is a machine rather than a listener. We introduce an efficient method that compresses and quantizes any intermediate feature representation of an already trained downstream model, combining task-specific loss guidance with residual vector quantization. The resulting tokenizer reaches ultra-low bitrates (below 200 bps) with minimal loss in downstream performance and adapts to different bitrates and model sizes, evaluated on speech recognition and audio classification. Published at IEEE WASPAA 2025.

A systematic review and benchmark of discrete audio tokenizers across speech, music and general audio. We propose a taxonomy of tokenization approaches and evaluate tokenizers on reconstruction, downstream performance and acoustic language modeling, analyzing trade-offs through controlled ablations. A collaboration across more than 20 researchers, published in TMLR.

Speaker-agnostic speech enhancement models are large and not always desirable on resource-constrained devices. Personalized speech enhancement (PSE) focuses on a particular test-time user, which is an easier problem for small models, but personal data is typically scarce. We investigate data augmentation based on neural speech synthesis and show that when the synthetic data is of sufficient quality, the resulting PSE systems outperform the speaker-agnostic baseline while significantly reducing model complexity.

Conventional data feeding pipelines can be suboptimal for low-resource speech recognition. We propose an automated curriculum learning approach that orders training examples based on both the model's training progress and prior knowledge of example difficulty. We introduce compression ratio as a new difficulty measure that scores raw audio across noise conditions. The method improves word error rate by up to 33% relative over the baseline system.

A finite-state morphological analyser for Paraguayan Guaraní, an indigenous language spoken in Paraguay. The analyser uses finite-state transducers to model the morphology of the language, enabling analysis and generation of word forms.

In partnership with Mozilla, Apertium and the Academía de la Lengua Guaraní (Paraguay), we collected audio and text for the open-source Common Voice dataset. The dataset was officially released and currently contains 27 hours of speech from 188 speakers and 5,894 sentences.