CLIP1

ICASSP 2026 Cadenza Challenge: Predicting Lyric Intelligibility (CLIP1)

This page is written as a practical introduction to the first Cadenza Lyric Intelligibility Prediction Challenge (CLIP1): what we built, what we learned, and how others can still use the materials. Although the original challenge concluded at ICASSP 2026 in Barcelona, its dataset, submitted systems, and evaluation results remain openly available for further research.

If you are new to the project, the best way to start is reading the official CLIP1 introduction, which explains all the details of the challenge. This page complements it with a more research-project-oriented summary.

Why this challenge?

In the Cadenza project, we want to improve music for people with hearing loss. Understanding the lyrics is a crucial part of music enjoyment. However, unlike in spoken speech where we have STOI and HASPI, we don’t have a well established metric on lyric intelligibility. Therefore, CLIP1 asked: can we automatically predict how intelligible the lyrics of a song are to a listener?

Understanding lyric intelligibility matters because:

• People with hearing loss often struggle to follow lyrics even when they can hear the music.
• In speech technology, intelligibility metrics have driven major improvements in speech enhancement. The same potential exists for music.
• No reliable metric existed for sung language, and existing speech metrics fail because sung and spoken language differ in rhythm, intonation, and production techniques.

CLIP1 was designed to catalyse this area by providing the first large-scale dataset of song excerpts paired with human intelligibility scores, and by challenging the community to build better prediction models.

What we did?

Participants were given thousands of audio segments of accompanied singing from a diverse range of genres and styles. Their task was to predict the word correct rate, computed as the proportion of words correctly transcribed by normal-hearing listeners in a perceptual test.

The dataset contained audios in three conditions:

• No hearing loss simulation — the original signal.
• Mild hearing loss simulated — the original signal processed to model a mild audiogram.
• Moderate hearing loss simulated — the original processed to model a moderate audiogram.

This design meant that systems had to generalise across diverse hearing characteristics without necessarily requiring knowledge of hearing loss modelling.

The CLIP dataset

The CLIP dataset (Roa-Dabike et al., 2026) is one of the main contribution of this challenge. It consists of 3,700 audio excerpts drawn from the FMA dataset, each paired with:

1. The stereo audio as heard by listeners (with or without hearing loss simulation).
2. The unprocessed audio without hearing loss simulation.
3. The severity of hearing loss applied.
4. Ground-truth lyric transcriptions.
5. Listener transcriptions and the resulting intelligibility score.

)

Baseline system

We provided two baselines:

1. A Whisper-based automatic speech recognition (ASR) approach adapted for sung speech. It applied music source separation to isolate the vocal track, then used Whisper to generate a predicted transcription, which was compared against the ground-truth text to estimate intelligibility.
2. A STOI-based model. This model employed the traditional STOI model and used the separated vocals as reference signal.

These two baseline were intentionally straightforward to give participants a concrete and well-understood starting point.

Results

The challenge attracted 29 submitted systems from teams around the world, with the top teams invited to present at ICASSP 2026 in Barcelona. Systems were evaluated using RMSE between predicted and human intelligibility scores, and Pearson correlation.

Selected results

(* Invited to present at ICASSP 2026. Eval RMSE: lower is better. Eval Corr: higher is better.)

What we learned?

Several participant systems substantially outperformed both baselines. The Whisper-based baseline, despite being a strong ASR system, left considerable room for improvement — confirming that lyric intelligibility prediction is a genuinely hard problem that benefits from purpose-built approaches.

The STOI baseline, adapted from speech intelligibility, performed poorly, validating the core motivation for the challenge: speech metrics do not transfer reliably to music.

The top systems generally made use of ground-truth reference text and modelled hearing loss severity explicitly, suggesting these are important signals for accurate prediction. However, this comes with a trade-off: systems that rely on reference text or hearing loss information are harder to deploy in practice, as such inputs are rarely available in real-world applications. A fully agnostic system - one that requires only the audio - would generalise better across different contexts and use cases. This tension between prediction accuracy and practical applicability remains an open and interesting research question for future challenges.

How to use the materials now?

Even though the competition is over, CLIP1 remains a useful open research resource.

You can get started by:

1. Reading the original challenge docs at cadenzachallenge.org.
2. Downloading the dataset from Zenodo.
3. Reproducing the baselines from PyClarity GitHub and comparing new systems against the published results.
4. Citing the dataset paper (Roa-Dabike et al., 2026) if you build on the data or task.
5. Citing the ICASSP 2026 overview paper (Roa-Dabike et al., 2026)