NeuroCodec uses EEG attention signals to guide speech extraction

The NeuroCodec preprint, submitted on October 21, combines EEG attention signals with a neural audio codec to extract a target speaker from overlapping speech. It works in a learned audio representation instead of relying only on a mask over the original sound.

The authors report improvements on two public datasets and in a listening experiment involving 20 people. They also describe a causal version intended to support streaming operation.

The result is a research method for hearing assistance. It is not yet a validated everyday hearing device, where electrode setup, signal delay, changing speakers and performance outside the laboratory all affect whether the approach is useful.