People can simultaneously identify the pitch and timing of a sound signal much more precisely than allowed by conventional linear analysis. That is the conclusion of a study of human subjects done by physicists in the US. The findings are not just of theoretical interest but could potentially lead to better software for speech recognition and sonar.

Human hearing is remarkably good at isolating sounds, allowing us to pick out individual voices in a crowded room, for example. However, the neural algorithms that our brains use to analyse sound are still not properly understood. Most researchers had assumed that the brain decomposes the signals and treats them as the sum of their parts – a process that can be likened to Fourier analysis, which decomposes an arbitrary waveform into pure sine waves.

However, the information available from Fourier analysis is bound by an uncertainty relation called the Gabor limit. This says that you cannot know the timing of a sound and its frequency – or pitch – beyond a certain degree of accuracy. The more accurate the measurement of the timing of a sound, the less accurate the measurement of its pitch and vice versa.

Getting around Gabor

Unlike the Heisenberg uncertainty principle, the Gabor limit is not an intrinsic property of the signal but is a result of the method used to analyse it. If you can find a way to analyse a complex waveform without decomposing it into sine waves, you can in theory track the frequency at a particular time to much greater accuracy. However, whatever analytical technique you choose must be nonlinear because any technique that represents the waveform as a sum of simpler waveforms will be bound by the Gabor limit.

Researchers such as Brian Moore at the University of Cambridge first showed, in the 1970s, that the human auditory system could beat the Gabor limit, implying the brain could perform some kind of nonlinear analysis of the signals that it received from the ear. However, this work was not picked up by the broader scientific community, partly because cochlear processes were not then understood.

Pitch and timing

In this latest study, Jacob Oppenheim and Marcelo Magnasco of Rockefeller University gave volunteers a series of tasks in order to determine precisely how sensitive humans are to the pitch and timing of sounds. One test involved playing two notes widely spaced in time but at the same pitch. In-between the two, they were played a third note, and they were asked to identify whether it was slightly higher or slightly lower than the other two. In another, the subjects were played two notes widely spaced in pitch almost simultaneously: they were then asked whether the higher or the lower one had been played first.

The final test combined the first two tasks: a low note was played followed by a high note. At almost the same time as the high note was played, a third note was played at almost the same pitch as the low note, and the volunteers were asked whether it was pitched above or below the low note and whether it was before or after the high note.

Oppenheim and Magnasco discovered that the accuracy with which the volunteers determined pitch and timing simultaneously was usually much better, on average, than the Gabor limit. In one case, subjects beat the Gabor limit for the product of frequency and time uncertainty by a factor of 50, clearly implying their brains were using a nonlinear algorithm.

Highly nonlinear hearing

“In signal processing, there are a very large number of time–frequency distributions that have been proposed in order to analyse these signals,” says Magnasco. “The question is, since there are very many different ways in which you can do this and only the highly nonlinear ways can offer performance comparable to what humans do, which one of these is in the same family as what the brain does?”

Mike Lewicki, a computational neuroscientist at Case Western Reserve University in Ohio, says the research is “a nice demonstration that our perceptual system is doing complex things – which, of course, people have always known – but this is a nice quantitative demonstration by which, even at the most basic level, using the most straightforward stimuli, you can demonstrate that the auditory system is doing something quite remarkable”.

The research is published in Physical Review Letters.