View on GitHub

Chaos Hearing

At the intersection of sound physics, auditory cognition, and interface design.

02 — Spectral Cognition

Sound Physics, Musical Imagery, and the Perception of Pitch

← Home

Erick Oduniyi

Motivation

You’re walking down the street and a song starts playing in your head. Not from headphones — from inside. The melody is vivid, the timbre recognizable, the pitch precise. Where is this sound? What is your brain doing?

This project explores the space between the physics of sound and the cognition of hearing — particularly the phenomena that reveal how deeply the brain constructs auditory experience rather than passively receiving it.

Threads

Perfect Pitch (Absolute Pitch)

About 1 in 10,000 people can identify or produce a musical note without a reference tone. This isn’t just good ears — it’s a cognitive labeling system that maps continuous frequency space onto discrete categories. Questions:

Involuntary Musical Imagery (Earworms)

Nearly everyone experiences involuntary musical imagery — songs “stuck” in your head. This is the auditory system running in generative mode without external input. It reveals:

Auditory Hallucinations

In psychosis, the generative capacity of the auditory system goes further — producing voices, music, or sounds that are experienced as fully external and real. This is not a failure of hearing but an excess of the same constructive process that lets us parse cocktail parties and imagine melodies.

The continuum: auditory scene analysis → musical imagery → auditory hallucination — all manifestations of the brain’s predictive model of sound, operating at different levels of constraint.

Stephon Alexander’s The Jazz of Physics (2016) argues that jazz improvisation and theoretical physics share deep structural parallels — both are acts of pattern recognition, constraint satisfaction, and creative exploration within formal systems. This is not loose metaphor. Alexander draws a direct line from John Coltrane’s harmonic substitution patterns (“Coltrane Changes” — the symmetric divisions of the octave that generate Giant Steps) to symmetry principles in cosmology and quantum gravity.

The connection to spectral cognition is concrete:

The improvisation framing connects directly to the computational models in this project: the GP spectral inference framework is a formal version of what a jazz musician does — maintain a probabilistic model of the harmonic context (the prior), update it continuously as new notes arrive (the likelihood), and use it to predict, select, and generate the next phrase (the posterior). The spectral mixture kernel is the chord chart.

Probabilistic Spectral Analysis

From the signal processing side, Wilkinson et al. (2019) show that audio analysis can be formulated as Gaussian process inference with spectral mixture kernels. This is interesting not just as engineering but as a model of perception:

This mirrors what the auditory system does: maintain a probabilistic model of the sound scene, update it continuously, and use it to predict, separate, and interpret incoming signals.

The Bridge

The physics of sound production (nonlinear dynamics, contact mechanics, wave propagation) determines what signals arrive at the ear. The cognition of hearing (scene analysis, pitch perception, imagery) determines what we experience. The bridge between them is spectral cognition — the computational and neural processes that transform physical vibration into perceptual meaning.

Production → Propagation → Transduction → Representation → Perception → Imagery
   ↑                                                                        ↓
   └──────────── Interface Design (closing the loop) ──────────────────────┘

Building interfaces for sound means understanding this entire chain — and finding the right points to give people control.

Open Questions