Piano Image Duet
A dreamed image that doesn't just filter the piano — it re-composes it. A glowing scan-line reads the AI image as a spectral score, singing new partial harmonics that duet Karel's real piano. Cycle 3 of The Latent Piano Room.
Image-as-spectrogram resynthesis · additive partial bank quantised to detected key · Xenakis UPIC / MetaSynth lineage · Art2Mus arXiv 2602.17599
Auto-starting in ~10s…
Drop your own audio to replace Karel's piano
mp3 · wav · ogg · m4a · flac
Design Notes
The new move (cycle 3): Prior cycles used the dreamed image only to FILTER the piano (color → lowpass cutoff, brightness → reverb wet). Here the image RE-COMPOSES it. A scan-line sweeps the image as a spectrogram: each column's vertical brightness profile is sampled at 24 log-spaced Y positions, each mapped to a frequency in the detected key (diatonic scale degrees across 3 octaves, quantised so every partial is consonant). Brightness → that partial's amplitude. The result is an additive shimmer/choir voice that duets the real piano.
Musical quantisation (stays warm): The partial frequency grid is rebuilt at every phrase boundary using the chromagram's best-fit key (major diatonic: 0,2,4,5,7,9,11 semitones; minor: 0,2,3,5,7,8,10; chromatic fallback: pentatonic). Every partial is IN KEY — never atonal. The image is constrained to sing consonant harmonics.
No-key fallback (complete without API): When FAL_KEY is absent, the route returns 501 and the UI generates a plasma / gradient blob field on canvas from the same musical features. The scanner still sweeps THAT synthesized field — so you hear the generated blobs sing back to the piano. Status: amber “synthesized field (no image key)” vs. emerald “dreaming live” when real images arrive.
Subsystems: audio fetch + CORS-taint check + warm synth fallback (C–Am–F–G7–Cmaj9) · FFT 2048 chromagram + spectral-flux onset + phrase-boundary heuristic · FAL flux/schnell image gen + Ken-Burns crossfade · image-as-spectrogram additive resynthesis (UPIC / MetaSynth / Art2Mus) · synthesized plasma field fallback · scan-line partial-bar HUD.
References: Xenakis UPIC (1977) · MetaSynth (Wenger & Spiegel 1997) · Art2Mus arXiv 2602.17599 (Feb 2026) · Bello et al. 2005 onset detection · Russell 1980 circumplex · Refik Anadol (Machine Hallucinations) · Memo Akten (Learning to See)