jawhng/hear-on-out
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Optimizations, increased variability

Browse Source
This commit is contained in:
Georgi Durankiev
2026-03-02 13:42:14 +02:00
parent 7aacb29dbd
commit 4fbe4eb23e
6 changed files with 204 additions and 179 deletions
Download Patch File Download Diff File Expand all files Collapse all files

178
README.md
View File

@@ -1,156 +1,75 @@
# hear-on-out
A generative ambient music application that transforms real-time weather and air quality data into dynamic soundscapes.
A generative ambient music application that transforms real-time weather and air quality data into dynamic, evolving soundscapes.
## Overview
hear-on-out uses your geolocation to fetch current weather and air quality data, then generates unique ambient music that responds to environmental conditions. Audio parameters are calculated from weather conditions at page load and remain static during playback - refresh the page to get updated soundscapes based on new weather data.
hear-on-out uses your geolocation to fetch current weather and air quality data, then generates ambient music that responds to environmental conditions. Audio parameters are derived from weather at page load and the soundscape evolves continuously — refresh to respond to updated weather data.
## Features
### Weather-Reactive Generative Music
**Chord Progressions**
- 6 distinct mood-based progressions (bright, dreamy, melancholic, tense, warm, ethereal)
- Intelligent selection based on weather comfort scoring
- Smooth transitions between progressions
6 mood-based chord progressions (bright, dreamy, melancholic, tense, warm, ethereal) selected by a comfort score across temperature, humidity, cloud cover, wind, and precipitation. Chord order is randomised on each playback start.
**Instruments**
- **Ambient Pad**: Lush detuned fat oscillator (3 voices, 30 cent spread) with day/night reactive envelopes for rich, warm soundscapes
- **Main Arpeggio**: Temperature-reactive note patterns with variable speed (1n → 8n) and volume, bypasses filter for clarity
- **Ping Arpeggio**: High-pitched reverse arpeggios that intensify with weather extremity, speed scales with temperature (1n → 16n)
- **Bass Synth**: Root note foundation with randomized release times for subtle variation
**Weather-Mapped Audio Parameters**
- **Ambient Pad**: Cloud-reactive presence and detune. Clear sky = loud, tight (10 cents). Overcast = quiet, diffuse (60 cents). Envelope varies by day/night.
- **Main Arpeggio**: Temperature-reactive speed and volume. Pattern shape varies by condition: ascending, descending (hot), sparse (cold), randomised (stormy), probabilistic (foggy). Fires in irregular phrase bursts and silences; occasionally fades out entirely for longer sectional breathing.
- **Ping**: High-register melodic voice wandering freely through the progression's scale tones. Fires in irregular bursts and silences, independent of BPM. Denser in hot/extreme weather, sparser in cold. Per-step volume variation (±6 dB).
- **Bass**: Root note two octaves below the chord. Occasional skipped beats for rhythmic breathing.
Audio parameters are calculated from weather data when the page loads and set once when playback starts:
**Weather → Audio**
- **BPM**: Scales linearly with temperature (5 BPM at 0°C → 30 BPM at 30°C)
- **Reverb**: Wet signal responds to humidity (more humid = more reverb, 0.3-1.0 range)
- **Delay**: Time responds to wind speed (0.1-0.8 seconds), feedback increases with wind (0.2-0.7 range)
- **Filter**: Applied only to ambient pad, cutoff frequency brightens with temperature (0°C = 400Hz dark, 30°C = 8000Hz bright)
- **Resonance**: Increases with wind speed for sharper filtering
- **Arpeggio Speed**: Temperature affects note intervals (cold = slow, hot = fast)
- **Ping Intensity**: Volume scales with weather extremity (pleasant = quiet, extreme = loud)
| Parameter | Mapping |
|---|---|
| BPM | Temperature (5 at 0°C → 30 at 30°C) |
| Reverb wet | Humidity (0.3–1.0) |
| Delay time / feedback | Wind speed |
| Filter cutoff | Temperature — pad brightness (400 Hz → 8 kHz) |
| Filter resonance | Wind speed |
| Pad volume | Cloud cover (clear = −8 dB, overcast = −22 dB) |
| Pad detune spread | Cloud cover (clear = 10 cents, overcast = 60 cents) |
**Continuous Evolution**
- **Chord shuffle**: Progression order randomised each playback
- **Parameter drift**: Filter and reverb breathe around weather baselines on a 20s cycle
- **Arp burst/silence**: Phrase-level gaps in the arpeggio, wall-clock timed, temperature-scaled
- **Arp dropout**: Occasional full-section fade-out and return (15% drop / 75% recover per 25s)
- **Ping burst/silence**: Irregular firing bursts and silences, wall-clock timed
### Air Quality Monitor
- Geiger-counter style noise bursts that trigger based on pollution levels
- Higher pollution = more frequent bursts (6s intervals for clean air → 0.3s for heavy pollution)
- Spacious reverb (30s decay) for atmospheric depth
- Monitors PM2.5, PM10, and dust particles
Geiger-counter style noise that scales with PM2.5, PM10, and dust levels — 6s intervals for clean air, 0.3s for heavy pollution.
### Audio Visualization
- Real-time particle system using p5.js
- **Desktop**: 40 particles at 30fps with connection lines
- **Mobile**: 15 particles at 20fps, no connections (performance optimized)
- FFT analysis for bass, mid, and treble frequency detection
- Particles react to audio levels with dynamic size and movement
- Adaptive sizing: 400x400px (desktop) or 280x280px (mobile)
- Automatic pause during scrolling to prevent audio interference on mobile
Real-time FFT-reactive particle system (p5.js). Desktop: 40 particles at 30fps with connection lines. Mobile: 15 particles at 20fps, no connections.
## Technologies
- **SvelteKit** - Web framework with Svelte 5 runes
- **Tone.js** - Web Audio API wrapper for synthesis and effects
- **p5.js** - Creative coding for visualizations
- **Tailwind CSS v4** - Styling with black/white monochrome theme
- **Open-Meteo API** - Weather and air quality data
## Performance Optimizations
The application includes several optimizations for smooth performance:
**Desktop Optimizations:**
- 40 particles with 30fps rendering
- Audio-reactive displacement updates every 3rd frame
- Particle connections drawn every other frame with limited neighbor checking
- Tab visibility detection: Visualization pauses when tab is not focused
- Efficient audio routing: Arpeggios bypass filter for cleaner CPU usage
**Mobile-Specific Optimizations:**
- **Reduced particle count**: 15 particles (vs 40 on desktop)
- **Lower framerate**: 20fps (vs 30fps on desktop)
- **Disabled connection lines**: Removes expensive O(n²) calculations
- **Smaller canvas**: 280x280px visualizations (vs 400x400px on desktop)
- **Scroll handling**: Visualization pauses during scrolling to prevent audio stuttering
- **Less frequent updates**: Displacement updates every 5th frame (vs 3rd on desktop)
- **Screen Wake Lock API**: Prevents screen sleep during audio playback
- **Screen sleep recovery**: Audio context automatically resumes after device wakes up
- **Visibility API**: Properly handles audio context suspension/resumption
**Mobile Performance Note:**
Running both generators simultaneously on mobile devices may cause performance issues. For optimal experience on mobile, consider running one generator at a time.
- **SvelteKit** / Svelte 5 runes
- **Tone.js** — synthesis and effects
- **p5.js** — visualisation
- **Tailwind CSS v4**
- **Open-Meteo API** — weather and air quality
## Setup
```bash
# Install dependencies
npm install
# Run development server
npm run dev
# Build for production
npm run build
npm run dev # development server
npm run build # production build
```
## How It Works
## Geolocation
1. **Location Input**: User can either:
- Grant browser geolocation permissions (automatic location detection)
- Manually enter latitude/longitude coordinates
2. **Data Fetch**: Current weather and air quality data retrieved from Open-Meteo API
3. **Weather Comfort Scoring**: Algorithm calculates comfort based on temperature, humidity, cloud cover, wind, and precipitation
4. **Progression Selection**: Chooses appropriate chord progression based on weather mood
5. **Parameter Calculation**: Audio parameters (BPM, reverb, delay, filter, etc.) are derived from weather conditions
6. **Audio Generation**: When playback starts, all parameters are set and remain static until page refresh
7. **Static Playback**: Audio continues with the same settings - refresh the page to update based on new weather data
Requires HTTPS in production (localhost works without). Falls back to manual coordinate entry. Development server uses `@vitejs/plugin-basic-ssl`.
## Geolocation & Browser Compatibility
## Mobile
The app supports two methods for location input:
**Automatic Geolocation:**
- Requires HTTPS in production (localhost works without HTTPS)
- User must grant browser permission when prompted
- May not work on some mobile browsers or restrictive browser settings
**Manual Coordinate Entry:**
- Fallback option for when geolocation is unavailable
- Accepts latitude (-90 to 90) and longitude (-180 to 180)
- Works on all devices and browsers
- Useful for exploring weather from different locations
**Production Requirements:**
- Must be served over HTTPS for geolocation to work
- Development server uses `@vitejs/plugin-basic-ssl` for local HTTPS
## Mobile Experience
The application is optimized for mobile devices with several performance enhancements:
**Audio Playback:**
- **Screen Wake Lock**: Prevents screen from sleeping while audio is playing
- Automatically requested when playback starts
- Released when playback stops or app is closed
- Supported on Chrome/Edge 84+, Safari 16.4+, Opera 70+
- Gracefully degrades on unsupported browsers (Firefox)
- Automatic audio context recovery after screen sleep/wake
- Handles browser audio suspension gracefully
- Transport automatically restarts when app becomes visible
**Visual Performance:**
- Adaptive particle count and framerate based on device
- Visualization pauses during scrolling to prevent stuttering
- Smaller canvas size for reduced GPU load
**Known Limitations:**
- Running both generators simultaneously may impact performance on lower-end devices
- Some mobile browsers may require user interaction to start audio playback
- Audio may briefly pause during heavy scrolling or multitasking
- Wake Lock not supported on Firefox (audio may stop when screen locks)
Screen Wake Lock prevents sleep during playback (Chrome/Edge 84+, Safari 16.4+). Audio context recovers automatically after screen wake. Firefox does not support Wake Lock.
## Project Structure
@@ -158,31 +77,20 @@ The application is optimized for mobile devices with several performance enhance
src/
├── lib/
│ ├── audio/
│ │ ├── instruments/ # Modular synth instruments
│ │ ├── audio-effects.ts # Shared audio effects
│ │ ├── instruments/
│ │ ├── audio-effects.ts
│ │ ├── chord-progressions.ts
│ │ └── weather-mood.ts # Comfort scoring & progression selection
│ │ └── weather-mood.ts
│ ├── components/
│ │ └── AudioVisualization.svelte
│ └── generators/
│ ├── weather/WeatherGen.svelte
│ └── air-quality/AirQualityGen.svelte
└── routes/
├── +page.svelte # Geolocation entry point
└── on-out/+page.svelte # Audio generators
├── +page.svelte
└── on-out/+page.svelte
```
## Weather Mood Mapping
| Conditions | Progression | Characteristics |
|------------|-------------|-----------------|
| Stormy/Extreme | Tense | Dissonant, unsettling |
| Very Hot (30°C+) | Warm | Energetic, bright |
| Cold/Rainy | Melancholic | Somber, introspective |
| Foggy/Misty | Ethereal | Spacious, mysterious |
| Pleasant Day | Bright | Uplifting, major tonality |
| Pleasant Night | Dreamy | Calm, flowing |
## License
MIT

2
src/lib/audio/audio-effects.ts
View File

@@ -2,7 +2,7 @@ import * as Tone from 'tone';
export function createReverb(wetValue: number): Tone.Reverb {
const reverb = new Tone.Reverb({
decay: 16,
decay: 4,
preDelay: 0.5
});
reverb.wet.value = wetValue;

2
src/lib/audio/instruments/padSynth.ts
View File

@@ -4,7 +4,7 @@ export function createPadSynth(isDay: boolean): Tone.PolySynth {
return new Tone.PolySynth(Tone.Synth, {
oscillator: {
type: 'fatsine', // Fat oscillator creates multiple detuned voices for lush sound
count: 3, // Number of detuned oscillators
count: 2, // Number of detuned oscillators
spread: 30 // Amount of detune in cents for width and movement
},
envelope: {

2
src/lib/components/AudioVisualization.svelte
View File

@@ -130,7 +130,7 @@
}
p.background(0, 30); // Fade effect
if (isPlaying && analyser) {
if (isPlaying && analyser && p.frameCount % 2 === 0) {
// Get FFT data (frequency analysis)
audioData = analyser.getValue() as Float32Array;

2
src/lib/generators/air-quality/AirQualityGen.svelte
View File

@@ -114,7 +114,7 @@
// Create effects with much more spacious reverb for air quality
reverb = new Tone.Reverb({
decay: 30, // Very long decay for spacious sound
decay: 5, // Reduced from 30 for better performance
preDelay: 0.1
});
reverb.wet.value = 0.8; // Higher wet signal for more reverb

197
src/lib/generators/weather/WeatherGen.svelte
View File

@@ -46,7 +46,7 @@
let pingSynth: Tone.Synth | null = null;
let bassSynth: Tone.Synth | null = null;
let arpSequence: Tone.Sequence | null = null;
let pingSequence: Tone.Sequence | null = null;
let pingSequence: Tone.Loop | null = null;
let bassSequence: Tone.Sequence | null = null;
let reverb: Tone.Reverb | null = null;
let delay: Tone.FeedbackDelay | null = null;
@@ -54,6 +54,11 @@
let sequence: Tone.Sequence | null = null;
let gain: Tone.Gain | null = null;
let analyser: Tone.Analyser | null = null;
let driftLoop: Tone.Loop | null = null;
let dropoutLoop: Tone.Loop | null = null;
let arpActive = true;
let pingActive = true;
let bassActive = true;
// Select chord progression based on weather mood
const currentProgression: ChordProgression = $derived.by(() => {
@@ -142,6 +147,27 @@
return 1 - comfortScore; // Invert: higher = more extreme
});
// Pad volume: louder/present in clear sky, quieter/receded in overcast
const padVolume = $derived.by(() => {
const clouds = cloudCover ?? 50;
return -8 - (Math.min(100, Math.max(0, clouds)) / 100) * 14;
});
// Pad spread: tight detune in clear sky, wide/diffuse in overcast
const padSpread = $derived.by(() => {
const clouds = cloudCover ?? 50;
return 10 + (Math.min(100, Math.max(0, clouds)) / 100) * 50;
});
// Arp pattern shape: changes based on weather conditions
const arpPatternType = $derived.by(() => {
if (precipitation > 5 || weatherExtremity > 0.8) return 'random';
if (temperature2m < 5) return 'sparse';
if (relativeHumidity2m > 80 && cloudCover > 60) return 'probabilistic';
if (temperature2m > 25) return 'descending';
return 'ascending';
});
// Ping volume: quieter in pleasant weather, louder in extreme weather
const pingVolume = $derived.by(() => {
// Map extremity: 0 (pleasant) = -22dB (present), 1 (extreme) = -8dB (prominent)
@@ -239,6 +265,9 @@
sequence = null;
}
// Shuffle chord order — different harmonic arrangement each playback
const shuffledProgression = [...currentProgression].sort(() => Math.random() - 0.5);
// Set all audio parameters once (static until page refresh)
Tone.getTransport().bpm.value = bpm;
@@ -268,13 +297,18 @@
pingSynth.volume.value = pingVolume;
}
if (synth) {
synth.volume.value = padVolume;
synth.set({ oscillator: { spread: padSpread } });
}
sequence = new Tone.Sequence(
(time: number, chord) => {
if (synth && chord) {
synth!.triggerAttackRelease(chord.notes, '4n', time);
}
},
currentProgression,
shuffledProgression,
'4n'
);
@@ -286,56 +320,103 @@
arpSequence = null;
}
// Arp burst/silence — phrase-level gaps independent of BPM
const arpT = temperature2m ?? 20;
const arpBurstMax = arpT <= 10 ? 6 : arpT <= 20 ? 8 : 12; // seconds
const arpSilenceMax = arpT <= 10 ? 5 : arpT <= 20 ? 4 : 3; // seconds
let inArpBurst = true;
let arpStateUntil = Tone.now() + Math.random() * arpBurstMax + 2;
arpSequence = new Tone.Sequence(
(time: number, chord) => {
const now = Tone.now();
if (now >= arpStateUntil) {
inArpBurst = !inArpBurst;
arpStateUntil = now + (inArpBurst
? Math.random() * arpBurstMax + 2
: Math.random() * arpSilenceMax + 1);
}
if (!inArpBurst) return;
if (arpSynth && chord && chord.notes) {
// Capture synth reference for TypeScript
const synth = arpSynth;
// Calculate time between notes based on 16th notes
const sixteenthNote = Tone.Time('16n').toSeconds();
// Play arpeggio pattern through the chord notes
chord.notes.forEach((note: string, index: number) => {
let notes = [...chord.notes];
if (arpPatternType === 'sparse') {
// Root (index 0) and fifth (index 2) only — hollow, minimal
notes = [chord.notes[0], chord.notes[2]].filter(Boolean);
} else if (arpPatternType === 'descending') {
notes = [...chord.notes].reverse();
} else if (arpPatternType === 'random') {
// Fisher-Yates shuffle — chaotic, unsettled
for (let i = notes.length - 1; i > 0; i--) {
const j = Math.floor(Math.random() * (i + 1));
[notes[i], notes[j]] = [notes[j], notes[i]];
}
}
notes.forEach((note: string, index: number) => {
if (arpPatternType === 'probabilistic' && Math.random() > 0.5) return;
synth.triggerAttackRelease(note, '16n', time + index * sixteenthNote);
});
}
},
currentProgression,
shuffledProgression,
arpInterval
);
arpSequence.start(0);
// Create ping sequence (reverse arpeggio - evenly spaced through chord duration)
// Create ping — free melodic random walk over progression's scale tones
if (pingSequence) {
pingSequence.dispose();
pingSequence = null;
}
// Build a flat array of notes: 4 notes per chord, in reverse order, transposed up 2 octaves
const pingNotes: string[] = [];
// Build sorted note pool from all unique pitch classes in the progression
const notePool: string[] = [];
const seenNames = new Set<string>();
currentProgression.forEach((chord) => {
if (chord && chord.notes) {
// Get 4 notes in reverse order (last to first, cycling if needed)
const reversedNotes = [...chord.notes].reverse();
for (let i = 0; i < 4; i++) {
const note = reversedNotes[i % reversedNotes.length];
if (chord?.notes) {
chord.notes.forEach((note: string) => {
const noteName = note.slice(0, -1);
const octave = parseInt(note.slice(-1));
const highNote = noteName + (octave + 2);
pingNotes.push(highNote);
}
if (!seenNames.has(noteName)) {
seenNames.add(noteName);
if (octave + 1 <= 6) notePool.push(noteName + (octave + 1));
if (octave + 2 <= 6) notePool.push(noteName + (octave + 2));
}
});
}
});
// Captured once at sequence start — same pattern as all other params
// Burst/silence state tracked via Tone.now() inside the callback itself —
// completely independent of pingInterval and BPM.
const temp = temperature2m ?? 20;
const burstMax = temp <= 10 ? 5 : temp <= 20 ? 7 : 10; // seconds
const silenceMax = temp <= 10 ? 4 : temp <= 20 ? 3 : 2; // seconds — shorter for more presence
pingSequence = new Tone.Sequence(
(time: number, note: string) => {
if (pingSynth && note) {
pingSynth.triggerAttackRelease(note, '32n', time);
}
},
pingNotes,
pingInterval // Temperature-reactive: 1n (cold) to 16n (hot)
);
let inBurst = Math.random() < 0.5;
let stateUntil = Tone.now() + (inBurst
? Math.random() * burstMax + 1
: Math.random() * silenceMax + 1);
pingSequence = new Tone.Loop((time: number) => {
const now = Tone.now();
if (now >= stateUntil) {
inBurst = !inBurst;
stateUntil = now + (inBurst
? Math.random() * burstMax + 1
: Math.random() * silenceMax + 1);
}
if (!inBurst) return;
if (pingSynth && notePool.length > 0) {
const note = notePool[Math.floor(Math.random() * notePool.length)];
pingSynth.volume.value = pingVolume + (Math.random() * 12 - 6);
pingSynth.triggerAttackRelease(note, '32n', time);
}
}, pingInterval);
pingSequence.start(0);
@@ -347,27 +428,58 @@
bassSequence = new Tone.Sequence(
(time: number, chord) => {
if (bassSynth && chord && chord.notes && chord.notes.length > 0) {
// Get root note (first note of chord) and transpose down 2 octaves
const rootNote = chord.notes[0];
const noteName = rootNote.slice(0, -1); // e.g., 'C' from 'C4'
const octave = parseInt(rootNote.slice(-1)); // e.g., 4 from 'C4'
const bassNote = noteName + (octave - 2); // e.g., 'C2'
if (!bassSynth || !chord?.notes?.length) return;
// Randomize note duration: half to full chord duration (2n to 4n)
const randomDuration = 0.5 + Math.random() * 0.5;
const noteDuration = Tone.Time('4n').toSeconds() * randomDuration;
// 20% chance to skip — rhythmic breathing without harmonic risk
if (Math.random() < 0.2) return;
// Trigger bass note at the scheduled time
bassSynth.triggerAttackRelease(bassNote, noteDuration, time);
}
const noteName = chord.notes[0].slice(0, -1);
const octave = parseInt(chord.notes[0].slice(-1));
const bassNote = noteName + (octave - 2);
// Random duration: 50–100% of a quarter note
const noteDuration = Tone.Time('4n').toSeconds() * (0.5 + Math.random() * 0.5);
bassSynth.triggerAttackRelease(bassNote, noteDuration, time);
},
currentProgression,
shuffledProgression,
'4n'
);
bassSequence.start(0);
// Slow parameter drift — filter and reverb breathe around weather baselines
if (driftLoop) { driftLoop.dispose(); driftLoop = null; }
driftLoop = new Tone.Loop(() => {
const filterRange = filterCutoff * 0.4;
const newFreq = filterCutoff + (Math.random() * 2 - 1) * filterRange;
filter?.frequency.rampTo(Math.max(200, Math.min(12000, newFreq)), 20);
const newWet = reverbWet + (Math.random() * 2 - 1) * 0.2;
reverb?.wet.rampTo(Math.max(0.1, Math.min(1.0, newWet)), 25);
}, 20); // every 20 seconds — independent of BPM
driftLoop.start('+10'); // first fire 10 seconds after playback starts
// Arp dropout — asymmetric: low chance to go silent, high chance to recover
// Ping has its own burst/silence mechanism — excluded here to avoid conflict
// Bass stays as harmonic foundation
arpActive = true;
if (dropoutLoop) { dropoutLoop.dispose(); dropoutLoop = null; }
dropoutLoop = new Tone.Loop(() => {
if (arpActive) {
// Only 15% chance to go silent — arp stays on most of the time
if (Math.random() < 0.15) {
arpActive = false;
arpSynth?.volume.rampTo(-80, 2);
}
} else {
// 75% chance to come back — silences are short
if (Math.random() < 0.75) {
arpActive = true;
arpSynth?.volume.rampTo(arpVolume, 1.5);
}
}
}, 25);
dropoutLoop.start('+30'); // wait 30s before first dropout decision
Tone.getTransport().start();
// Request wake lock to prevent screen sleep
@@ -401,6 +513,9 @@
bassSequence.dispose();
bassSequence = null;
}
if (driftLoop) { driftLoop.stop(); driftLoop.dispose(); driftLoop = null; }
if (dropoutLoop) { dropoutLoop.stop(); dropoutLoop.dispose(); dropoutLoop = null; }
arpActive = true;
Tone.getTransport().stop();
Tone.getTransport().cancel();
isPlaying = false;
@@ -492,6 +607,8 @@
if (analyser) {
analyser.dispose();
}
if (driftLoop) { driftLoop.dispose(); }
if (dropoutLoop) { dropoutLoop.dispose(); }
});
</script>