beasty/infinitune
1
0
Fork 0
Code Issues 1 Pull requests 57 Releases 1 Packages 1 Activity Actions

feat: cover-first radio — LLM album planner, source acquisition, Sources UI #2

Merged
beasty merged 22 commits from feature/radio-cover-first into main 2026-06-13 21:30:29 +00:00
Conversation 0 Commits 22 Files changed 88 +11032 -7112
beasty commented 2026-06-13 01:37:24 +00:00
Owner
Copy link

What

The autonomous radio becomes cover-first: instead of generating all-new music, most album tracks are now ACE cover renders of real popular songs, reimagined into one cohesive album genre.

This branch also carries the groundwork it builds on: the Reimagine page (re-render a song/URL in a new style via ACE cover), ACE generation tuning, provider cleanup (Anthropic/ComfyUI removal, codex consolidation), raw oneshot, and the radio ops console polish.

Album composition (configurable via settings)

Per 12-track album: 8 covers + 1 brand-new + 1 cover-of-cover + 2 random fill, shuffled. Keys: radioCoversPerAlbum, radioNewPerAlbum, radioCoverOfCoverPerAlbum, radioRandomFill, radioSearchRatio (0.75), radioSourceLibraryDir, radioCoverNoiseStrength.

LLM album planner (services/album-planner.ts)

One callLlmObject call designs the album (target genre/era/vibe, band name, title) plus 12 track specs: real popular searchTarget songs for covers, per-cover variation (genre / lyrics / both), full lyrics, captions, vocal casting. normalizeAlbumPlanTracks forces the LLM output back onto the requested slot plan. Planner failure → deterministic template fallback, so the radio never stalls.

Source acquisition (services/cover-source-service.ts)

Priority per cover slot:

  1. Seeded pool (cover_sources table, atomic claim, genre-tag aware)
  2. Online search vs NAS library by configured ratio, each falling back to the other
  3. NAS non-mp3 files transcoded via ffmpeg with hash-keyed caching
  4. Cover-of-cover picks an existing ready cover; falls back when none exist

Worker (worker/song-worker.ts)

sourceUrl (direct URL or ytsearch… query) is resolved to a local file before taking an ACE audio slot — downloads parallelize across songs. Download failure demotes the track to text2music so albums always complete; persistence failures propagate normally.

Downloader hardening (external/youtube-audio.ts)

  • --use-extractors "youtube.*,soundcloud.*,bandcamp.*" — kills the arbitrary-URL SSRF/redirect vector at the source (public-IP DNS pre-check kept as defense in depth)
  • Download caching by target hash (zero-byte poisoning guarded, sidecar meta validated)
  • URL userinfo redaction; yt-dlp errors logged as message+code only (no fetched-host content in logs)

UI

New /autoplayer/sources page: seed URLs (+optional genre tag), queue status (pending/used/failed), NAS dir status incl. scan errors, album mix + ratio + fidelity settings. Nav links on the radio home page.

Data model

  • songs.source_url (additive, mirrored in schema/migrate/test-db/shared types)
  • New cover_sources table

Verified live

Forced an album end-to-end: planner produced "Midnight Firmware Soul" (soft circuit neo soul) with 8 genre-cohesive covers (Sweet Dreams, No Scrubs, Toxic, Passionfruit, Blinding Lights, Running Up That Hill, Fast Car, Levitating); all reference audio downloaded via search through the extractor allowlist, cached, and queued into ACE cover tasks. Planner-failure fallback also observed live (albums complete without covers).

Testing

  • 229 server tests pass (pnpm --filter @infinitune/server test); new cover-first-radio.test.ts covers track-mix composition, plan normalization, source-resolution priority, seeded-pool lifecycle, SSRF guard, search-target sanitation, createRadioAlbum planner→fallback wiring, and demotion persistence
  • LLM client now mocked in radio tests — suite no longer touches the network (26s → 1s)
  • pnpm typecheck and Biome clean (pre-existing warnings unchanged)

Notes

  • POST /api/radio/sources is unauthenticated, matching the existing radio mutation surface (force-generate-album, feedback, requests)
  • A seeded source is marked used at claim time; if its song dies before the audio phase the row stays used with no resolvedAudioPath — reconciliation pass is a possible follow-up
  • Download dir grows unbounded; cache cap/LRU cleanup is a follow-up if disk becomes a concern

Somewhere a GPU is overheating so I don't have to think.

## What The autonomous radio becomes **cover-first**: instead of generating all-new music, most album tracks are now ACE `cover` renders of real popular songs, reimagined into one cohesive album genre. This branch also carries the groundwork it builds on: the Reimagine page (re-render a song/URL in a new style via ACE cover), ACE generation tuning, provider cleanup (Anthropic/ComfyUI removal, codex consolidation), raw oneshot, and the radio ops console polish. ### Album composition (configurable via settings) Per 12-track album: **8 covers + 1 brand-new + 1 cover-of-cover + 2 random fill**, shuffled. Keys: `radioCoversPerAlbum`, `radioNewPerAlbum`, `radioCoverOfCoverPerAlbum`, `radioRandomFill`, `radioSearchRatio` (0.75), `radioSourceLibraryDir`, `radioCoverNoiseStrength`. ### LLM album planner (`services/album-planner.ts`) One `callLlmObject` call designs the album (target genre/era/vibe, band name, title) plus 12 track specs: real popular `searchTarget` songs for covers, per-cover `variation` (genre / lyrics / both), full lyrics, captions, vocal casting. `normalizeAlbumPlanTracks` forces the LLM output back onto the requested slot plan. **Planner failure → deterministic template fallback**, so the radio never stalls. ### Source acquisition (`services/cover-source-service.ts`) Priority per cover slot: 1. **Seeded pool** (`cover_sources` table, atomic claim, genre-tag aware) 2. **Online search** vs **NAS library** by configured ratio, each falling back to the other 3. NAS non-mp3 files transcoded via ffmpeg with hash-keyed caching 4. **Cover-of-cover** picks an existing ready cover; falls back when none exist ### Worker (`worker/song-worker.ts`) `sourceUrl` (direct URL or `ytsearch…` query) is resolved to a local file **before** taking an ACE audio slot — downloads parallelize across songs. Download failure demotes the track to text2music so albums always complete; persistence failures propagate normally. ### Downloader hardening (`external/youtube-audio.ts`) - `--use-extractors "youtube.*,soundcloud.*,bandcamp.*"` — kills the arbitrary-URL SSRF/redirect vector at the source (public-IP DNS pre-check kept as defense in depth) - Download caching by target hash (zero-byte poisoning guarded, sidecar meta validated) - URL userinfo redaction; yt-dlp errors logged as message+code only (no fetched-host content in logs) ### UI New `/autoplayer/sources` page: seed URLs (+optional genre tag), queue status (pending/used/failed), NAS dir status incl. scan errors, album mix + ratio + fidelity settings. Nav links on the radio home page. ### Data model - `songs.source_url` (additive, mirrored in schema/migrate/test-db/shared types) - New `cover_sources` table ## Verified live Forced an album end-to-end: planner produced "Midnight Firmware Soul" (soft circuit neo soul) with 8 genre-cohesive covers (Sweet Dreams, No Scrubs, Toxic, Passionfruit, Blinding Lights, Running Up That Hill, Fast Car, Levitating); all reference audio downloaded via search through the extractor allowlist, cached, and queued into ACE cover tasks. Planner-failure fallback also observed live (albums complete without covers). ## Testing - 229 server tests pass (`pnpm --filter @infinitune/server test`); new `cover-first-radio.test.ts` covers track-mix composition, plan normalization, source-resolution priority, seeded-pool lifecycle, SSRF guard, search-target sanitation, `createRadioAlbum` planner→fallback wiring, and demotion persistence - LLM client now mocked in radio tests — suite no longer touches the network (26s → 1s) - `pnpm typecheck` and Biome clean (pre-existing warnings unchanged) ## Notes - `POST /api/radio/sources` is unauthenticated, matching the existing radio mutation surface (`force-generate-album`, `feedback`, `requests`) - A seeded source is marked `used` at claim time; if its song dies before the audio phase the row stays `used` with no `resolvedAudioPath` — reconciliation pass is a possible follow-up - Download dir grows unbounded; cache cap/LRU cleanup is a follow-up if disk becomes a concern Somewhere a GPU is overheating so I don't have to think. 
Adds radio domain (apps/server/src/radio/), radio routes, and new services
for album generation, radio mixing, requests, and station management.
Reworks worker queues/song-worker for radio-eligible tracks (retry instead
of delete on stale), extends schema/migrations, event bus and WS bridge.

Powered by human calories and mass GPU cycles.
Rewrites queue page as Radio Operations dashboard, reworks autoplayer
routes (library, mini, orchestrator, playlists, settings, house) for the
radio model, strips old oneshot page to a redirect, and updates API
hooks/provider and endpoints for radio queue/library data.

Powered by human calories and mass GPU cycles.
- Upgrade better-sqlite3 to ^12.10.0 (12.6.2 fails to compile against
  Node 26's V8; no prebuilt binary for ABI 147)
- Move build-script allowlist from package.json "pnpm" field to
  pnpm-workspace.yaml allowBuilds (pnpm 11 no longer reads the old key)

Powered by human calories and mass GPU cycles.
OpenAI Codex is now the only text LLM provider. Stored "anthropic"
settings degrade gracefully: normalizeLlmProvider maps legacy values
(anthropic, ollama, openrouter) to openai-codex on every read/write,
so no DB migration is needed. Provider toggles in Settings, the
playlist creator, and the LLM testlab become static Codex labels.

Powered by human calories and mass GPU cycles.
ComfyUI is no longer available. Cover art now dispatches to
inference-sh (default) or codex-imagegen. New normalizeImageProvider
in shared degrades stored legacy values (comfyui, ollama, openrouter)
to inference-sh at every consumer, so existing DBs keep working.
Drops the comfyuiUrl setting, the Network-tab field/connection test,
the bundled ComfyUI workflow JSONs, and the WebSocket client code.

Powered by human calories and mass GPU cycles.
New settings key coversEnabled (default true). When disabled,
SongWorker.startCover() returns before touching the image queue, so
songs complete without cover art and no image-gen credits are spent.
Settings UI gains an ENABLED/DISABLED toggle in the image section
that hides the provider/model controls when off.

Powered by human calories and mass GPU cycles.
ACE_GENERATION_DEFAULTS (inferenceSteps, lmTemperature, lmCfgScale,
inferMethod) in @infinitune/shared/ace-settings replaces the literals
duplicated across server/web ACE clients, the settings page defaults,
and the AudioEngine reset button. Values unchanged in this commit;
tuned values land separately after benchmarking.

Powered by human calories and mass GPU cycles.
New POST /api/songs/oneshot-raw creates an oneshot playlist and a
metadata_ready song in one call: lyrics and style tags go verbatim
into the ACE payload, no LLM metadata stage runs, and no cover is
generated. The playlist is inserted with emitCreated=false and
announced only after the song row exists, so the worker's oneshot
buffer check never races in an auto-created pending (LLM) song.
aceAutoDuration is pinned false so the user's chosen duration wins.

Rebuilds /autoplayer/oneshot as a minimal raw page (lyrics textarea,
style tags, duration select) with playlist-key URL restore, progress
phases, inline player and download.

Powered by human calories and mass GPU cycles.
Shared broadcast-console primitives: OpsPageHeader (amber accent rail,
scanline texture, status slot) and a Stat tile with tone rail and
tabular numerals, replacing the per-page duplicates. Settings gains a
numbered tab rail, an unsaved-changes badge, and a sticky save bar
that reflects dirty state. Radio Operations gains a live rendering
status pill and segmented two-tone chart bars.

Powered by human calories and mass GPU cycles.
Sets ACE_GENERATION_DEFAULTS to 12 inference steps + ODE. Evidence
from production timing data: the previous stored profile (14 steps,
SDE, ACE thinking on) averaged ~287s per 180s song — 1.6x realtime —
while lighter profiles historically landed at 0.6-1.1x. Turbo models
are step-distilled with a quality plateau around 8-12 steps, so 12
steps keeps the quality headroom while ODE + thinking-off removes the
LM-rewrite overhead. Also updates the stored settings rows (which
override defaults) to the same profile.

A controlled steps-sweep benchmark was blocked by ~133 orphaned jobs
in the ACE queue (see PR notes); re-verify gen time once it drains.

Powered by human calories and mass GPU cycles.
Node's Happy Eyeballs connection path calls the custom lookup with
{ all: true } and expects an array of LookupAddress objects; the old
(err, address, family) reply left the address undefined and every
pinned public-http request (inference.sh cover downloads) failed with
ERR_INVALID_IP_ADDRESS. Answer the array form when all=true.

Powered by human calories and mass GPU cycles.
Adds an Oneshot link (yellow accent) to the header nav and a Zap
quick-action card next to Request/Airing Plan/Inventory, so the raw
oneshot page is reachable without typing the URL.

Powered by human calories and mass GPU cycles.
The worker never subscribed to song.deleted, so deleting a song left
its SongWorker running and its queue items occupying audio/image/llm
slots — after a bulk delete the audio queue was saturated with ghost
entries for songs that no longer existed, starving real submissions.
New handleSongDeleted cancels the worker, drops queue items for the
song (which can exist without a registered worker, e.g. audio tasks
resumed at startup), and stops the song actor.

Powered by human calories and mass GPU cycles.
The homelab ACE build clamps turbo (dmd_gan) inference to 8 steps
(infer_steps 12 was silently reduced), and its logs show a 180s song
costs only ~27s of diffusion at 8 steps — far below realtime. Setting
12 in the UI was misleading; 8 is both the actual ceiling and the
model's designed operating point. Also corrects the doc comment:
the server substitutes acestep-v15-turbo when xl-turbo isn't
installed.

Powered by human calories and mass GPU cycles.
New /autoplayer/reimagine: pick a library song (or paste a YouTube/URL
source), give a target style, and choose fidelity. The server submits
an ACE "cover" task with the reference audio uploaded as multipart
src_audio, so structure and melody follow the original while the style
follows the prompt — unlike oneshot's text2music, which invents a new
melody.

- songs gain aceTaskType / sourceSongId / sourceAudioPath /
  coverNoiseStrength; submitToAce uploads multipart when a source file
  is present
- POST /api/songs/reimagine (library source) and
  /api/songs/reimagine-url (yt-dlp download, max 10 min, pasted lyrics)
- fidelity maps to ACE cover_noise_strength (0=loose, 1=faithful)
- nav + quick-action links on the radio home page

Powered by human calories and mass GPU cycles.
Validates the source URL before invoking yt-dlp: http(s) schemes only,
and the hostname must resolve exclusively to public addresses
(reuses isPrivateIp from public-http — loopback, RFC1918, link-local,
0.0.0.0/8, IPv6 ULA/link-local, v4-mapped). Adds "--" before the URL
so it can never be parsed as a yt-dlp flag, and replaces raw yt-dlp
errors with a generic client message (details go to the log) so probed
hosts' response content can't leak.

Known limits, acceptable for this deployment: the DNS pre-check has an
inherent TOCTOU window and yt-dlp follows redirects; airtight SSRF
defense would need a network-layer egress policy.

Powered by human calories and mass GPU cycles.
The autonomous radio now generates albums that are mostly ACE covers of
real popular songs instead of all-new music:

- LLM album planner (album-planner.ts): one callLlmObject call designs a
  cohesive target genre/era/vibe, band, and 12 track specs (cover / new /
  cover-of-cover) with real searchTarget songs and per-cover variation
  (genre/lyrics/both). Deterministic template fallback keeps the radio
  alive when the planner fails.
- Source acquisition (cover-source-service.ts): seeded cover_sources pool
  first, then online search vs NAS library by configurable ratio; NAS
  files transcoded to mp3 via ffmpeg with caching; cover-of-cover picks
  an existing ready cover song.
- Worker resolves songs.sourceUrl (direct URL or ytsearch query) to a
  local file before taking the ACE audio slot; failed downloads demote
  the track to text2music so albums always complete.
- Downloader hardening: yt-dlp extractor allowlist (youtube/soundcloud/
  bandcamp) kills the arbitrary-URL SSRF vector, download caching by
  target hash, URL userinfo redaction and message-only error logging.
- Sources UI (/autoplayer/sources): seed URLs, queue status, NAS dir +
  album mix + search ratio + cover fidelity settings.

Powered by human calories and mass GPU cycles.
Address findings from the pre-PR review:

- Worker: narrow acquireSourceAudio catch to the download only, so DB
  errors after a successful download propagate instead of being
  misreported as download failures (and the good reference kept)
- Log every designed fallback that was previously silent: NAS scan
  errors (now also surfaced as NasStatus.error + UI warn tone), cover
  slots demoted to "new" when no source is acquirable, cover-of-cover
  candidates whose audio is unreachable
- claimSeededSource: conditional UPDATE makes the claim atomic against
  concurrent album creation
- Download cache: discard zero-byte mp3s, validate sidecar meta, log
  non-ENOENT sidecar read failures with the cache key
- ffmpeg transcode failures now include a stderr tail; NAS paths
  resolved absolute (no leading-dash argv ambiguity)
- Planner failures log the full error object (schema/auth debugging)
- Tests: createRadioAlbum planner→fallback wiring, source-spec wiring,
  seeded-source claim, clearCoverSource/updateSourceAudioPath,
  pickCoverOfCoverSource positive path, normalize edge cases; mock
  llm-client in radio-services tests (no real network calls — suite
  drops from 26s to 1s)

Powered by human calories and mass GPU cycles.
- playlists routes: exclude mode==="radio" from GET /playlists and
  /playlists/current so the hidden global-radio generation playlist
  never surfaces as a user playlist
- radio-station-service: kick off inventory top-up in the background on
  first listener instead of awaiting it, so Play resumes already-ready
  songs immediately and isn't blocked by planner LLM calls; track the
  promise so tests can flush it before teardown
- song-worker: guard startCover() with a coverStarted flag — the
  metadata and audio stages both call it before the async cover job
  sets song.cover, which enqueued duplicate image generations
- radio-ws-handler: track the last effective (client-supplied) listener
  id and deactivate it on socket close, not just the connection-local
  UUID, so disconnects don't leave stale active listeners

Powered by human calories and mass GPU cycles.
- markAlbumReadyIfComplete: only emit radio.album_ready on an actual
  status transition (guard the UPDATE on status != 'ready' and emit only
  when a row changed). Re-emitting for an already-ready album re-entered
  topUpInventory via the album-ready handler → unbounded loop
- startNextSong: when the schedule is empty, clear current_song_id /
  current_play_id too, not just pause. Leaving them set made the
  song-ready handler's "!currentSongId" auto-start guard never fire, so
  listeners stayed stranded on the finished song once new tracks landed
- autoplayer radio socket: re-send "play" on (re)connect when the user
  is still joined, so a transient WS drop (which deactivates the listener
  server-side) doesn't leave the station paused with ignored heartbeats

Powered by human calories and mass GPU cycles.
- topUpInventory: serialize runs through a shared promise chain so
  overlapping triggers (listener activation + album_ready/song-start
  handlers) each see the prior run's results before computing their
  deficit, instead of both enqueueing a full target of albums from the
  same stale inventory read
- radio-ws-handler: reference-count sockets per listener id. Multiple
  tabs share one persisted listenerId; a listener is now deactivated
  only when its last holding socket pauses or closes, so closing one tab
  no longer drops a listener another tab is still playing
- cover-source-service: clamp per-album count settings to the 12-track
  album bound so a corrupt/hostile setting can't make buildTrackTypeMix
  allocate a huge array and block album generation

Powered by human calories and mass GPU cycles.
Audit findings from /plugin-audit:

- radio-station-service: wrap the fire-and-forget topUpInventory() and
  markAlbumFirstPlayed() in startSong with .catch — an LLM planner
  failure on song start must not surface as an unhandled rejection
  (the activateListener path already did this)
- radio-ws-handler: a repeat "pause" on a socket holding nothing no
  longer force-deactivates the (possibly shared) listener id, which
  bypassed the per-socket refcount
- routes/radio: document that HTTP play/pause are a no-refcount
  WebSocket fallback by design
- album-planner: single RADIO_TRACK_TYPES source of truth for the track
  literals (type + random-fill picker + Zod enum), removing drift risk
- schema: type cover_sources.status as a "pending|used|failed" enum so
  it matches the frontend union instead of bare text
- album-generation-service: fold duplicated repair condition, dedup the
  createCount deficit expression, drop a redundant stats alias
- tests: lock in the markAlbumReadyIfComplete loop guard (emits
  radio.album_ready exactly once; no emit when a track isn't ready)

Powered by human calories and mass GPU cycles.
beasty merged commit 5f952efff0 into main 2026-06-13 21:30:29 +00:00
beasty deleted branch feature/radio-cover-first 2026-06-13 21:30:29 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
beasty/infinitune!2
No description provided.