Document YuE2 install, weight cleanup, and regression tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Towsty
@@ -0,0 +1,84 @@
|
||||
# YuE2 music engine
|
||||
|
||||
`engine: "yue2"` is standalone [YuE2](https://github.com/multimodal-art-projection/YuE) (`m-a-p/YuE2-3B` + `m-a-p/YuE2-Vae`). It never submits a Comfy graph. `engine: "yue"` remains YuEGP. ACE / ACE 1.5 keep their existing Comfy workflows.
|
||||
|
||||
## Install once (desktop RTX 5080 host)
|
||||
|
||||
```powershell
|
||||
# From the aigen checkout on the GPU PC
|
||||
.\scripts\setup-yue2.ps1 -Uv <path-to-uv.exe>
|
||||
```
|
||||
|
||||
Default checkout: `C:\Users\ianjm\Development\YuE2` (isolated venv, not Comfy Python). Setup does not download model weights; Hugging Face pulls them on the first song.
|
||||
|
||||
Restart the existing host connector (`comfy-host-agent`) after pulling these scripts so `/yue2/jobs` is mounted. Do not restart Coolify or Comfy for this wiring.
|
||||
|
||||
## Env vars
|
||||
|
||||
| Variable | Purpose |
|
||||
| --- | --- |
|
||||
| `YUE2_ROOT` | Standalone YuE checkout (default `C:\Users\ianjm\Development\YuE2`) |
|
||||
| `YUE2_PYTHON` | Isolated worker Python (`YUE2_ROOT\.venv\Scripts\python.exe`) |
|
||||
| `YUE2_JOBS_DIR` | Per-job logs, request.json, intermediates, WAV |
|
||||
| `YUE2_MODEL` | Default `m-a-p/YuE2-3B` |
|
||||
| `YUE2_VAE` | Default `m-a-p/YuE2-Vae` |
|
||||
|
||||
App still uses `COMFY_CONTROL_URL` + token. YuE2 shares the same GPU reservation: Comfy is stopped while a YuE2 job owns the card. One song at a time. No `keep_model_loaded`. OOM is a visible error; the app does not fall back to `yue` or ACE.
|
||||
|
||||
## Call path
|
||||
|
||||
```json
|
||||
{"engine":"yue2","duration":60,"seed":42,"tags":"pop, warm vocals","lyrics":"[Verse 1]\nYour lyrics here","folderId":"<library-folder>","name":"My song"}
|
||||
```
|
||||
|
||||
`startMusicJob` → `startYue2Job` → `POST <COMFY_CONTROL_URL>/yue2/jobs` → `yue2-worker.py` (`plan` → `generate_semantic` → `synthesize` → `decode`) → library WAV.
|
||||
|
||||
Duration clamp is 30–150 seconds (default 60), same as Yue field validation: one non-empty lyric section. Upstream song length follows the plan; duration is validated and stored for library metadata.
|
||||
|
||||
## Success check
|
||||
|
||||
1. Music page Model picker shows YuE2.
|
||||
2. POST with `engine:"yue2"` hits `/yue2/jobs` (host log), not Comfy queue.
|
||||
3. `engine:"yue"` still hits `/yuegp/jobs`.
|
||||
4. ACE graphs unchanged.
|
||||
5. After a real 5080 run: WAV in library, worker unloaded, Comfy can wake again.
|
||||
|
||||
This agent session did not generate a song.
|
||||
|
||||
## Remove old YuE / YuEGP data only (host checklist)
|
||||
|
||||
Run these **only after** YuE2 has produced a real song and you no longer need YuEGP. Named folders only. Do **not** recursively delete Comfy `models/` or `custom_nodes/`.
|
||||
|
||||
Adjust paths if your `YUEGP_ROOT` / HF cache differ. Review each path before deleting.
|
||||
|
||||
```powershell
|
||||
# --- YuEGP checkout weights / jobs (not the aigen scripts) ---
|
||||
# Example defaults from setup-yuegp.ps1 / docs/yuegp-replacement.md
|
||||
$YueGpRoot = 'C:\Users\ianjm\Development\YuEGP'
|
||||
$YueGpJobs = Join-Path $YueGpRoot 'aigen-jobs'
|
||||
$YueModels = Join-Path $env:LOCALAPPDATA 'Comfy-Desktop\ComfyUI-Shared\models\yue'
|
||||
|
||||
# Stage 1 / Stage 2 local folders (only if clearly YuE v1)
|
||||
Remove-Item -LiteralPath (Join-Path $YueModels 'YuE-s1-7B-anneal-en-cot') -Recurse -Force -ErrorAction SilentlyContinue
|
||||
Remove-Item -LiteralPath (Join-Path $YueModels 'YuE-s2-1B-general') -Recurse -Force -ErrorAction SilentlyContinue
|
||||
|
||||
# Old codec under YuEGP inference (YuEGP-only copy)
|
||||
Remove-Item -LiteralPath (Join-Path $YueGpRoot 'inference\xcodec_mini_infer') -Recurse -Force -ErrorAction SilentlyContinue
|
||||
|
||||
# Old YuEGP song dumps
|
||||
Remove-Item -LiteralPath $YueGpJobs -Recurse -Force -ErrorAction SilentlyContinue
|
||||
|
||||
# Hugging Face cache folders that are clearly YuE v1 / old codec only
|
||||
$Hf = Join-Path $env:USERPROFILE '.cache\huggingface\hub'
|
||||
Remove-Item -LiteralPath (Join-Path $Hf 'models--m-a-p--YuE-s1-7B-anneal-en-cot') -Recurse -Force -ErrorAction SilentlyContinue
|
||||
Remove-Item -LiteralPath (Join-Path $Hf 'models--m-a-p--YuE-s2-1B-general') -Recurse -Force -ErrorAction SilentlyContinue
|
||||
Remove-Item -LiteralPath (Join-Path $Hf 'models--m-a-p--xcodec_mini_infer') -Recurse -Force -ErrorAction SilentlyContinue
|
||||
|
||||
# Leftover Comfy YuE Stage A/B weight copies (YuE-only names). Leave ACE / H3 / Klein / Krea / LTX / upscale alone.
|
||||
$ComfyYue = Join-Path $env:LOCALAPPDATA 'Comfy-Desktop\ComfyUI-Shared\models'
|
||||
Get-ChildItem -LiteralPath $ComfyYue -Recurse -ErrorAction SilentlyContinue |
|
||||
Where-Object { $_.Name -match 'YUE_Stage_|YuE-s1-|YuE-s2-|yue_stage' } |
|
||||
ForEach-Object { Remove-Item -LiteralPath $_.FullName -Recurse -Force -ErrorAction SilentlyContinue }
|
||||
```
|
||||
|
||||
**Do not delete:** ACE-Step weights, ACE workflow JSON, MiniMax H3 / LTX / Flux Klein / Krea assets, LoRAs, `LIBRARY_DIR`, Studio 2 graphs, or `scripts/yuegp-*` / `server/utils/yueGp.ts` until YuE2 has proven a real song. Data off disk ≠ rip out the `engine:"yue"` code path.
|
||||
@@ -0,0 +1,21 @@
|
||||
"""CPU-only YuE2 worker regression tests; never import torch or run inference."""
|
||||
import importlib.util
|
||||
from pathlib import Path
|
||||
import unittest
|
||||
|
||||
spec = importlib.util.spec_from_file_location('yue2_worker', Path(__file__).parents[1] / 'scripts/yue2-worker.py')
|
||||
worker = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(worker)
|
||||
|
||||
|
||||
class WorkerTests(unittest.TestCase):
|
||||
def test_lyrics_preserve_words_and_normalize_ui_headings(self):
|
||||
self.assertEqual(worker.normalize_lyrics('[Pre-Chorus]\nEvery word stays\n[Outro]\n'), '[prechorus]\nEvery word stays\n\n')
|
||||
self.assertEqual(worker.normalize_lyrics('[Verse 1]\nHello'), '[verse1]\nHello\n\n')
|
||||
for lyrics in ['', 'No heading', '[Verse]\nA\n[Chorus]\nB']:
|
||||
with self.assertRaises(ValueError):
|
||||
worker.normalize_lyrics(lyrics)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,95 @@
|
||||
import test from 'node:test'
|
||||
import assert from 'node:assert/strict'
|
||||
import { EventEmitter } from 'node:events'
|
||||
import { PassThrough } from 'node:stream'
|
||||
import { mkdtempSync, writeFileSync, readFileSync, existsSync } from 'node:fs'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import ts from 'typescript'
|
||||
import { createYue2Host, validateYue2Request } from '../scripts/yue2-host.mjs'
|
||||
import { createYueGpHost, validateYueGpRequest } from '../scripts/yuegp-host.mjs'
|
||||
|
||||
const request = { id: 'test-song-123456', tags: 'pop, warm vocals', lyrics: '[Verse 1]\nA quiet morning, a new day', seed: 0 }
|
||||
|
||||
test('YuE2 defaults are 60 seconds and keep Yue lyric validation', () => {
|
||||
assert.deepEqual(validateYue2Request(request), { ...request, duration: 60 })
|
||||
assert.throws(() => validateYue2Request({ ...request, lyrics: '[Verse]\nA\n[Chorus]\nB' }))
|
||||
assert.throws(() => validateYue2Request({ ...request, duration: 20 }))
|
||||
assert.throws(() => validateYue2Request({ ...request, id: '../escape' }))
|
||||
})
|
||||
|
||||
test('yue still validates as YuEGP and stays separate from yue2', () => {
|
||||
assert.equal(validateYueGpRequest(request).profile, 1)
|
||||
assert.equal(validateYue2Request(request).duration, 60)
|
||||
assert.notEqual(createYueGpHost.name, createYue2Host.name)
|
||||
})
|
||||
|
||||
function fixture() {
|
||||
const root = mkdtempSync(join(tmpdir(), 'aigen-yue2-'))
|
||||
const python = join(root, 'python.exe')
|
||||
writeFileSync(python, '')
|
||||
writeFileSync(join(root, 'aigen-ready.json'), JSON.stringify({ model: 'm-a-p/YuE2-3B', vae: 'm-a-p/YuE2-Vae' }))
|
||||
const child = new EventEmitter()
|
||||
child.stdout = new PassThrough(); child.stderr = new PassThrough()
|
||||
child.kill = () => { setImmediate(() => child.emit('close', 1)); return true }
|
||||
const calls = []
|
||||
const dataDir = join(root, 'jobs')
|
||||
const host = createYue2Host({ root, python, dataDir, prepare: async () => {}, leaseValid: () => true,
|
||||
spawnProcess: (...args) => { calls.push(args); return child } })
|
||||
return { host, child, calls, root, dataDir }
|
||||
}
|
||||
|
||||
test('host launches yue2-worker, preserves seed 0 and lyrics, and deduplicates POST retries', async () => {
|
||||
const f = fixture()
|
||||
await f.host.start(request, 'lease')
|
||||
await f.host.start(request, 'lease')
|
||||
assert.equal(f.calls.length, 1)
|
||||
const [, args, opts] = f.calls[0]
|
||||
assert.ok(String(args[1]).endsWith('yue2-worker.py'))
|
||||
assert.equal(opts.windowsHide, true)
|
||||
assert.equal(opts.shell, false)
|
||||
const posted = JSON.parse(readFileSync(join(f.dataDir, request.id, 'request.json')))
|
||||
assert.equal(posted.seed, 0)
|
||||
assert.equal(posted.lyrics, request.lyrics)
|
||||
assert.equal(posted.model, 'm-a-p/YuE2-3B')
|
||||
assert.equal(posted.vae, 'm-a-p/YuE2-Vae')
|
||||
await assert.rejects(f.host.start({ ...request, id: 'second-song-12345' }, 'lease'), /already running/)
|
||||
await f.host.cancel(request.id)
|
||||
assert.equal(f.host.busy(), false)
|
||||
assert.equal(f.host.read(request.id).status, 'cancelled')
|
||||
})
|
||||
|
||||
test('OOM stays on yue2 with no engine switch', async () => {
|
||||
const f = fixture()
|
||||
await f.host.start(request, 'lease')
|
||||
f.child.stdout.write('AIGEN_EVENT {"stage":"error","error":"CUDA out of memory"}\n')
|
||||
f.child.emit('close', 1)
|
||||
assert.match(f.host.read(request.id).error, /out of memory/)
|
||||
assert.equal(f.calls.length, 1)
|
||||
})
|
||||
|
||||
test('music workflow refuses yue and yue2 while ACE graphs still build', () => {
|
||||
const source = readFileSync(new URL('../server/utils/musicWorkflow.ts', import.meta.url), 'utf8')
|
||||
const code = ts.transpileModule(source, { compilerOptions: { target: ts.ScriptTarget.ES2022, module: ts.ModuleKind.CommonJS } }).outputText
|
||||
const exports = {}
|
||||
new Function('require', 'exports', 'useRuntimeConfig', code)(id => {
|
||||
if (id.endsWith('.json')) return { default: JSON.parse(readFileSync(new URL(`../server/assets/${id.split('/').pop()}`, import.meta.url))) }
|
||||
if (id === '~/utils/music') return { parseMusicBpm: () => 120 }
|
||||
throw new Error(id)
|
||||
}, exports, () => ({}))
|
||||
const params = { tags: 'pop', lyrics: '[Verse]\nhello', duration: 60, steps: 8, seed: 0, cfg: 1, lyricsStrength: 0.9 }
|
||||
assert.throws(() => exports.buildMusicWorkflow({ ...params, engine: 'yue' }), /standalone YuEGP/)
|
||||
assert.throws(() => exports.buildMusicWorkflow({ ...params, engine: 'yue2' }), /standalone YuE2/)
|
||||
for (const engine of ['ace-step', 'ace-step-1.5']) {
|
||||
const graph = exports.buildMusicWorkflow({ ...params, engine })
|
||||
assert.ok(Object.values(graph).some(n => n.class_type === 'KSampler' && n.inputs.seed === 0))
|
||||
}
|
||||
assert.equal(existsSync(new URL('../server/assets/workflow_yue.json', import.meta.url)), false)
|
||||
})
|
||||
|
||||
test('musicChain routes yue2 to startYue2Job and leaves yue on YuEGP', () => {
|
||||
const source = readFileSync(new URL('../server/utils/musicChain.ts', import.meta.url), 'utf8')
|
||||
assert.match(source, /engine === 'yue'\) return startYueGpJob/)
|
||||
assert.match(source, /engine === 'yue2'\) return startYue2Job/)
|
||||
assert.match(source, /from '\.\/yue2'/)
|
||||
})
|
||||
Reference in New Issue
Block a user