Compare commits

...
6 Commits
Author SHA1 Message Date
TowstyandCursor fae3b23e37 Finish YuE2 host setup, weight download paths, and v1 cleanup.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 18:46:35 -05:00
TowstyandCursor d1ff05089f Document YuE2 install, weight cleanup, and regression tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 18:46:35 -05:00
TowstyandCursor 71a61b1765 Add YuE2 to the music engine picker and generate API.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 18:46:35 -05:00
TowstyandCursor ec5791c595 Mount YuE2 on the host agent and share GPU ownership.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 18:46:35 -05:00
TowstyandCursor 5f68205f01 Route engine yue2 through a standalone job helper.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 18:45:57 -05:00
TowstyandCursor 85ef087364 Add YuE2 host, worker, and setup scripts.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 18:45:57 -05:00
20 changed files with 851 additions and 47 deletions
+15 -6
View File
@@ -99,7 +99,7 @@
</svg>
<span class="mt-3 block font-display text-lg font-bold text-zinc-100">Music</span>
</NuxtLink>
<div class="grid grid-cols-2 border-t border-white/10">
<div class="grid grid-cols-3 border-t border-white/10">
<button
type="button"
class="border-r border-white/10 px-3 py-2.5 text-left text-sm transition"
@@ -111,12 +111,21 @@
</button>
<button
type="button"
class="px-3 py-2.5 text-left text-sm transition"
class="border-r border-white/10 px-3 py-2.5 text-left text-sm transition"
:class="engineBtnClass(current === 'music' && musicEngine === 'yue')"
@click="pickMusic('yue')"
>
<span class="block font-semibold">YuE</span>
<span class="text-xs opacity-80">Lyrics</span>
<span class="text-xs opacity-80">v1</span>
</button>
<button
type="button"
class="px-3 py-2.5 text-left text-sm transition"
:class="engineBtnClass(current === 'music' && musicEngine === 'yue2')"
@click="pickMusic('yue2')"
>
<span class="block font-semibold">YuE2</span>
<span class="text-xs opacity-80">3B</span>
</button>
</div>
</div>
@@ -131,7 +140,7 @@ const props = withDefaults(defineProps<{
videoEngine?: 'minimax' | 'ltx'
videoStart?: 'still' | 'text'
imageEngine?: 'flux' | 'krea'
musicEngine?: 'ace' | 'yue'
musicEngine?: 'ace' | 'yue' | 'yue2'
}>(), {
ltxEnabled: false,
videoEngine: 'minimax',
@@ -144,7 +153,7 @@ const emit = defineEmits<{
select: [kind: 'video' | 'image']
'video-engine': [engine: 'minimax' | 'ltx']
'image-engine': [engine: 'flux' | 'krea']
'music-engine': [engine: 'ace' | 'yue']
'music-engine': [engine: 'ace' | 'yue' | 'yue2']
}>()
function engineBtnClass(active: boolean) {
@@ -162,7 +171,7 @@ function pickImage(engine: 'flux' | 'krea') {
if (props.current !== 'image') emit('select', 'image')
}
function pickMusic(engine: 'ace' | 'yue') {
function pickMusic(engine: 'ace' | 'yue' | 'yue2') {
if (props.current === 'music') {
emit('music-engine', engine)
return
+78
View File
@@ -0,0 +1,78 @@
# YuE2 music engine
`engine: "yue2"` is standalone [YuE2](https://github.com/multimodal-art-projection/YuE) (`m-a-p/YuE2-3B` + `m-a-p/YuE2-Vae`). It never submits a Comfy graph. `engine: "yue"` remains YuEGP in code. ACE / ACE 1.5 keep their Comfy workflows.
## What this host already has (5080 desktop)
Ran on this machine:
1. `scripts/setup-yue2.ps1` → checkout + venv + CUDA torch + weight download
2. `scripts/cleanup-yue-v1.ps1` → named YuE v1 / YuEGP weight folders removed
3. Host agent code mounts `POST/GET /yue2/jobs` (same pattern as `/yuegp/jobs`)
Paths written:
| Item | Path |
| --- | --- |
| `YUE2_ROOT` | `C:\Users\ianjm\Development\YuE2` |
| Python | `C:\Users\ianjm\Development\YuE2\.venv\Scripts\python.exe` |
| Model | `C:\Users\ianjm\Development\YuE2\models\YuE2-3B` (~7.3 GB) |
| VAE | `C:\Users\ianjm\Development\YuE2\models\YuE2-Vae` (~531 MB) |
| Ready file | `C:\Users\ianjm\Development\YuE2\aigen-ready.json` |
| Jobs dir default | `C:\Users\ianjm\Development\YuE2\aigen-jobs` |
`aigen-ready.json` points `model` / `vae` at those local folders so Generate does not need a first-click HF pull. CUDA check: `torch 2.11.0+cu128` on `NVIDIA GeForce RTX 5080`.
After deploying this branch, restart the **desktop host connector** once so it loads `yue2-host.mjs`. Do not restart Coolify from the agent; the user triggers the existing xAIGen-dev Coolify rebuild.
## Env vars
| Variable | Purpose |
| --- | --- |
| `YUE2_ROOT` | Standalone YuE checkout |
| `YUE2_PYTHON` | Isolated worker Python |
| `YUE2_JOBS_DIR` | Per-job logs / request / WAV |
| `YUE2_MODEL` | Local model dir or HF id |
| `YUE2_VAE` | Local VAE dir or HF id |
App uses existing `COMFY_CONTROL_URL` + token. One GPU owner via the shared reservation (Comfy stopped while YuE2 runs). One song. Unload after decode. OOM is visible; no fallback to `yue`.
## Call path
```json
{"engine":"yue2","duration":60,"seed":42,"tags":"pop, warm vocals","lyrics":"[Verse 1]\nYour lyrics here","folderId":"<library-folder>","name":"My song"}
```
Picker → `POST /api/generate/music` → `startYue2Job` → `POST <COMFY_CONTROL_URL>/yue2/jobs` → `yue2-worker.py` (`plan` → `generate_semantic` → `synthesize` → `decode`) → library WAV.
Duration 30–150 s (default 60). One non-empty lyric section (same validation as Yue).
## Old YuE / YuEGP data deleted on this host
`scripts/cleanup-yue-v1.ps1` logged deletions to `scripts/cleanup-yue-v1.log`. Deleted:
- `...\ComfyUI-Shared\models\yue\YuE-s1-7B-anneal-en-cot`
- `...\ComfyUI-Shared\models\yue\YuE-s2-1B-general`
- `...\ComfyUI-Shared\models\yue\xcodec_mini_infer`
- `...\ComfyUI-Shared\models\yue\ckpt_00360000.pth`
- `...\ComfyUI-Shared\models\yue\decoder_131000.pth`
- `...\ComfyUI-Shared\models\yue\decoder_151000.pth`
- `C:\Users\ianjm\Development\YuEGP\inference\xcodec_mini_infer`
- `C:\Users\ianjm\Development\YuEGP\aigen-jobs`
HF cache YuE-s1 / YuE-s2 / xcodec entries were already absent (skipped). ACE checkpoints (`ace_step_v1_3.5b.safetensors`, `ace_step_1.5_turbo_aio.safetensors`) and other Comfy model trees were left alone. `scripts/yuegp-*` and `server/utils/yueGp.ts` remain in git.
## Re-run setup (only if rebuilding the machine)
```powershell
.\scripts\setup-yue2.ps1
.\scripts\cleanup-yue-v1.ps1
```
Setup reuses existing `models\YuE2-3B` / `YuE2-Vae` when present, then force-reinstalls CUDA torch after the editable package install.
## Validation
- JS: `node --test tests/yue2.test.mjs tests/yuegp.test.mjs`
- Python: `py -3 -m unittest tests.test_yue2_worker tests.test_yuegp_worker`
- No full song was generated in this agent session.
+34 -16
View File
@@ -25,7 +25,7 @@
</div>
<GenerationSettings :advanced="advanced">
<label class="block space-y-2 text-sm"><span>Model</span><select :value="engineFamily" class="w-full rounded-xl border border-white/15 bg-zinc-900 p-3" @change="selectEngineFamily(($event.target as HTMLSelectElement).value as 'ace' | 'yue')"><option value="ace">ACE-Step</option><option value="yue">YuE</option></select></label>
<label class="block space-y-2 text-sm"><span>Model</span><select :value="engineFamily" class="w-full rounded-xl border border-white/15 bg-zinc-900 p-3" @change="selectEngineFamily(($event.target as HTMLSelectElement).value as 'ace' | 'yue' | 'yue2')"><option value="ace">ACE-Step</option><option value="yue">YuE</option><option value="yue2">YuE2</option></select></label>
<label v-if="engineFamily === 'yue'" class="block space-y-2 text-sm">
<span>YuEGP profile</span>
<select v-model.number="yueProfile" class="w-full rounded-xl border border-white/15 bg-zinc-900 p-3">
@@ -34,8 +34,9 @@
</select>
<span class="block text-xs text-zinc-400">One lyric section. Compile off. Profile 3 is never selected automatically.</span>
</label>
<p v-else-if="engineFamily === 'yue2'" class="text-xs text-zinc-400">YuE2-3B · one lyric section · plan → synthesize → decode · no score editor</p>
<div v-if="!advanced" class="space-y-4">
<label v-if="engineFamily !== 'yue'" class="block text-sm">Steps<input v-model.number="steps" type="number" :min="stepsMin" :max="stepsMax" class="mt-2 w-full rounded-xl border border-white/15 bg-zinc-900 p-3"></label>
<label v-if="engineFamily === 'ace'" class="block text-sm">Steps<input v-model.number="steps" type="number" :min="stepsMin" :max="stepsMax" class="mt-2 w-full rounded-xl border border-white/15 bg-zinc-900 p-3"></label>
<label class="block text-sm">Seed<input v-model="seed" placeholder="random" class="mt-2 w-full rounded-xl border border-white/15 bg-zinc-900 p-3"></label>
</div>
<div v-if="engineFamily === 'ace'" class="flex items-center justify-between gap-3 rounded-2xl border border-white/10 bg-zinc-950/40 px-4 py-3 text-sm">
@@ -73,7 +74,7 @@
<div class="mb-1 flex flex-wrap items-center justify-between gap-2">
<span class="text-sm font-medium text-zinc-300">Lyrics</span>
<label class="flex cursor-pointer items-center gap-2 text-xs text-zinc-400">
<input v-model="instrumental" :disabled="engineFamily === 'yue'" type="checkbox" class="accent-amber-400">
<input v-model="instrumental" :disabled="engineFamily === 'yue' || engineFamily === 'yue2'" type="checkbox" class="accent-amber-400">
Instrumental
</label>
</div>
@@ -108,9 +109,9 @@
:max="durationMax"
step="5"
>
<span class="mt-1 block text-[11px] text-zinc-500">{{ engineFamily === 'yue' ? 'Approximate length · one lyric section' : `${durationMin}–${durationMax} seconds` }}</span>
<span class="mt-1 block text-[11px] text-zinc-500">{{ engineFamily === 'yue' || engineFamily === 'yue2' ? 'Approximate length · one lyric section' : `${durationMin}–${durationMax} seconds` }}</span>
</label>
<label v-if="advanced && engineFamily !== 'yue'" class="block text-sm">
<label v-if="advanced && engineFamily === 'ace'" class="block text-sm">
<span class="mb-1 block font-medium text-zinc-300">Steps · {{ steps }}</span>
<input
v-model.number="steps"
@@ -208,7 +209,7 @@
</div>
<p v-if="busy && !queued && jobId" class="text-xs text-zinc-400">
{{ musicElapsed }} elapsed · {{ musicActivityText }}
<span v-if="activeMusicEngine === 'yue' && !indeterminateMusic" class="mt-1 block">Progress is for the current stage.</span>
<span v-if="(activeMusicEngine === 'yue' || activeMusicEngine === 'yue2') && !indeterminateMusic" class="mt-1 block">Progress is for the current stage.</span>
<span v-if="indeterminateMusic" class="mt-1 block">Waiting for a progress report.</span>
</p>
<AudioPlayer
@@ -258,7 +259,7 @@ const tagsPlaceholder = DEFAULT_MUSIC_TAGS
const lyricsPlaceholder = DEFAULT_MUSIC_LYRICS
const chips = MUSIC_STRUCTURE_CHIPS
const durationMin = MUSIC_DURATION_MIN
const durationMax = computed(() => engineFamily.value === 'yue' ? 150 : MUSIC_DURATION_MAX)
const durationMax = computed(() => (engineFamily.value === 'yue' || engineFamily.value === 'yue2') ? 150 : MUSIC_DURATION_MAX)
const stepsMin = MUSIC_STEPS_MIN
const stepsMax = MUSIC_STEPS_MAX
@@ -275,7 +276,7 @@ const folderId = ref('')
const folders = ref<LibraryFolder[]>([])
const lyricsEl = ref<HTMLTextAreaElement | null>(null)
const ace15 = ref(false)
const engineFamily = ref<'ace' | 'yue'>('ace')
const engineFamily = ref<'ace' | 'yue' | 'yue2'>('ace')
const yueProfile = ref<1 | 3>(1)
const busy = ref(false)
const forceClearing = ref(false)
@@ -287,7 +288,7 @@ const musicStartedAt = ref(0)
const activeMusicEngine = ref('')
const musicActivity = ref<{ checkedAt: number; running: boolean } | null>(null)
let musicClockTimer: ReturnType<typeof setInterval> | null = null
const indeterminateMusic = computed(() => !queued.value && activeMusicEngine.value === 'yue' && progress.value < 98 && !musicActivity.value?.running)
const indeterminateMusic = computed(() => !queued.value && (activeMusicEngine.value === 'yue' || activeMusicEngine.value === 'yue2') && progress.value < 98 && !musicActivity.value?.running)
const recentMusicCheck = computed(() => Boolean(musicActivity.value?.running && musicClock.value - musicActivity.value.checkedAt < 15000))
const musicElapsed = computed(() => {
const seconds = musicStartedAt.value ? Math.max(0, Math.floor((musicClock.value - musicStartedAt.value) / 1000)) : 0
@@ -318,27 +319,29 @@ let recoverPoll: ReturnType<typeof setInterval> | null = null
const blocked = computed(() => Boolean(blockReason.value))
const headerSubtitle = computed(() => {
if (engineFamily.value === 'yue2') return 'YuE2 · lyrics-to-song'
if (engineFamily.value === 'yue') return 'YuE · lyrics-to-song'
return ace15.value ? 'ACE-Step 1.5 · vocals' : 'ACE-Step · vocals'
})
const selectedEngine = computed<MusicEngine>(() => {
if (engineFamily.value === 'yue2') return 'yue2'
if (engineFamily.value === 'yue') return 'yue'
return ace15.value ? 'ace-step-1.5' : 'ace-step'
})
const blockReason = computed(() => {
if (!folderId.value) return 'Choose a library folder before generating.'
if (engineFamily.value === 'yue' && instrumental.value) return 'YuEGP requires lyrics. Use ACE for instrumental music.'
if (engineFamily.value === 'yue' && duration.value > 150) return 'YuEGP supports up to 150 seconds per section.'
if ((engineFamily.value === 'yue' || engineFamily.value === 'yue2') && instrumental.value) return 'YuE requires lyrics. Use ACE for instrumental music.'
if ((engineFamily.value === 'yue' || engineFamily.value === 'yue2') && duration.value > 150) return 'YuE supports up to 150 seconds per section.'
if (!tags.value.trim()) return 'Add genre and style tags.'
if (engineFamily.value === 'yue') { const problem = yueLyricsProblem(lyrics.value); if (problem) return problem }
if (engineFamily.value === 'yue' || engineFamily.value === 'yue2') { const problem = yueLyricsProblem(lyrics.value); if (problem) return problem }
if (!instrumental.value && !lyrics.value.trim()) return 'Write lyrics, or turn on Instrumental.'
return ''
})
function selectEngineFamily(family: 'ace' | 'yue') {
function selectEngineFamily(family: 'ace' | 'yue' | 'yue2') {
engineFamily.value = family
if (family === 'yue') { instrumental.value = false; duration.value = Math.min(150, duration.value) }
if (family === 'yue' && lyrics.value === DEFAULT_MUSIC_LYRICS) lyrics.value = '[Verse 1]\n'
if (family === 'yue' || family === 'yue2') { instrumental.value = false; duration.value = Math.min(150, duration.value) }
if ((family === 'yue' || family === 'yue2') && lyrics.value === DEFAULT_MUSIC_LYRICS) lyrics.value = '[Verse 1]\n'
if (family === 'ace' && ace15.value && steps.value === MUSIC_STEPS_DEFAULT) {
steps.value = MUSIC_STEPS_DEFAULT_15
}
@@ -352,6 +355,11 @@ function goStudio(kind: 'video' | 'image') {
}
function applyEngine(engine: MusicEngine) {
if (engine === 'yue2') {
engineFamily.value = 'yue2'
ace15.value = false
return
}
if (engine === 'yue') {
engineFamily.value = 'yue'
ace15.value = false
@@ -454,6 +462,8 @@ async function attachLiveMusic(opts: {
else if (!status.value || /queueing|waiting for a generate|waiting in the job queue|waiting for gpu/i.test(status.value)) {
status.value = selectedEngine.value === 'yue'
? 'YuEGP is generating your song'
: selectedEngine.value === 'yue2'
? 'YuE2 is generating your song'
: 'Generating…'
}
if (jobId.value !== liveId) {
@@ -499,6 +509,8 @@ async function refreshStudioQueue() {
? 'Paused in the job queue'
: (selectedEngine.value === 'yue'
? 'Waiting for the GPU to run YuEGP'
: selectedEngine.value === 'yue2'
? 'Waiting for the GPU to run YuE2'
: 'Waiting in the job queue…')
}
@@ -511,6 +523,8 @@ async function refreshStudioQueue() {
if (jobId.value && /waiting in the job queue/i.test(status.value)) {
status.value = selectedEngine.value === 'yue'
? 'YuEGP is generating your song'
: selectedEngine.value === 'yue2'
? 'YuE2 is generating your song'
: 'Generating…'
}
return
@@ -522,6 +536,8 @@ async function refreshStudioQueue() {
? 'Paused in the job queue'
: (selectedEngine.value === 'yue'
? 'Waiting for the GPU to run YuEGP'
: selectedEngine.value === 'yue2'
? 'Waiting for the GPU to run YuE2'
: 'Waiting in the job queue…')
return
}
@@ -667,6 +683,8 @@ async function generate() {
studioJobId.value = ''
status.value = engineFamily.value === 'yue'
? 'Queueing YuE…'
: engineFamily.value === 'yue2'
? 'Queueing YuE2…'
: ace15.value
? 'Queueing ACE-Step 1.5…'
: 'Queueing ACE-Step…'
@@ -796,7 +814,7 @@ onMounted(async () => {
await loadMusicPresets()
const route = useRoute()
const engineQ = String(route.query.engine || '').toLowerCase()
if (engineQ === 'yue' || engineQ === 'ace') selectEngineFamily(engineQ)
if (engineQ === 'yue' || engineQ === 'yue2' || engineQ === 'ace') selectEngineFamily(engineQ as 'ace' | 'yue' | 'yue2')
applyIncomingMusic()
await resumeActiveMusic()
await refreshStudioQueue()
+66
View File
@@ -0,0 +1,66 @@
# Delete named YuE v1 / YuEGP weight folders only. Never recurse-wipe Comfy models\ or custom_nodes\.
[CmdletBinding()]
param(
[string]$YueGpRoot = 'C:\Users\ianjm\Development\YuEGP',
[string]$YueModels = '',
[string]$LogPath = ''
)
$ErrorActionPreference = 'Continue'
if (-not $YueModels) {
$YueModels = Join-Path $env:LOCALAPPDATA 'Comfy-Desktop\ComfyUI-Shared\models\yue'
}
if (-not $LogPath) {
$LogPath = Join-Path $PSScriptRoot 'cleanup-yue-v1.log'
}
function Log([string]$Message) {
$line = "$(Get-Date -Format o) $Message"
Add-Content -LiteralPath $LogPath -Value $line
Write-Host $line
}
function RemoveNamed([string]$Path) {
if (-not $Path) { return }
if (!(Test-Path -LiteralPath $Path)) {
Log "SKIP missing $Path"
return
}
try {
Remove-Item -LiteralPath $Path -Recurse -Force
Log "DELETED $Path"
} catch {
Log "FAIL $Path :: $($_.Exception.Message)"
}
}
Log "START cleanup-yue-v1 YueGpRoot=$YueGpRoot YueModels=$YueModels"
# Stage 1 / Stage 2 + codec under the shared yue models dir (named folders only)
RemoveNamed (Join-Path $YueModels 'YuE-s1-7B-anneal-en-cot')
RemoveNamed (Join-Path $YueModels 'YuE-s2-1B-general')
RemoveNamed (Join-Path $YueModels 'xcodec_mini_infer')
# Loose codec files that live beside the stage folders in this install
foreach ($name in @('ckpt_00360000.pth', 'decoder_131000.pth', 'decoder_151000.pth')) {
RemoveNamed (Join-Path $YueModels $name)
}
# YuEGP checkout codec copy + job dumps
RemoveNamed (Join-Path $YueGpRoot 'inference\xcodec_mini_infer')
RemoveNamed (Join-Path $YueGpRoot 'aigen-jobs')
# Hugging Face cache dirs that are clearly YuE v1 / old codec
$Hf = Join-Path $env:USERPROFILE '.cache\huggingface\hub'
RemoveNamed (Join-Path $Hf 'models--m-a-p--YuE-s1-7B-anneal-en-cot')
RemoveNamed (Join-Path $Hf 'models--m-a-p--YuE-s2-1B-general')
RemoveNamed (Join-Path $Hf 'models--m-a-p--xcodec_mini_infer')
# Leftover Comfy YuE Stage A/B weight copies by name only (no blanket models\ purge)
$ComfyModels = Join-Path $env:LOCALAPPDATA 'Comfy-Desktop\ComfyUI-Shared\models'
if (Test-Path -LiteralPath $ComfyModels) {
Get-ChildItem -LiteralPath $ComfyModels -Recurse -ErrorAction SilentlyContinue |
Where-Object { $_.Name -match '^(YUE_Stage_|YuE-s1-|YuE-s2-|yue_stage)' } |
ForEach-Object { RemoveNamed $_.FullName }
} else {
Log "SKIP missing $ComfyModels"
}
Log "DONE cleanup-yue-v1 log=$LogPath"
Write-Host "Left in place: ACE / H3 / Klein / Krea / LTX / upscale / LoRAs / LIBRARY_DIR / yuegp code paths."
+39 -3
View File
@@ -4,6 +4,7 @@ import { stableMemoryArgs } from './comfy-memory-policy.mjs'
import { createGpuReservation } from './gpu-reservation.mjs'
import { createGpuProxy } from './gpu-proxy.mjs'
import { createYueGpHost } from './yuegp-host.mjs'
import { createYue2Host } from './yue2-host.mjs'
import http from 'node:http'
import net from 'node:net'
import { execFile, spawn } from 'node:child_process'
@@ -147,7 +148,7 @@ let proxyTarget = 0
function ensureProxyListening() {
if (proxyServer) return
proxyServer = createGpuProxy({ target: () => proxyTarget, reservation: gpuReservation, authorized, markWork, externalBusy: () => yueGp.busy() || upscale.busy() })
proxyServer = createGpuProxy({ target: () => proxyTarget, reservation: gpuReservation, authorized, markWork, externalBusy: () => yueGp.busy() || yue2.busy() || upscale.busy() })
proxyServer.on('error', (error) => {
console.log(JSON.stringify({ src: 'comfy-host-agent', event: 'proxy-error', error: String(error.message || error) }))
})
@@ -819,7 +820,7 @@ function purgeDesktopFiles(body) {
}
const gpuReservation = createGpuReservation({ idle: async () => {
if (yueGp.busy() || upscale.busy()) return false
if (yueGp.busy() || yue2.busy() || upscale.busy()) return false
if ((await trainingLock()).busy) return false
const healthy = await syncProxy()
if (healthy) {
@@ -834,6 +835,7 @@ const yueGp = createYueGpHost({
leaseValid: lease => gpuReservation.isOwner(lease),
prepare: async () => {
if ((await trainingLock()).busy) throw new Error('GPU is busy with training.')
if (yue2.busy()) throw new Error('YuE2 is using the GPU.')
const healthy = await syncProxy()
if (healthy) {
const queue = await fetchLocalQueue(healthy)
@@ -845,6 +847,22 @@ const yueGp = createYueGpHost({
}
})
const yue2 = createYue2Host({
leaseValid: lease => gpuReservation.isOwner(lease),
prepare: async () => {
if ((await trainingLock()).busy) throw new Error('GPU is busy with training.')
if (yueGp.busy()) throw new Error('YuEGP is using the GPU.')
const healthy = await syncProxy()
if (healthy) {
const queue = await fetchLocalQueue(healthy)
if (!queue.ok || queue.running || queue.pending) throw new Error('Comfy is busy; YuE2 cannot start.')
await stopComfyProcesses()
markAsleep()
}
if (await pythonMainUp()) throw new Error('Comfy has not stopped; retry after the GPU is free.')
}
})
const upscale = createUpscaleHost({ leaseValid: token => gpuReservation.isOwner(token) })
async function handleControl(req, res) {
@@ -874,6 +892,22 @@ async function handleControl(req, res) {
}
return json(res, 404, { error: 'Unknown YuEGP endpoint' })
}
if (url.pathname.startsWith('/yue2/')) {
if (yueGp.busy()) return json(res, 409, { message: 'YuEGP is using the GPU.' })
const match = url.pathname.match(/^\/yue2\/jobs\/([a-zA-Z0-9-]{12,80})(\/audio|\/cancel)?$/)
if (req.method === 'GET' && url.pathname === '/yue2/status') return json(res, 200, { configured: yue2.configured(), busy: yue2.busy(), backend: 'yue2' })
if (req.method === 'POST' && url.pathname === '/yue2/jobs') return json(res, 200, await yue2.start(await readJson(req), String(req.headers['x-aigen-gpu-lease'] || '')))
if (match && req.method === 'POST' && match[2] === '/cancel') return json(res, 200, await yue2.cancel(match[1]))
if (match && req.method === 'GET' && match[2] === '/audio') {
const path = yue2.audio(match[1])
return path ? streamFile(res, path) : json(res, 404, { error: 'Audio not ready' })
}
if (match && req.method === 'GET' && !match[2]) {
const job = yue2.read(match[1])
return json(res, job ? 200 : 404, job || { error: 'YuE2 job not found' })
}
return json(res, 404, { error: 'Unknown YuE2 endpoint' })
}
if (req.method === 'POST' && url.pathname.startsWith('/gpu/')) {
const body = await readJson(req)
let result
@@ -906,12 +940,14 @@ async function handleControl(req, res) {
gpu: gpuReservation.availability(),
training: { busy: lastTraining.busy },
yuegp: { busy: yueGp.busy(), configured: yueGp.configured() },
yue2: { busy: yue2.busy(), configured: yue2.configured() },
upscale: { busy: upscale.busy(), engine: 'realesrgan-rife', local: true }
})
}
if (req.method === 'POST' && url.pathname === '/start') {
if (upscale.busy()) return json(res, 409, { message: 'Local upscale is using the GPU.' })
if (yueGp.busy()) return json(res, 409, { message: 'YuEGP is using the GPU.' })
if (yue2.busy()) return json(res, 409, { message: 'YuE2 is using the GPU.' })
const training = await trainingLock()
if (training.busy) {
return json(res, 409, {
@@ -1013,7 +1049,7 @@ const server = http.createServer(async (req, res) => {
} else await handleControl(req, res)
} catch (error) {
req.resume()
if ((String(req.url || '').startsWith('/yuegp/') || String(req.url || '').startsWith('/upscale/')) && !res.headersSent) return json(res, error.statusCode || 400, { error: error.message || 'YuEGP request failed' })
if ((String(req.url || '').startsWith('/yuegp/') || String(req.url || '').startsWith('/yue2/') || String(req.url || '').startsWith('/upscale/')) && !res.headersSent) return json(res, error.statusCode || 400, { error: error.message || 'Music host request failed' })
if (!res.headersSent) json(res, error.statusCode || 400, { ok: false, message: error.statusCode === 409 ? 'GPU is in use. Waiting for availability.' : 'GPU coordination request failed.' })
}
})
+1 -1
View File
@@ -55,7 +55,7 @@ export function createGpuProxy({ target, reservation, authorized = () => true, m
try {
if (req.method === 'GET' || req.method === 'HEAD') await forward()
else {
if (externalBusy()) { req.resume(); return reply(res, 409, { error: { message: 'YuEGP is using the GPU.' }, code: 'GPU_BUSY' }) }
if (externalBusy()) { req.resume(); return reply(res, 409, { error: { message: 'A music or upscale job is using the GPU.' }, code: 'GPU_BUSY' }) }
if (!authorized(req)) { req.resume(); return reply(res, 401, { error: 'Unauthorized' }) }
await reservation.permit(String(req.headers['x-aigen-gpu-lease'] || ''), async () => { markWork(); await forward() })
}
+66
View File
@@ -0,0 +1,66 @@
# Standalone YuE2 only. Never install into Comfy's Python environment.
# Downloads YuE2-3B + YuE2-Vae into YUE2_ROOT\models so the UI does not hit HF on first Generate.
[CmdletBinding()]
param(
[string]$Root = 'C:\Users\ianjm\Development\YuE2',
[string]$Uv = 'uv',
[string]$Models = ''
)
$ErrorActionPreference = 'Stop'
$Root = [IO.Path]::GetFullPath($Root)
if (-not $Models) { $Models = Join-Path $Root 'models' }
$Models = [IO.Path]::GetFullPath($Models)
function Run([string]$Exe, [string[]]$Arguments) {
& $Exe @Arguments
if ($LASTEXITCODE -ne 0) { throw "$Exe failed ($LASTEXITCODE)" }
}
Write-Host "YuE2 setup root=$Root models=$Models"
if (!(Test-Path -LiteralPath $Root)) {
Run git @('clone', 'https://github.com/multimodal-art-projection/YuE.git', $Root)
}
Run $Uv @('venv', '--allow-existing', '--python', '3.12', (Join-Path $Root '.venv'))
$python = Join-Path $Root '.venv\Scripts\python.exe'
# Install the package first, then force CUDA torch back on (editable deps can pull CPU wheels).
Run $Uv @('pip', 'install', '--python', $python, '-e', $Root)
Run $Uv @('pip', 'install', '--python', $python, 'soundfile', 'psutil', 'huggingface_hub', 'hf_xet')
Run $Uv @('pip', 'install', '--python', $python, 'torch', 'torchaudio', '--reinstall', '--index-url', 'https://download.pytorch.org/whl/cu128')
New-Item -ItemType Directory -Force -Path $Models | Out-Null
$modelDir = Join-Path $Models 'YuE2-3B'
$vaeDir = Join-Path $Models 'YuE2-Vae'
if (!(Test-Path -LiteralPath (Join-Path $modelDir 'config.json')) -and !(Test-Path -LiteralPath (Join-Path $modelDir 'model.safetensors.index.json'))) {
Write-Host "Downloading m-a-p/YuE2-3B → $modelDir"
Run $python @(
'-c',
'from huggingface_hub import snapshot_download; import sys; snapshot_download(sys.argv[1], local_dir=sys.argv[2])',
'm-a-p/YuE2-3B',
$modelDir
)
} else {
Write-Host "Reusing existing model at $modelDir"
}
if (!(Test-Path -LiteralPath (Join-Path $vaeDir 'config.json'))) {
Write-Host "Downloading m-a-p/YuE2-Vae → $vaeDir"
Run $python @(
'-c',
'from huggingface_hub import snapshot_download; import sys; snapshot_download(sys.argv[1], local_dir=sys.argv[2])',
'm-a-p/YuE2-Vae',
$vaeDir
)
} else {
Write-Host "Reusing existing VAE at $vaeDir"
}
Run $python @('-c', 'import torch; from yue2 import YuE2Pipeline; assert torch.cuda.is_available(); print(torch.__version__, torch.cuda.get_device_name(0))')
$ready = @{
model = $modelDir
vae = $vaeDir
modelId = 'm-a-p/YuE2-3B'
vaeId = 'm-a-p/YuE2-Vae'
modelsDir = $Models
python = $python
root = $Root
}
$ready | ConvertTo-Json | Set-Content -LiteralPath (Join-Path $Root 'aigen-ready.json') -Encoding utf8
Write-Host "YuE2 ready at $Root"
Write-Host "Local model=$modelDir"
Write-Host "Local vae=$vaeDir"
Write-Host "Set YUE2_ROOT=$Root YUE2_PYTHON=$python YUE2_MODEL=$modelDir YUE2_VAE=$vaeDir if not using defaults. No Comfy environment was modified."
+136
View File
@@ -0,0 +1,136 @@
import { spawn } from 'node:child_process'
import { existsSync, mkdirSync, readFileSync, writeFileSync, renameSync, appendFileSync, readdirSync } from 'node:fs'
import { join, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
export function validateYue2Request(body) {
const duration = body.duration ?? 60
if (!Number.isInteger(duration) || duration < 30 || duration > 150) throw new Error('YuE2 duration must be 30–150 seconds.')
if (!Number.isInteger(body.seed) || body.seed < 0 || body.seed > 2147483647) throw new Error('Invalid seed.')
if (!/^[a-zA-Z0-9-]{12,80}$/.test(body.id || '')) throw new Error('Invalid job ID.')
const tags = String(body.tags || '').trim()
const lyrics = String(body.lyrics || '').trim()
if (!tags || tags.length > 2000 || !lyrics || lyrics.length > 8000) throw new Error('Genre tags and one lyric section are required.')
const sections = [...lyrics.matchAll(/\[([^\]]+)\]\s*([^\[]*)/gs)].filter(m => m[2].trim())
if (sections.length !== 1) throw new Error('YuE2 requires one non-empty lyric section.')
return { id: body.id, duration, seed: body.seed, tags, lyrics }
}
/** One isolated Python process per song. A process exit is the GPU release boundary. */
export function createYue2Host({ prepare, leaseValid, spawnProcess = spawn, root, python, dataDir } = {}) {
const repo = resolve(root || process.env.YUE2_ROOT || 'C:\\Users\\ianjm\\Development\\YuE2')
const executable = python || process.env.YUE2_PYTHON || join(repo, '.venv', 'Scripts', 'python.exe')
const data = resolve(dataDir || process.env.YUE2_JOBS_DIR || join(repo, 'aigen-jobs'))
const worker = fileURLToPath(new URL('./yue2-worker.py', import.meta.url))
let active = null
// A worker whose parent died exits within two seconds. Block new GPU owners
// across a host restart until that watchdog has had time to run.
const restartHoldUntil = existsSync(data) && readdirSync(data).some(id => {
try { return ['running', 'starting', 'cancelling'].includes(JSON.parse(readFileSync(join(data, id, 'status.json'), 'utf8')).status) }
catch { return false }
}) ? Date.now() + 10000 : 0
const dir = id => {
if (!/^[a-zA-Z0-9-]{12,80}$/.test(id || '')) throw new Error('Invalid job ID.')
return join(data, id)
}
const persist = job => {
const target = join(dir(job.id), 'status.json')
writeFileSync(target + '.tmp', JSON.stringify(job))
renameSync(target + '.tmp', target)
}
const read = id => {
if (active?.job.id === id) return { ...active.job }
const path = join(dir(id), 'status.json')
if (!existsSync(path)) return null
const job = JSON.parse(readFileSync(path, 'utf8'))
if (['running', 'starting', 'cancelling'].includes(job.status)) {
job.status = 'error'; job.error = 'YuE2 host restarted. Intermediate files are preserved.'
}
return job
}
return {
busy: () => Boolean(active) || Date.now() < restartHoldUntil,
configured: () => existsSync(executable) && existsSync(join(repo, 'aigen-ready.json')),
read,
audio(id) {
return read(id)?.status === 'complete' ? join(dir(id), 'audio.wav') : null
},
async start(body, lease) {
const request = validateYue2Request(body)
const previous = read(request.id)
if (previous) return previous // POST retries cannot generate the same song twice.
if (active || Date.now() < restartHoldUntil) throw Object.assign(new Error('YuE2 is already running or releasing a previous worker.'), { statusCode: 409 })
if (!existsSync(executable) || !existsSync(join(repo, 'aigen-ready.json'))) throw new Error('YuE2 is not installed. Run scripts/setup-yue2.ps1 on the GPU host.')
const job = { id: request.id, status: 'starting', message: 'Preparing GPU for YuE2', progress: 0, startedAt: Date.now(), checkedAt: Date.now() }
active = { job, child: null, cancelled: false }
const run = active
mkdirSync(dir(job.id), { recursive: true })
persist(job)
try {
await prepare()
if (run.cancelled || !leaseValid(lease)) throw new Error('YuE2 start cancelled or GPU reservation expired.')
const installation = JSON.parse(readFileSync(join(repo, 'aigen-ready.json'), 'utf8').replace(/^\uFEFF/, ''))
const localModel = process.env.YUE2_MODEL || installation.model || 'm-a-p/YuE2-3B'
const localVae = process.env.YUE2_VAE || installation.vae || 'm-a-p/YuE2-Vae'
writeFileSync(join(dir(job.id), 'request.json'), JSON.stringify({
...request,
model: localModel,
vae: localVae
}))
const child = spawnProcess(executable, ['-u', worker, '--root', repo, '--request', join(dir(job.id), 'request.json')],
{ cwd: repo, windowsHide: true, shell: false, stdio: ['ignore', 'pipe', 'pipe'], env: { ...process.env, PYTHONUTF8: '1', TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD: '1' } })
run.child = child
job.status = 'running'; job.message = 'Loading YuE2'; persist(job)
let tail = ''
let lines = ''
const log = chunk => { appendFileSync(join(dir(job.id), 'worker.log'), chunk); tail = (tail + chunk.toString()).slice(-6000) }
child.stderr.on('data', log)
child.stdout.on('data', chunk => {
log(chunk); lines += chunk.toString()
const parts = lines.split(/\r?\n/); lines = parts.pop().slice(-64000)
for (const line of parts) {
if (!line.startsWith('AIGEN_EVENT ')) continue
try {
const event = JSON.parse(line.slice(12))
for (const key of ['stage', 'message', 'progress', 'step', 'maxStep', 'error', 'duration']) if (event[key] !== undefined) job[key] = event[key]
job.checkedAt = Date.now(); persist(job)
} catch { /* Invalid log lines are not state transitions. */ }
}
})
const watchdog = setInterval(() => {
if (!leaseValid(lease)) { job.error = 'GPU reservation expired; YuE2 stopped.'; child.kill() }
}, 5000)
let finished = false
const finish = (code, error) => {
if (finished) return
finished = true
clearInterval(watchdog)
job.status = run.cancelled ? 'cancelled' : !error && code === 0 && existsSync(join(dir(job.id), 'audio.wav')) ? 'complete' : 'error'
if (job.status === 'error') job.error ||= error?.message || tail || `YuE2 exited with code ${code}`
job.message = job.status === 'complete' ? 'Audio ready' : job.status === 'cancelled' ? 'Cancelled' : job.error
job.checkedAt = Date.now(); persist(job)
if (active === run) active = null
}
child.once('error', error => finish(null, error))
child.once('close', code => finish(code))
return { ...job }
} catch (error) {
job.status = run.cancelled ? 'cancelled' : 'error'; job.error = error.message; persist(job)
if (active === run) active = null
throw error
}
},
async cancel(id) {
if (active?.job.id !== id) return read(id)
const run = active
run.cancelled = true
run.job.status = 'cancelling'
persist(run.job)
if (run.child) {
const child = run.child
await new Promise(resolve => { child.once('close', resolve); child.kill() })
}
return read(id)
}
}
}
+104
View File
@@ -0,0 +1,104 @@
"""Headless YuE2 adapter: plan → generate_semantic → synthesize → decode.
No Comfy imports. No score editor, covers, or auto-fallback to YuEGP.
Stdout AIGEN_EVENT lines are consumed by the device-local host agent.
"""
import argparse
import gc
import json
import os
from pathlib import Path
import sys
import time
import threading
def emit(**event):
print('AIGEN_EVENT ' + json.dumps(event), flush=True)
def normalize_lyrics(text):
# Same one-section rule as YuEGP field validation.
import re
sections = re.findall(r'\[([^\]]+)\]\s*([^\[]*)', text.strip(), re.S)
sections = [(name, words.strip()) for name, words in sections if words.strip()]
if len(sections) != 1:
raise ValueError('YuE2 requires one non-empty lyric section. Combine the lyrics under one heading.')
name, words = sections[0]
return '[' + re.sub(r'\W+', '', name).lower() + ']\n' + words + '\n\n'
def main():
import psutil
parent = psutil.Process(os.getppid())
def parent_watchdog():
while parent.is_running():
time.sleep(2)
os._exit(2) # The host died: never leave an orphan consuming the GPU.
threading.Thread(target=parent_watchdog, daemon=True).start()
parser = argparse.ArgumentParser()
parser.add_argument('--root', required=True)
parser.add_argument('--request', required=True)
cli = parser.parse_args()
request = json.loads(Path(cli.request).read_text(encoding='utf-8-sig'))
root = Path(cli.root).resolve()
output = Path(cli.request).resolve().parent
duration = int(request.get('duration', 60))
if not 30 <= duration <= 150:
raise ValueError('Duration must be 30–150 seconds.')
lyrics = normalize_lyrics(request['lyrics'])
style = ' '.join(str(request['tags']).split())
seed = int(request['seed'])
model = request.get('model') or os.environ.get('YUE2_MODEL') or 'm-a-p/YuE2-3B'
vae = request.get('vae') or os.environ.get('YUE2_VAE') or 'm-a-p/YuE2-Vae'
os.chdir(root)
if str(root) not in sys.path:
sys.path.insert(0, str(root))
emit(stage='loading', progress=1, message='Loading YuE2', model=model, vae=vae, duration=duration)
import torch
import soundfile as sf
from yue2 import YuE2Pipeline
if not torch.cuda.is_available():
raise RuntimeError('YuE2 requires a CUDA GPU; CPU fallback is disabled.')
pipe_kwargs = dict(style=style, lyrics=lyrics, cot='full', seed=seed)
# Duration is kept for library metadata and validation. Upstream one-shot
# requests do not take a seconds field; song length follows the plan.
with YuE2Pipeline.from_pretrained(model, vae=vae, device='cuda') as pipe:
emit(stage='plan', message='Planning melody and chords', progress=5)
plan = pipe.plan(**pipe_kwargs)
emit(stage='semantic', message='Generating semantic tokens', progress=25)
semantic = pipe.generate_semantic(plan)
emit(stage='synthesize', message='Synthesizing acoustic latents', progress=55)
latents = pipe.synthesize(semantic)
emit(stage='decode', message='Decoding audio', progress=80)
audio = pipe.decode(latents)
# Context exit unloads the pipeline. Clear any residual CUDA cache.
gc.collect()
if torch.cuda.is_available():
torch.cuda.empty_cache()
wave = audio
if hasattr(audio, 'detach'):
wave = audio.detach().cpu().numpy()
import numpy as np
wave = np.asarray(wave)
if wave.ndim == 1:
pass
elif wave.shape[0] <= 8 and wave.shape[0] < wave.shape[-1]:
wave = wave.T
sample_rate = 48000
target = output / 'audio.wav'
sf.write(str(target), wave, sample_rate, subtype='PCM_16')
info = sf.info(str(target))
if info.frames <= 0:
raise RuntimeError('YuE2 produced empty audio.')
emit(stage='complete', message='Audio ready', progress=100, duration=info.duration)
if __name__ == '__main__':
try:
main()
except Exception as error:
emit(stage='error', message=str(error), error=str(error))
raise
+7 -5
View File
@@ -1,4 +1,5 @@
import { yueGpConfigured } from '~/server/utils/yueGp'
import { yue2Configured } from '~/server/utils/yue2'
import { addStudioJob, kickStudioQueue, listStudioJobs, videoJobsBusy } from '~/server/utils/studioQueue'
import { comfyConfigured } from '~/server/utils/comfy'
import {
@@ -37,7 +38,8 @@ export default defineEventHandler(async (event) => {
const engine = parseMusicEngine(body.engine)
if (engine === 'yue' && !yueGpConfigured()) throw createError({ statusCode: 503, statusMessage: 'YuEGP host is not configured. Set COMFY_CONTROL_URL.' })
if (engine !== 'yue' && !comfyConfigured()) {
if (engine === 'yue2' && !yue2Configured()) throw createError({ statusCode: 503, statusMessage: 'YuE2 host is not configured. Set COMFY_CONTROL_URL.' })
if (engine !== 'yue' && engine !== 'yue2' && !comfyConfigured()) {
throw createError({
statusCode: 503,
statusMessage: 'Beast ComfyUI is not configured. Set COMFY_HOST.'
@@ -66,15 +68,15 @@ export default defineEventHandler(async (event) => {
const destFolder = library.folders.find(folder => folder.id === folderId)
const folderLocked = Boolean(destFolder?.protected && !destFolder.unlocked)
if (engine === 'yue' && instrumental) throw createError({ statusCode: 400, statusMessage: 'YuEGP currently requires lyrics. Use ACE for instrumental music.' })
if ((engine === 'yue' || engine === 'yue2') && instrumental) throw createError({ statusCode: 400, statusMessage: 'YuE currently requires lyrics. Use ACE for instrumental music.' })
const yueProfile = body.yueProfile ?? 1
if (engine === 'yue' && yueProfile !== 1 && yueProfile !== 3) throw createError({ statusCode: 400, statusMessage: 'Choose YuEGP profile 1 or manual fallback 3.' })
if (engine === 'yue' && !instrumental) {
if ((engine === 'yue' || engine === 'yue2') && !instrumental) {
const problem = yueLyricsProblem(lyrics)
if (problem) throw createError({ statusCode: 400, statusMessage: problem })
}
const duration = clampMusicDuration(body.duration)
if (engine === 'yue' && duration > 150) throw createError({ statusCode: 400, statusMessage: 'YuEGP supports up to 150 seconds per section.' })
if ((engine === 'yue' || engine === 'yue2') && duration > 150) throw createError({ statusCode: 400, statusMessage: 'YuE supports up to 150 seconds per section.' })
const steps = clampMusicSteps(
body.steps,
engine === 'ace-step-1.5' ? MUSIC_STEPS_DEFAULT_15 : MUSIC_STEPS_DEFAULT
@@ -86,7 +88,7 @@ export default defineEventHandler(async (event) => {
const lyricsStrength = clampLyricsStrength(body.lyricsStrength)
const seed = parseMusicSeed(body.seed)
const clipName = allocateOutputName(ownerKey, folderId, String(body.name || '').trim(), {
engine: engine === 'ace-step-1.5' ? 'ace15' : engine === 'yue' ? 'yue' : 'ace'
engine: engine === 'ace-step-1.5' ? 'ace15' : engine === 'yue' ? 'yue' : engine === 'yue2' ? 'yue2' : 'ace'
})
const samplerName = String(body.sampler_name || 'euler')
+4
View File
@@ -0,0 +1,4 @@
import { resumeYue2Jobs } from '~/server/utils/yue2'
// Restore live music IDs before queue repair examines their durable studio rows.
export default defineNitroPlugin(() => { resumeYue2Jobs() })
+1
View File
@@ -35,6 +35,7 @@ export interface Job {
studio2?: boolean
upscale?: boolean
yueGp?: boolean
yue2?: boolean
musicActivity?: { checkedAt: number; running: boolean }
id: string
kind?: 'video' | 'edit' | 'music'
+4
View File
@@ -1788,6 +1788,10 @@ export function findYueGpTrack(owner: string, jobId: string) {
return readCatalog(owner).tracks.find(track => track.comfyFilename === `yuegp-${jobId}.wav`)
}
export function findYue2Track(owner: string, jobId: string) {
return readCatalog(owner).tracks.find(track => track.comfyFilename === `yue2-${jobId}.wav`)
}
export async function saveTrack(params: {
ownerKey: string
folderId: string
+2
View File
@@ -1,4 +1,5 @@
import { startYueGpJob } from './yueGp'
import { startYue2Job } from './yue2'
import { createJob, emitJob, type Job } from '~/server/utils/jobs'
import { extractAudio, fetchHistory, fetchHistoryAll, findHistoryAudio, purgeComfyArtifacts, queuePrompt } from '~/server/utils/comfy'
import { comfyWsUrl, comfyFetch } from '~/server/utils/comfy'
@@ -341,6 +342,7 @@ function watchMusicJob(job: Job): Promise<void> {
export async function startMusicJob(params: MusicJobParams) {
if (params.engine === 'yue') return startYueGpJob(params)
if (params.engine === 'yue2') return startYue2Job(params)
const job = createJob('music')
job.library = {
ownerKey: params.ownerKey,
+2
View File
@@ -101,6 +101,7 @@ function buildAce15Workflow(params: MusicWorkflowParams): WorkflowGraph {
export function buildMusicWorkflow(params: MusicWorkflowParams): WorkflowGraph {
const engine = params.engine || 'ace-step'
if (engine === 'yue') throw new Error('YuE requires the standalone YuEGP backend.')
if (engine === 'yue2') throw new Error('YuE2 requires the standalone YuE2 backend.')
if (engine === 'ace-step-1.5') return buildAce15Workflow(params)
return buildAceV1Workflow(params)
}
@@ -108,6 +109,7 @@ export function buildMusicWorkflow(params: MusicWorkflowParams): WorkflowGraph {
export async function assertMusicEngineNodes(engine: MusicEngine | undefined) {
const { comfyHasClassType } = await import('~/server/utils/comfy')
if (engine === 'yue') throw new Error('YuE cannot run on Comfy.')
if (engine === 'yue2') throw new Error('YuE2 cannot run on Comfy.')
if (engine === 'ace-step-1.5') {
const present = await comfyHasClassType('TextEncodeAceStepAudio1.5')
if (present === false) {
+16 -7
View File
@@ -286,7 +286,7 @@ function failZombieLiveJob(job: Job, error: string) {
function sweepStaleLiveJobs() {
const now = Date.now()
for (const job of listJobs()) {
if (job.studio2 || job.yueGp || job.upscale) continue
if (job.studio2 || job.yueGp || job.yue2 || job.upscale) continue
if (job.saving) continue
if (job.status === 'queued' && !job.promptId && now - job.startedAt >= QUEUED_GRACE_MS) {
failZombieLiveJob(job, 'Job never started')
@@ -306,7 +306,7 @@ async function reapZombieLiveJobs() {
const { fetchHistory } = await import('~/server/utils/comfy')
for (const job of listJobs()) {
// Never interrupt download/stitch/library save — Comfy is idle then by design.
if (job.studio2 || job.yueGp || job.upscale) continue
if (job.studio2 || job.yueGp || job.yue2 || job.upscale) continue
if (job.saving) continue
if (job.library?.chainContinuing) continue
if (job.status !== 'running' && job.status !== 'uploading' && job.status !== 'queued') continue
@@ -351,7 +351,7 @@ async function reapZombieLiveJobs() {
}
function liveJobOwnsGpu(job: Job) {
if ((job.studio2 || job.yueGp || job.upscale) && ['running', 'queued', 'uploading'].includes(job.status)) return true
if ((job.studio2 || job.yueGp || job.yue2 || job.upscale) && ['running', 'queued', 'uploading'].includes(job.status)) return true
if (job.library?.stopAfterCurrent) return false
if (job.library?.chainContinuing) return true
if (job.saving) return true
@@ -371,7 +371,7 @@ function liveJobOwnsGpu(job: Job) {
*/
function clearDeadGpuClaimsForForceStart() {
for (const live of listJobs()) {
if (live.studio2 || live.yueGp || live.upscale) continue
if (live.studio2 || live.yueGp || live.yue2 || live.upscale) continue
if (live.saving) continue
if (jobIsLocallySubmitting(live)) continue
if (live.status !== 'running' && live.status !== 'queued' && live.status !== 'uploading') {
@@ -598,6 +598,10 @@ export async function clearStuckStudioWork(owner: string) {
const { cancelYueGpJob } = await import('./yueGp')
await cancelYueGpJob(job)
}
if (job.yue2) {
const { cancelYue2Job } = await import('./yue2')
await cancelYue2Job(job)
}
job.status = 'cancelled'
job.error = 'Cleared by force reset'
if (job.library) {
@@ -665,6 +669,11 @@ async function stopLiveGeneration(liveJobId?: string, shotQueueId?: string) {
await cancelYueGpJob(live)
return
}
if (live?.yue2) {
const { cancelYue2Job } = await import('./yue2')
await cancelYue2Job(live)
return
}
if (live) {
live.status = 'cancelled'
if (live.library) {
@@ -914,7 +923,7 @@ function repairStaleJobs(jobs: StudioJob[]) {
job.updatedAt = Date.now()
continue
}
if (!job.payload.upscale && job.payload.musicEngine !== 'yue' && job.status === 'error' && isTransientComfyError(job.lastError)) {
if (!job.payload.upscale && job.payload.musicEngine !== 'yue' && job.payload.musicEngine !== 'yue2' && job.status === 'error' && isTransientComfyError(job.lastError)) {
job.status = 'waiting'
job.liveJobId = undefined
job.lastError = undefined
@@ -923,7 +932,7 @@ function repairStaleJobs(jobs: StudioJob[]) {
job.updatedAt = Date.now()
continue
}
if (!job.payload.upscale && job.payload.musicEngine !== 'yue' && job.status === 'held' && isTransientComfyError(job.lastError)) {
if (!job.payload.upscale && job.payload.musicEngine !== 'yue' && job.payload.musicEngine !== 'yue2' && job.status === 'held' && isTransientComfyError(job.lastError)) {
job.status = 'waiting'
job.liveJobId = undefined
job.lastError = undefined
@@ -1687,7 +1696,7 @@ export async function onLiveVideoSettled(job: Job) {
return
}
const remaining = remainingStudioShots(job)
const wakeFail = !job.studio2 && !job.yueGp && !job.upscale && job.status === 'error' && remaining > 0 && isTransientComfyError(job.error)
const wakeFail = !job.studio2 && !job.yueGp && !job.yue2 && !job.upscale && job.status === 'error' && remaining > 0 && isTransientComfyError(job.error)
const failed = (job.status === 'error' || job.status === 'cancelled') && !wakeFail
await mutateStore(owner, (store) => {
+148
View File
@@ -0,0 +1,148 @@
import { existsSync, mkdirSync, readdirSync, readFileSync, writeFileSync, renameSync, unlinkSync } from 'node:fs'
import { join } from 'node:path'
import { createJob, emitJob, getJob, restoreMusicJob, type Job } from './jobs'
import { saveTrack, findYue2Track } from './library'
import { sharedGpuHeaders } from './sharedGpu'
import type { MusicJobParams } from './musicChain'
function settings() {
const config = useRuntimeConfig()
return { url: String(config.comfyControlUrl || process.env.COMFY_CONTROL_URL || '').replace(/\/$/, ''),
token: String(config.comfyControlToken || process.env.COMFY_CONTROL_TOKEN || '') }
}
export function yue2Configured() { return Boolean(settings().url) }
async function request(path: string, body?: unknown) {
const { url, token } = settings()
if (!url) throw new Error('YuE2 host is not configured. Set COMFY_CONTROL_URL.')
const response = await fetch(`${url}/yue2/${path}`, {
method: body === undefined ? 'GET' : 'POST',
headers: { ...(token ? { Authorization: `Bearer ${token}` } : {}),
...(body === undefined ? {} : { ...sharedGpuHeaders(), 'Content-Type': 'application/json' }) },
body: body === undefined ? undefined : JSON.stringify(body), signal: AbortSignal.timeout(60_000)
})
if (!response.ok) {
const detail = await response.json().catch(() => ({})) as { message?: string; error?: string }
throw Object.assign(new Error(detail.message || detail.error || `YuE2 host returned ${response.status}`), { statusCode: response.status })
}
return response
}
function pendingRoot() { return join(String(useRuntimeConfig().libraryDir || process.env.LIBRARY_DIR || '/data/library'), 'yue2-pending') }
function persist(job: Job) {
mkdirSync(pendingRoot(), { recursive: true })
const path = join(pendingRoot(), `${job.id}.json`)
writeFileSync(path + '.tmp', JSON.stringify({ id: job.id, clientId: job.clientId, startedAt: job.startedAt, library: job.library, trackId: job.trackId }))
renameSync(path + '.tmp', path)
}
async function settle(job: Job) {
const { onLiveVideoSettled } = await import('./studioQueue')
await onLiveVideoSettled(job)
const path = join(pendingRoot(), `${job.id}.json`)
if (existsSync(path)) unlinkSync(path)
}
export async function cancelYue2Job(job: Job) {
await request(`jobs/${job.id}/cancel`, {})
job.status = 'cancelled'
emitJob(job, { type: 'error', error: 'Cancelled', message: 'Cancelled' })
}
async function watch(job: Job) {
let failures = 0
while (job.status !== 'cancelled') {
try {
const state = await (await request(`jobs/${job.id}`)).json() as {
status: string; message: string; error?: string; stage?: string; progress?: number; step?: number; maxStep?: number; duration?: number
}
if (job.status === 'cancelled') { await settle(job); return }
job.musicActivity = { checkedAt: Date.now(), running: ['starting', 'running', 'cancelling'].includes(state.status) }
if (state.status === 'error' || state.status === 'cancelled') {
job.status = state.status === 'cancelled' ? 'cancelled' : 'error'
job.error = state.error || state.message
if (/out of memory/i.test(job.error || '')) job.error += ' YuE2 does not switch engines automatically.'
emitJob(job, { type: 'error', error: job.error, message: job.error })
await settle(job)
return
}
if (state.status === 'complete') {
job.saving = true
emitJob(job, { type: 'status', message: 'Saving audio to library…', progress: 98 })
const lib = job.library!
lib.audioExt = 'wav'
job.trackId ||= findYue2Track(lib.ownerKey, job.id)?.id
if (!job.trackId) {
const audio = Buffer.from(await (await request(`jobs/${job.id}/audio`)).arrayBuffer())
const track = await saveTrack({ ownerKey: lib.ownerKey, folderId: lib.folderId, name: lib.name,
tags: lib.tags || lib.prompt, lyrics: lib.lyrics || '', duration: state.duration || lib.duration || 60,
seed: lib.seed, steps: 0, cfg: 0, instrumental: lib.instrumental === true,
engine: 'yue2', audio, ext: 'wav', comfyFilename: `yue2-${job.id}.wav` })
job.trackId = track.id
persist(job)
}
job.status = 'complete'; job.saving = false
emitJob(job, { type: 'complete', message: lib.folderLocked ? 'Saved to the locked folder.' : 'Track ready',
progress: 100, trackId: job.trackId, folderLocked: lib.folderLocked, audioExt: 'wav' })
await settle(job)
return
}
failures = 0
job.status = 'running'
const percent = Number(state.progress || 0)
emitJob(job, { type: 'progress', message: `${state.message}${state.maxStep ? ` · ${state.step}/${state.maxStep}` : ''}`,
progress: Math.min(95, Math.max(1, percent)), step: state.step || 0, maxStep: state.maxStep || 0 })
} catch (error) {
failures++
job.saving = false
emitJob(job, { type: 'status', message: `YuE2 connection/save check failed; retrying: ${error instanceof Error ? error.message : String(error)}` })
if (failures >= 10) {
job.status = 'error'; job.error = 'YuE2 host unreachable or audio save failed. Job files are retained for recovery.'
emitJob(job, { type: 'error', error: job.error, message: job.error })
const { onLiveVideoSettled } = await import('./studioQueue')
await onLiveVideoSettled(job)
return
}
}
await new Promise(resolve => setTimeout(resolve, 2000))
}
await settle(job)
}
export function startYue2Job(params: MusicJobParams) {
const job = createJob('music')
job.yue2 = true
job.library = { ...params, prompt: params.tags, engine: 'yue2', aspect: 'audio', width: 0, height: 0,
hideThumbnail: false, turbo: false, sound: true }
persist(job)
setTimeout(() => { void (async () => {
try {
if (job.status === 'cancelled') { await settle(job); return }
await request('jobs', { id: job.id, tags: params.tags, lyrics: params.lyrics,
duration: params.duration, seed: params.seed })
await watch(job)
} catch (error) {
const statusCode = (error as { statusCode?: number })?.statusCode
if (statusCode && statusCode >= 400 && statusCode < 500) {
job.status = 'error'; job.error = error instanceof Error ? error.message : String(error)
emitJob(job, { type: 'error', error: job.error, message: job.error })
await settle(job)
return
}
emitJob(job, { type: 'status', message: `Checking YuE2 submission: ${error instanceof Error ? error.message : String(error)}` })
await watch(job)
}
})() }, 0)
return job
}
export function resumeYue2Jobs() {
if (!existsSync(pendingRoot())) return
for (const file of readdirSync(pendingRoot()).filter(file => /^[a-zA-Z0-9-]+\.json$/.test(file))) {
try {
const record = JSON.parse(readFileSync(join(pendingRoot(), file), 'utf8'))
if (getJob(record.id)) continue
const job = restoreMusicJob(record)
job.yueGp = false
job.yue2 = true
job.message = 'Reconnecting to YuE2'
void watch(job)
} catch { /* Preserve invalid records for diagnosis. */ }
}
}
+21
View File
@@ -0,0 +1,21 @@
"""CPU-only YuE2 worker regression tests; never import torch or run inference."""
import importlib.util
from pathlib import Path
import unittest
spec = importlib.util.spec_from_file_location('yue2_worker', Path(__file__).parents[1] / 'scripts/yue2-worker.py')
worker = importlib.util.module_from_spec(spec)
spec.loader.exec_module(worker)
class WorkerTests(unittest.TestCase):
def test_lyrics_preserve_words_and_normalize_ui_headings(self):
self.assertEqual(worker.normalize_lyrics('[Pre-Chorus]\nEvery word stays\n[Outro]\n'), '[prechorus]\nEvery word stays\n\n')
self.assertEqual(worker.normalize_lyrics('[Verse 1]\nHello'), '[verse1]\nHello\n\n')
for lyrics in ['', 'No heading', '[Verse]\nA\n[Chorus]\nB']:
with self.assertRaises(ValueError):
worker.normalize_lyrics(lyrics)
if __name__ == '__main__':
unittest.main()
+95
View File
@@ -0,0 +1,95 @@
import test from 'node:test'
import assert from 'node:assert/strict'
import { EventEmitter } from 'node:events'
import { PassThrough } from 'node:stream'
import { mkdtempSync, writeFileSync, readFileSync, existsSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import ts from 'typescript'
import { createYue2Host, validateYue2Request } from '../scripts/yue2-host.mjs'
import { createYueGpHost, validateYueGpRequest } from '../scripts/yuegp-host.mjs'
const request = { id: 'test-song-123456', tags: 'pop, warm vocals', lyrics: '[Verse 1]\nA quiet morning, a new day', seed: 0 }
test('YuE2 defaults are 60 seconds and keep Yue lyric validation', () => {
assert.deepEqual(validateYue2Request(request), { ...request, duration: 60 })
assert.throws(() => validateYue2Request({ ...request, lyrics: '[Verse]\nA\n[Chorus]\nB' }))
assert.throws(() => validateYue2Request({ ...request, duration: 20 }))
assert.throws(() => validateYue2Request({ ...request, id: '../escape' }))
})
test('yue still validates as YuEGP and stays separate from yue2', () => {
assert.equal(validateYueGpRequest(request).profile, 1)
assert.equal(validateYue2Request(request).duration, 60)
assert.notEqual(createYueGpHost.name, createYue2Host.name)
})
function fixture() {
const root = mkdtempSync(join(tmpdir(), 'aigen-yue2-'))
const python = join(root, 'python.exe')
writeFileSync(python, '')
writeFileSync(join(root, 'aigen-ready.json'), JSON.stringify({ model: 'm-a-p/YuE2-3B', vae: 'm-a-p/YuE2-Vae' }))
const child = new EventEmitter()
child.stdout = new PassThrough(); child.stderr = new PassThrough()
child.kill = () => { setImmediate(() => child.emit('close', 1)); return true }
const calls = []
const dataDir = join(root, 'jobs')
const host = createYue2Host({ root, python, dataDir, prepare: async () => {}, leaseValid: () => true,
spawnProcess: (...args) => { calls.push(args); return child } })
return { host, child, calls, root, dataDir }
}
test('host launches yue2-worker, preserves seed 0 and lyrics, and deduplicates POST retries', async () => {
const f = fixture()
await f.host.start(request, 'lease')
await f.host.start(request, 'lease')
assert.equal(f.calls.length, 1)
const [, args, opts] = f.calls[0]
assert.ok(String(args[1]).endsWith('yue2-worker.py'))
assert.equal(opts.windowsHide, true)
assert.equal(opts.shell, false)
const posted = JSON.parse(readFileSync(join(f.dataDir, request.id, 'request.json')))
assert.equal(posted.seed, 0)
assert.equal(posted.lyrics, request.lyrics)
assert.equal(posted.model, 'm-a-p/YuE2-3B')
assert.equal(posted.vae, 'm-a-p/YuE2-Vae')
await assert.rejects(f.host.start({ ...request, id: 'second-song-12345' }, 'lease'), /already running/)
await f.host.cancel(request.id)
assert.equal(f.host.busy(), false)
assert.equal(f.host.read(request.id).status, 'cancelled')
})
test('OOM stays on yue2 with no engine switch', async () => {
const f = fixture()
await f.host.start(request, 'lease')
f.child.stdout.write('AIGEN_EVENT {"stage":"error","error":"CUDA out of memory"}\n')
f.child.emit('close', 1)
assert.match(f.host.read(request.id).error, /out of memory/)
assert.equal(f.calls.length, 1)
})
test('music workflow refuses yue and yue2 while ACE graphs still build', () => {
const source = readFileSync(new URL('../server/utils/musicWorkflow.ts', import.meta.url), 'utf8')
const code = ts.transpileModule(source, { compilerOptions: { target: ts.ScriptTarget.ES2022, module: ts.ModuleKind.CommonJS } }).outputText
const exports = {}
new Function('require', 'exports', 'useRuntimeConfig', code)(id => {
if (id.endsWith('.json')) return { default: JSON.parse(readFileSync(new URL(`../server/assets/${id.split('/').pop()}`, import.meta.url))) }
if (id === '~/utils/music') return { parseMusicBpm: () => 120 }
throw new Error(id)
}, exports, () => ({}))
const params = { tags: 'pop', lyrics: '[Verse]\nhello', duration: 60, steps: 8, seed: 0, cfg: 1, lyricsStrength: 0.9 }
assert.throws(() => exports.buildMusicWorkflow({ ...params, engine: 'yue' }), /standalone YuEGP/)
assert.throws(() => exports.buildMusicWorkflow({ ...params, engine: 'yue2' }), /standalone YuE2/)
for (const engine of ['ace-step', 'ace-step-1.5']) {
const graph = exports.buildMusicWorkflow({ ...params, engine })
assert.ok(Object.values(graph).some(n => n.class_type === 'KSampler' && n.inputs.seed === 0))
}
assert.equal(existsSync(new URL('../server/assets/workflow_yue.json', import.meta.url)), false)
})
test('musicChain routes yue2 to startYue2Job and leaves yue on YuEGP', () => {
const source = readFileSync(new URL('../server/utils/musicChain.ts', import.meta.url), 'utf8')
assert.match(source, /engine === 'yue'\) return startYueGpJob/)
assert.match(source, /engine === 'yue2'\) return startYue2Job/)
assert.match(source, /from '\.\/yue2'/)
})
+5 -2
View File
@@ -33,22 +33,25 @@ export const DEFAULT_MUSIC_LYRICS = `[Verse 1]
[Outro]
`
export type MusicEngine = 'ace-step' | 'ace-step-1.5' | 'yue'
export type MusicEngine = 'ace-step' | 'ace-step-1.5' | 'yue' | 'yue2'
export function parseMusicEngine(raw: unknown): MusicEngine {
const value = String(raw || '').trim().toLowerCase()
if (value === 'ace-step-1.5' || value === 'ace15' || value === 'v1.5') return 'ace-step-1.5'
if (value === 'yue2' || value === 'yue-2') return 'yue2'
if (value === 'yue' || value === 'yue-s1' || value === 'lyrics2song') return 'yue'
return 'ace-step'
}
export function isAceMusicEngine(engine: MusicEngine | string | undefined) {
return parseMusicEngine(engine) !== 'yue'
const value = parseMusicEngine(engine)
return value !== 'yue' && value !== 'yue2'
}
export function musicEngineLabel(engine: MusicEngine | string | undefined) {
const value = parseMusicEngine(engine)
if (value === 'ace-step-1.5') return 'ACE-Step 1.5'
if (value === 'yue2') return 'YuE2'
if (value === 'yue') return 'YuE'
return 'ACE-Step'
}