Finish YuE2 host setup, weight download paths, and v1 cleanup.
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Towsty
+49
-55
@@ -1,29 +1,41 @@
|
||||
# YuE2 music engine
|
||||
|
||||
`engine: "yue2"` is standalone [YuE2](https://github.com/multimodal-art-projection/YuE) (`m-a-p/YuE2-3B` + `m-a-p/YuE2-Vae`). It never submits a Comfy graph. `engine: "yue"` remains YuEGP. ACE / ACE 1.5 keep their existing Comfy workflows.
|
||||
`engine: "yue2"` is standalone [YuE2](https://github.com/multimodal-art-projection/YuE) (`m-a-p/YuE2-3B` + `m-a-p/YuE2-Vae`). It never submits a Comfy graph. `engine: "yue"` remains YuEGP in code. ACE / ACE 1.5 keep their Comfy workflows.
|
||||
|
||||
## Install once (desktop RTX 5080 host)
|
||||
## What this host already has (5080 desktop)
|
||||
|
||||
```powershell
|
||||
# From the aigen checkout on the GPU PC
|
||||
.\scripts\setup-yue2.ps1 -Uv <path-to-uv.exe>
|
||||
```
|
||||
Ran on this machine:
|
||||
|
||||
Default checkout: `C:\Users\ianjm\Development\YuE2` (isolated venv, not Comfy Python). Setup does not download model weights; Hugging Face pulls them on the first song.
|
||||
1. `scripts/setup-yue2.ps1` → checkout + venv + CUDA torch + weight download
|
||||
2. `scripts/cleanup-yue-v1.ps1` → named YuE v1 / YuEGP weight folders removed
|
||||
3. Host agent code mounts `POST/GET /yue2/jobs` (same pattern as `/yuegp/jobs`)
|
||||
|
||||
Restart the existing host connector (`comfy-host-agent`) after pulling these scripts so `/yue2/jobs` is mounted. Do not restart Coolify or Comfy for this wiring.
|
||||
Paths written:
|
||||
|
||||
| Item | Path |
|
||||
| --- | --- |
|
||||
| `YUE2_ROOT` | `C:\Users\ianjm\Development\YuE2` |
|
||||
| Python | `C:\Users\ianjm\Development\YuE2\.venv\Scripts\python.exe` |
|
||||
| Model | `C:\Users\ianjm\Development\YuE2\models\YuE2-3B` (~7.3 GB) |
|
||||
| VAE | `C:\Users\ianjm\Development\YuE2\models\YuE2-Vae` (~531 MB) |
|
||||
| Ready file | `C:\Users\ianjm\Development\YuE2\aigen-ready.json` |
|
||||
| Jobs dir default | `C:\Users\ianjm\Development\YuE2\aigen-jobs` |
|
||||
|
||||
`aigen-ready.json` points `model` / `vae` at those local folders so Generate does not need a first-click HF pull. CUDA check: `torch 2.11.0+cu128` on `NVIDIA GeForce RTX 5080`.
|
||||
|
||||
After deploying this branch, restart the **desktop host connector** once so it loads `yue2-host.mjs`. Do not restart Coolify from the agent; the user triggers the existing xAIGen-dev Coolify rebuild.
|
||||
|
||||
## Env vars
|
||||
|
||||
| Variable | Purpose |
|
||||
| --- | --- |
|
||||
| `YUE2_ROOT` | Standalone YuE checkout (default `C:\Users\ianjm\Development\YuE2`) |
|
||||
| `YUE2_PYTHON` | Isolated worker Python (`YUE2_ROOT\.venv\Scripts\python.exe`) |
|
||||
| `YUE2_JOBS_DIR` | Per-job logs, request.json, intermediates, WAV |
|
||||
| `YUE2_MODEL` | Default `m-a-p/YuE2-3B` |
|
||||
| `YUE2_VAE` | Default `m-a-p/YuE2-Vae` |
|
||||
| `YUE2_ROOT` | Standalone YuE checkout |
|
||||
| `YUE2_PYTHON` | Isolated worker Python |
|
||||
| `YUE2_JOBS_DIR` | Per-job logs / request / WAV |
|
||||
| `YUE2_MODEL` | Local model dir or HF id |
|
||||
| `YUE2_VAE` | Local VAE dir or HF id |
|
||||
|
||||
App still uses `COMFY_CONTROL_URL` + token. YuE2 shares the same GPU reservation: Comfy is stopped while a YuE2 job owns the card. One song at a time. No `keep_model_loaded`. OOM is a visible error; the app does not fall back to `yue` or ACE.
|
||||
App uses existing `COMFY_CONTROL_URL` + token. One GPU owner via the shared reservation (Comfy stopped while YuE2 runs). One song. Unload after decode. OOM is visible; no fallback to `yue`.
|
||||
|
||||
## Call path
|
||||
|
||||
@@ -31,54 +43,36 @@ App still uses `COMFY_CONTROL_URL` + token. YuE2 shares the same GPU reservation
|
||||
{"engine":"yue2","duration":60,"seed":42,"tags":"pop, warm vocals","lyrics":"[Verse 1]\nYour lyrics here","folderId":"<library-folder>","name":"My song"}
|
||||
```
|
||||
|
||||
`startMusicJob` → `startYue2Job` → `POST <COMFY_CONTROL_URL>/yue2/jobs` → `yue2-worker.py` (`plan` → `generate_semantic` → `synthesize` → `decode`) → library WAV.
|
||||
Picker → `POST /api/generate/music` → `startYue2Job` → `POST <COMFY_CONTROL_URL>/yue2/jobs` → `yue2-worker.py` (`plan` → `generate_semantic` → `synthesize` → `decode`) → library WAV.
|
||||
|
||||
Duration clamp is 30–150 seconds (default 60), same as Yue field validation: one non-empty lyric section. Upstream song length follows the plan; duration is validated and stored for library metadata.
|
||||
Duration 30–150 s (default 60). One non-empty lyric section (same validation as Yue).
|
||||
|
||||
## Success check
|
||||
## Old YuE / YuEGP data deleted on this host
|
||||
|
||||
1. Music page Model picker shows YuE2.
|
||||
2. POST with `engine:"yue2"` hits `/yue2/jobs` (host log), not Comfy queue.
|
||||
3. `engine:"yue"` still hits `/yuegp/jobs`.
|
||||
4. ACE graphs unchanged.
|
||||
5. After a real 5080 run: WAV in library, worker unloaded, Comfy can wake again.
|
||||
`scripts/cleanup-yue-v1.ps1` logged deletions to `scripts/cleanup-yue-v1.log`. Deleted:
|
||||
|
||||
This agent session did not generate a song.
|
||||
- `...\ComfyUI-Shared\models\yue\YuE-s1-7B-anneal-en-cot`
|
||||
- `...\ComfyUI-Shared\models\yue\YuE-s2-1B-general`
|
||||
- `...\ComfyUI-Shared\models\yue\xcodec_mini_infer`
|
||||
- `...\ComfyUI-Shared\models\yue\ckpt_00360000.pth`
|
||||
- `...\ComfyUI-Shared\models\yue\decoder_131000.pth`
|
||||
- `...\ComfyUI-Shared\models\yue\decoder_151000.pth`
|
||||
- `C:\Users\ianjm\Development\YuEGP\inference\xcodec_mini_infer`
|
||||
- `C:\Users\ianjm\Development\YuEGP\aigen-jobs`
|
||||
|
||||
## Remove old YuE / YuEGP data only (host checklist)
|
||||
HF cache YuE-s1 / YuE-s2 / xcodec entries were already absent (skipped). ACE checkpoints (`ace_step_v1_3.5b.safetensors`, `ace_step_1.5_turbo_aio.safetensors`) and other Comfy model trees were left alone. `scripts/yuegp-*` and `server/utils/yueGp.ts` remain in git.
|
||||
|
||||
Run these **only after** YuE2 has produced a real song and you no longer need YuEGP. Named folders only. Do **not** recursively delete Comfy `models/` or `custom_nodes/`.
|
||||
|
||||
Adjust paths if your `YUEGP_ROOT` / HF cache differ. Review each path before deleting.
|
||||
## Re-run setup (only if rebuilding the machine)
|
||||
|
||||
```powershell
|
||||
# --- YuEGP checkout weights / jobs (not the aigen scripts) ---
|
||||
# Example defaults from setup-yuegp.ps1 / docs/yuegp-replacement.md
|
||||
$YueGpRoot = 'C:\Users\ianjm\Development\YuEGP'
|
||||
$YueGpJobs = Join-Path $YueGpRoot 'aigen-jobs'
|
||||
$YueModels = Join-Path $env:LOCALAPPDATA 'Comfy-Desktop\ComfyUI-Shared\models\yue'
|
||||
|
||||
# Stage 1 / Stage 2 local folders (only if clearly YuE v1)
|
||||
Remove-Item -LiteralPath (Join-Path $YueModels 'YuE-s1-7B-anneal-en-cot') -Recurse -Force -ErrorAction SilentlyContinue
|
||||
Remove-Item -LiteralPath (Join-Path $YueModels 'YuE-s2-1B-general') -Recurse -Force -ErrorAction SilentlyContinue
|
||||
|
||||
# Old codec under YuEGP inference (YuEGP-only copy)
|
||||
Remove-Item -LiteralPath (Join-Path $YueGpRoot 'inference\xcodec_mini_infer') -Recurse -Force -ErrorAction SilentlyContinue
|
||||
|
||||
# Old YuEGP song dumps
|
||||
Remove-Item -LiteralPath $YueGpJobs -Recurse -Force -ErrorAction SilentlyContinue
|
||||
|
||||
# Hugging Face cache folders that are clearly YuE v1 / old codec only
|
||||
$Hf = Join-Path $env:USERPROFILE '.cache\huggingface\hub'
|
||||
Remove-Item -LiteralPath (Join-Path $Hf 'models--m-a-p--YuE-s1-7B-anneal-en-cot') -Recurse -Force -ErrorAction SilentlyContinue
|
||||
Remove-Item -LiteralPath (Join-Path $Hf 'models--m-a-p--YuE-s2-1B-general') -Recurse -Force -ErrorAction SilentlyContinue
|
||||
Remove-Item -LiteralPath (Join-Path $Hf 'models--m-a-p--xcodec_mini_infer') -Recurse -Force -ErrorAction SilentlyContinue
|
||||
|
||||
# Leftover Comfy YuE Stage A/B weight copies (YuE-only names). Leave ACE / H3 / Klein / Krea / LTX / upscale alone.
|
||||
$ComfyYue = Join-Path $env:LOCALAPPDATA 'Comfy-Desktop\ComfyUI-Shared\models'
|
||||
Get-ChildItem -LiteralPath $ComfyYue -Recurse -ErrorAction SilentlyContinue |
|
||||
Where-Object { $_.Name -match 'YUE_Stage_|YuE-s1-|YuE-s2-|yue_stage' } |
|
||||
ForEach-Object { Remove-Item -LiteralPath $_.FullName -Recurse -Force -ErrorAction SilentlyContinue }
|
||||
.\scripts\setup-yue2.ps1
|
||||
.\scripts\cleanup-yue-v1.ps1
|
||||
```
|
||||
|
||||
**Do not delete:** ACE-Step weights, ACE workflow JSON, MiniMax H3 / LTX / Flux Klein / Krea assets, LoRAs, `LIBRARY_DIR`, Studio 2 graphs, or `scripts/yuegp-*` / `server/utils/yueGp.ts` until YuE2 has proven a real song. Data off disk ≠ rip out the `engine:"yue"` code path.
|
||||
Setup reuses existing `models\YuE2-3B` / `YuE2-Vae` when present, then force-reinstalls CUDA torch after the editable package install.
|
||||
|
||||
## Validation
|
||||
|
||||
- JS: `node --test tests/yue2.test.mjs tests/yuegp.test.mjs`
|
||||
- Python: `py -3 -m unittest tests.test_yue2_worker tests.test_yuegp_worker`
|
||||
- No full song was generated in this agent session.
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
# Delete named YuE v1 / YuEGP weight folders only. Never recurse-wipe Comfy models\ or custom_nodes\.
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[string]$YueGpRoot = 'C:\Users\ianjm\Development\YuEGP',
|
||||
[string]$YueModels = '',
|
||||
[string]$LogPath = ''
|
||||
)
|
||||
$ErrorActionPreference = 'Continue'
|
||||
if (-not $YueModels) {
|
||||
$YueModels = Join-Path $env:LOCALAPPDATA 'Comfy-Desktop\ComfyUI-Shared\models\yue'
|
||||
}
|
||||
if (-not $LogPath) {
|
||||
$LogPath = Join-Path $PSScriptRoot 'cleanup-yue-v1.log'
|
||||
}
|
||||
function Log([string]$Message) {
|
||||
$line = "$(Get-Date -Format o) $Message"
|
||||
Add-Content -LiteralPath $LogPath -Value $line
|
||||
Write-Host $line
|
||||
}
|
||||
function RemoveNamed([string]$Path) {
|
||||
if (-not $Path) { return }
|
||||
if (!(Test-Path -LiteralPath $Path)) {
|
||||
Log "SKIP missing $Path"
|
||||
return
|
||||
}
|
||||
try {
|
||||
Remove-Item -LiteralPath $Path -Recurse -Force
|
||||
Log "DELETED $Path"
|
||||
} catch {
|
||||
Log "FAIL $Path :: $($_.Exception.Message)"
|
||||
}
|
||||
}
|
||||
|
||||
Log "START cleanup-yue-v1 YueGpRoot=$YueGpRoot YueModels=$YueModels"
|
||||
|
||||
# Stage 1 / Stage 2 + codec under the shared yue models dir (named folders only)
|
||||
RemoveNamed (Join-Path $YueModels 'YuE-s1-7B-anneal-en-cot')
|
||||
RemoveNamed (Join-Path $YueModels 'YuE-s2-1B-general')
|
||||
RemoveNamed (Join-Path $YueModels 'xcodec_mini_infer')
|
||||
# Loose codec files that live beside the stage folders in this install
|
||||
foreach ($name in @('ckpt_00360000.pth', 'decoder_131000.pth', 'decoder_151000.pth')) {
|
||||
RemoveNamed (Join-Path $YueModels $name)
|
||||
}
|
||||
|
||||
# YuEGP checkout codec copy + job dumps
|
||||
RemoveNamed (Join-Path $YueGpRoot 'inference\xcodec_mini_infer')
|
||||
RemoveNamed (Join-Path $YueGpRoot 'aigen-jobs')
|
||||
|
||||
# Hugging Face cache dirs that are clearly YuE v1 / old codec
|
||||
$Hf = Join-Path $env:USERPROFILE '.cache\huggingface\hub'
|
||||
RemoveNamed (Join-Path $Hf 'models--m-a-p--YuE-s1-7B-anneal-en-cot')
|
||||
RemoveNamed (Join-Path $Hf 'models--m-a-p--YuE-s2-1B-general')
|
||||
RemoveNamed (Join-Path $Hf 'models--m-a-p--xcodec_mini_infer')
|
||||
|
||||
# Leftover Comfy YuE Stage A/B weight copies by name only (no blanket models\ purge)
|
||||
$ComfyModels = Join-Path $env:LOCALAPPDATA 'Comfy-Desktop\ComfyUI-Shared\models'
|
||||
if (Test-Path -LiteralPath $ComfyModels) {
|
||||
Get-ChildItem -LiteralPath $ComfyModels -Recurse -ErrorAction SilentlyContinue |
|
||||
Where-Object { $_.Name -match '^(YUE_Stage_|YuE-s1-|YuE-s2-|yue_stage)' } |
|
||||
ForEach-Object { RemoveNamed $_.FullName }
|
||||
} else {
|
||||
Log "SKIP missing $ComfyModels"
|
||||
}
|
||||
|
||||
Log "DONE cleanup-yue-v1 log=$LogPath"
|
||||
Write-Host "Left in place: ACE / H3 / Klein / Krea / LTX / upscale / LoRAs / LIBRARY_DIR / yuegp code paths."
|
||||
@@ -55,7 +55,7 @@ export function createGpuProxy({ target, reservation, authorized = () => true, m
|
||||
try {
|
||||
if (req.method === 'GET' || req.method === 'HEAD') await forward()
|
||||
else {
|
||||
if (externalBusy()) { req.resume(); return reply(res, 409, { error: { message: 'YuEGP is using the GPU.' }, code: 'GPU_BUSY' }) }
|
||||
if (externalBusy()) { req.resume(); return reply(res, 409, { error: { message: 'A music or upscale job is using the GPU.' }, code: 'GPU_BUSY' }) }
|
||||
if (!authorized(req)) { req.resume(); return reply(res, 401, { error: 'Unauthorized' }) }
|
||||
await reservation.permit(String(req.headers['x-aigen-gpu-lease'] || ''), async () => { markWork(); await forward() })
|
||||
}
|
||||
|
||||
+48
-5
@@ -1,23 +1,66 @@
|
||||
# Standalone YuE2 only. Never install into Comfy's Python environment.
|
||||
# Downloads YuE2-3B + YuE2-Vae into YUE2_ROOT\models so the UI does not hit HF on first Generate.
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[string]$Root = 'C:\Users\ianjm\Development\YuE2',
|
||||
[string]$Uv = 'uv'
|
||||
[string]$Uv = 'uv',
|
||||
[string]$Models = ''
|
||||
)
|
||||
$ErrorActionPreference = 'Stop'
|
||||
$Root = [IO.Path]::GetFullPath($Root)
|
||||
if (-not $Models) { $Models = Join-Path $Root 'models' }
|
||||
$Models = [IO.Path]::GetFullPath($Models)
|
||||
function Run([string]$Exe, [string[]]$Arguments) {
|
||||
& $Exe @Arguments
|
||||
if ($LASTEXITCODE -ne 0) { throw "$Exe failed ($LASTEXITCODE)" }
|
||||
}
|
||||
Write-Host "YuE2 setup root=$Root models=$Models"
|
||||
if (!(Test-Path -LiteralPath $Root)) {
|
||||
Run git @('clone', 'https://github.com/multimodal-art-projection/YuE.git', $Root)
|
||||
}
|
||||
Run $Uv @('venv', '--allow-existing', '--python', '3.12', (Join-Path $Root '.venv'))
|
||||
$python = Join-Path $Root '.venv\Scripts\python.exe'
|
||||
Run $Uv @('pip', 'install', '--python', $python, 'torch', 'torchaudio', '--index-url', 'https://download.pytorch.org/whl/cu128')
|
||||
# Install the package first, then force CUDA torch back on (editable deps can pull CPU wheels).
|
||||
Run $Uv @('pip', 'install', '--python', $python, '-e', $Root)
|
||||
Run $Uv @('pip', 'install', '--python', $python, 'soundfile', 'psutil')
|
||||
Run $Uv @('pip', 'install', '--python', $python, 'soundfile', 'psutil', 'huggingface_hub', 'hf_xet')
|
||||
Run $Uv @('pip', 'install', '--python', $python, 'torch', 'torchaudio', '--reinstall', '--index-url', 'https://download.pytorch.org/whl/cu128')
|
||||
New-Item -ItemType Directory -Force -Path $Models | Out-Null
|
||||
$modelDir = Join-Path $Models 'YuE2-3B'
|
||||
$vaeDir = Join-Path $Models 'YuE2-Vae'
|
||||
if (!(Test-Path -LiteralPath (Join-Path $modelDir 'config.json')) -and !(Test-Path -LiteralPath (Join-Path $modelDir 'model.safetensors.index.json'))) {
|
||||
Write-Host "Downloading m-a-p/YuE2-3B → $modelDir"
|
||||
Run $python @(
|
||||
'-c',
|
||||
'from huggingface_hub import snapshot_download; import sys; snapshot_download(sys.argv[1], local_dir=sys.argv[2])',
|
||||
'm-a-p/YuE2-3B',
|
||||
$modelDir
|
||||
)
|
||||
} else {
|
||||
Write-Host "Reusing existing model at $modelDir"
|
||||
}
|
||||
if (!(Test-Path -LiteralPath (Join-Path $vaeDir 'config.json'))) {
|
||||
Write-Host "Downloading m-a-p/YuE2-Vae → $vaeDir"
|
||||
Run $python @(
|
||||
'-c',
|
||||
'from huggingface_hub import snapshot_download; import sys; snapshot_download(sys.argv[1], local_dir=sys.argv[2])',
|
||||
'm-a-p/YuE2-Vae',
|
||||
$vaeDir
|
||||
)
|
||||
} else {
|
||||
Write-Host "Reusing existing VAE at $vaeDir"
|
||||
}
|
||||
Run $python @('-c', 'import torch; from yue2 import YuE2Pipeline; assert torch.cuda.is_available(); print(torch.__version__, torch.cuda.get_device_name(0))')
|
||||
@{ model='m-a-p/YuE2-3B'; vae='m-a-p/YuE2-Vae'; python=$python } | ConvertTo-Json | Set-Content -LiteralPath (Join-Path $Root 'aigen-ready.json') -Encoding utf8
|
||||
Write-Host "YuE2 ready at $Root. Set YUE2_ROOT / YUE2_PYTHON on the GPU host if needed. Weights download from Hugging Face on first song. No Comfy environment was modified."
|
||||
$ready = @{
|
||||
model = $modelDir
|
||||
vae = $vaeDir
|
||||
modelId = 'm-a-p/YuE2-3B'
|
||||
vaeId = 'm-a-p/YuE2-Vae'
|
||||
modelsDir = $Models
|
||||
python = $python
|
||||
root = $Root
|
||||
}
|
||||
$ready | ConvertTo-Json | Set-Content -LiteralPath (Join-Path $Root 'aigen-ready.json') -Encoding utf8
|
||||
Write-Host "YuE2 ready at $Root"
|
||||
Write-Host "Local model=$modelDir"
|
||||
Write-Host "Local vae=$vaeDir"
|
||||
Write-Host "Set YUE2_ROOT=$Root YUE2_PYTHON=$python YUE2_MODEL=$modelDir YUE2_VAE=$vaeDir if not using defaults. No Comfy environment was modified."
|
||||
|
||||
@@ -70,10 +70,12 @@ export function createYue2Host({ prepare, leaseValid, spawnProcess = spawn, root
|
||||
await prepare()
|
||||
if (run.cancelled || !leaseValid(lease)) throw new Error('YuE2 start cancelled or GPU reservation expired.')
|
||||
const installation = JSON.parse(readFileSync(join(repo, 'aigen-ready.json'), 'utf8').replace(/^\uFEFF/, ''))
|
||||
const localModel = process.env.YUE2_MODEL || installation.model || 'm-a-p/YuE2-3B'
|
||||
const localVae = process.env.YUE2_VAE || installation.vae || 'm-a-p/YuE2-Vae'
|
||||
writeFileSync(join(dir(job.id), 'request.json'), JSON.stringify({
|
||||
...request,
|
||||
model: process.env.YUE2_MODEL || installation.model || 'm-a-p/YuE2-3B',
|
||||
vae: process.env.YUE2_VAE || installation.vae || 'm-a-p/YuE2-Vae'
|
||||
model: localModel,
|
||||
vae: localVae
|
||||
}))
|
||||
const child = spawnProcess(executable, ['-u', worker, '--root', repo, '--request', join(dir(job.id), 'request.json')],
|
||||
{ cwd: repo, windowsHide: true, shell: false, stdio: ['ignore', 'pipe', 'pipe'], env: { ...process.env, PYTHONUTF8: '1', TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD: '1' } })
|
||||
|
||||
Reference in New Issue
Block a user