towsty/aigen
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Finish YuE2 host setup, weight download paths, and v1 cleanup.

Browse Source 
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Towsty
2026-09-14 18:41:00 -05:00
co-authored by Cursor
parent e82f179688
commit 35a83648d5
5 changed files with 168 additions and 63 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/yue2.md
+49 -55
View File
@@ -1,29 +1,41 @@
# YuE2 music engine
`engine: "yue2"` is standalone [YuE2](https://github.com/multimodal-art-projection/YuE) (`m-a-p/YuE2-3B` + `m-a-p/YuE2-Vae`). It never submits a Comfy graph. `engine: "yue"` remains YuEGP. ACE / ACE 1.5 keep their existing Comfy workflows.
`engine: "yue2"` is standalone [YuE2](https://github.com/multimodal-art-projection/YuE) (`m-a-p/YuE2-3B` + `m-a-p/YuE2-Vae`). It never submits a Comfy graph. `engine: "yue"` remains YuEGP in code. ACE / ACE 1.5 keep their Comfy workflows.
## Install once (desktop RTX 5080 host)
## What this host already has (5080 desktop)
```powershell
# From the aigen checkout on the GPU PC
.\scripts\setup-yue2.ps1 -Uv <path-to-uv.exe>
```
Ran on this machine:
Default checkout: `C:\Users\ianjm\Development\YuE2` (isolated venv, not Comfy Python). Setup does not download model weights; Hugging Face pulls them on the first song.
1. `scripts/setup-yue2.ps1` → checkout + venv + CUDA torch + weight download
2. `scripts/cleanup-yue-v1.ps1` → named YuE v1 / YuEGP weight folders removed
3. Host agent code mounts `POST/GET /yue2/jobs` (same pattern as `/yuegp/jobs`)
Restart the existing host connector (`comfy-host-agent`) after pulling these scripts so `/yue2/jobs` is mounted. Do not restart Coolify or Comfy for this wiring.
Paths written:
| Item | Path |
| --- | --- |
| `YUE2_ROOT` | `C:\Users\ianjm\Development\YuE2` |
| Python | `C:\Users\ianjm\Development\YuE2\.venv\Scripts\python.exe` |
| Model | `C:\Users\ianjm\Development\YuE2\models\YuE2-3B` (~7.3 GB) |
| VAE | `C:\Users\ianjm\Development\YuE2\models\YuE2-Vae` (~531 MB) |
| Ready file | `C:\Users\ianjm\Development\YuE2\aigen-ready.json` |
| Jobs dir default | `C:\Users\ianjm\Development\YuE2\aigen-jobs` |
`aigen-ready.json` points `model` / `vae` at those local folders so Generate does not need a first-click HF pull. CUDA check: `torch 2.11.0+cu128` on `NVIDIA GeForce RTX 5080`.
After deploying this branch, restart the **desktop host connector** once so it loads `yue2-host.mjs`. Do not restart Coolify from the agent; the user triggers the existing xAIGen-dev Coolify rebuild.
## Env vars
| Variable | Purpose |
| --- | --- |
| `YUE2_ROOT` | Standalone YuE checkout (default `C:\Users\ianjm\Development\YuE2`) |
| `YUE2_PYTHON` | Isolated worker Python (`YUE2_ROOT\.venv\Scripts\python.exe`) |
| `YUE2_JOBS_DIR` | Per-job logs, request.json, intermediates, WAV |
| `YUE2_MODEL` | Default `m-a-p/YuE2-3B` |
| `YUE2_VAE` | Default `m-a-p/YuE2-Vae` |
| `YUE2_ROOT` | Standalone YuE checkout |
| `YUE2_PYTHON` | Isolated worker Python |
| `YUE2_JOBS_DIR` | Per-job logs / request / WAV |
| `YUE2_MODEL` | Local model dir or HF id |
| `YUE2_VAE` | Local VAE dir or HF id |
App still uses `COMFY_CONTROL_URL` + token. YuE2 shares the same GPU reservation: Comfy is stopped while a YuE2 job owns the card. One song at a time. No `keep_model_loaded`. OOM is a visible error; the app does not fall back to `yue` or ACE.
App uses existing `COMFY_CONTROL_URL` + token. One GPU owner via the shared reservation (Comfy stopped while YuE2 runs). One song. Unload after decode. OOM is visible; no fallback to `yue`.
## Call path
@@ -31,54 +43,36 @@ App still uses `COMFY_CONTROL_URL` + token. YuE2 shares the same GPU reservation
{"engine":"yue2","duration":60,"seed":42,"tags":"pop, warm vocals","lyrics":"[Verse 1]\nYour lyrics here","folderId":"<library-folder>","name":"My song"}
```
`startMusicJob` → `startYue2Job` → `POST <COMFY_CONTROL_URL>/yue2/jobs` → `yue2-worker.py` (`plan` → `generate_semantic` → `synthesize` → `decode`) → library WAV.
Picker → `POST /api/generate/music` → `startYue2Job` → `POST <COMFY_CONTROL_URL>/yue2/jobs` → `yue2-worker.py` (`plan` → `generate_semantic` → `synthesize` → `decode`) → library WAV.
Duration clamp is 30–150 seconds (default 60), same as Yue field validation: one non-empty lyric section. Upstream song length follows the plan; duration is validated and stored for library metadata.
Duration 30–150 s (default 60). One non-empty lyric section (same validation as Yue).
## Success check
## Old YuE / YuEGP data deleted on this host
1. Music page Model picker shows YuE2.
2. POST with `engine:"yue2"` hits `/yue2/jobs` (host log), not Comfy queue.
3. `engine:"yue"` still hits `/yuegp/jobs`.
4. ACE graphs unchanged.
5. After a real 5080 run: WAV in library, worker unloaded, Comfy can wake again.
`scripts/cleanup-yue-v1.ps1` logged deletions to `scripts/cleanup-yue-v1.log`. Deleted:
This agent session did not generate a song.
- `...\ComfyUI-Shared\models\yue\YuE-s1-7B-anneal-en-cot`
- `...\ComfyUI-Shared\models\yue\YuE-s2-1B-general`
- `...\ComfyUI-Shared\models\yue\xcodec_mini_infer`
- `...\ComfyUI-Shared\models\yue\ckpt_00360000.pth`
- `...\ComfyUI-Shared\models\yue\decoder_131000.pth`
- `...\ComfyUI-Shared\models\yue\decoder_151000.pth`
- `C:\Users\ianjm\Development\YuEGP\inference\xcodec_mini_infer`
- `C:\Users\ianjm\Development\YuEGP\aigen-jobs`
## Remove old YuE / YuEGP data only (host checklist)
HF cache YuE-s1 / YuE-s2 / xcodec entries were already absent (skipped). ACE checkpoints (`ace_step_v1_3.5b.safetensors`, `ace_step_1.5_turbo_aio.safetensors`) and other Comfy model trees were left alone. `scripts/yuegp-*` and `server/utils/yueGp.ts` remain in git.
Run these **only after** YuE2 has produced a real song and you no longer need YuEGP. Named folders only. Do **not** recursively delete Comfy `models/` or `custom_nodes/`.
Adjust paths if your `YUEGP_ROOT` / HF cache differ. Review each path before deleting.
## Re-run setup (only if rebuilding the machine)
```powershell
# --- YuEGP checkout weights / jobs (not the aigen scripts) ---
# Example defaults from setup-yuegp.ps1 / docs/yuegp-replacement.md
$YueGpRoot = 'C:\Users\ianjm\Development\YuEGP'
$YueGpJobs = Join-Path $YueGpRoot 'aigen-jobs'
$YueModels = Join-Path $env:LOCALAPPDATA 'Comfy-Desktop\ComfyUI-Shared\models\yue'
# Stage 1 / Stage 2 local folders (only if clearly YuE v1)
Remove-Item -LiteralPath (Join-Path $YueModels 'YuE-s1-7B-anneal-en-cot') -Recurse -Force -ErrorAction SilentlyContinue
Remove-Item -LiteralPath (Join-Path $YueModels 'YuE-s2-1B-general') -Recurse -Force -ErrorAction SilentlyContinue
# Old codec under YuEGP inference (YuEGP-only copy)
Remove-Item -LiteralPath (Join-Path $YueGpRoot 'inference\xcodec_mini_infer') -Recurse -Force -ErrorAction SilentlyContinue
# Old YuEGP song dumps
Remove-Item -LiteralPath $YueGpJobs -Recurse -Force -ErrorAction SilentlyContinue
# Hugging Face cache folders that are clearly YuE v1 / old codec only
$Hf = Join-Path $env:USERPROFILE '.cache\huggingface\hub'
Remove-Item -LiteralPath (Join-Path $Hf 'models--m-a-p--YuE-s1-7B-anneal-en-cot') -Recurse -Force -ErrorAction SilentlyContinue
Remove-Item -LiteralPath (Join-Path $Hf 'models--m-a-p--YuE-s2-1B-general') -Recurse -Force -ErrorAction SilentlyContinue
Remove-Item -LiteralPath (Join-Path $Hf 'models--m-a-p--xcodec_mini_infer') -Recurse -Force -ErrorAction SilentlyContinue
# Leftover Comfy YuE Stage A/B weight copies (YuE-only names). Leave ACE / H3 / Klein / Krea / LTX / upscale alone.
$ComfyYue = Join-Path $env:LOCALAPPDATA 'Comfy-Desktop\ComfyUI-Shared\models'
Get-ChildItem -LiteralPath $ComfyYue -Recurse -ErrorAction SilentlyContinue |
Where-Object { $_.Name -match 'YUE_Stage_|YuE-s1-|YuE-s2-|yue_stage' } |
ForEach-Object { Remove-Item -LiteralPath $_.FullName -Recurse -Force -ErrorAction SilentlyContinue }
.\scripts\setup-yue2.ps1
.\scripts\cleanup-yue-v1.ps1
```
**Do not delete:** ACE-Step weights, ACE workflow JSON, MiniMax H3 / LTX / Flux Klein / Krea assets, LoRAs, `LIBRARY_DIR`, Studio 2 graphs, or `scripts/yuegp-*` / `server/utils/yueGp.ts` until YuE2 has proven a real song. Data off disk ≠ rip out the `engine:"yue"` code path.
Setup reuses existing `models\YuE2-3B` / `YuE2-Vae` when present, then force-reinstalls CUDA torch after the editable package install.
## Validation
- JS: `node --test tests/yue2.test.mjs tests/yuegp.test.mjs`
- Python: `py -3 -m unittest tests.test_yue2_worker tests.test_yuegp_worker`
- No full song was generated in this agent session.