85 lines
4.6 KiB
Markdown
85 lines
4.6 KiB
Markdown
# YuE2 music engine
|
||
|
||
`engine: "yue2"` is standalone [YuE2](https://github.com/multimodal-art-projection/YuE) (`m-a-p/YuE2-3B` + `m-a-p/YuE2-Vae`). It never submits a Comfy graph. `engine: "yue"` remains YuEGP. ACE / ACE 1.5 keep their existing Comfy workflows.
|
||
|
||
## Install once (desktop RTX 5080 host)
|
||
|
||
```powershell
|
||
# From the aigen checkout on the GPU PC
|
||
.\scripts\setup-yue2.ps1 -Uv <path-to-uv.exe>
|
||
```
|
||
|
||
Default checkout: `C:\Users\ianjm\Development\YuE2` (isolated venv, not Comfy Python). Setup does not download model weights; Hugging Face pulls them on the first song.
|
||
|
||
Restart the existing host connector (`comfy-host-agent`) after pulling these scripts so `/yue2/jobs` is mounted. Do not restart Coolify or Comfy for this wiring.
|
||
|
||
## Env vars
|
||
|
||
| Variable | Purpose |
|
||
| --- | --- |
|
||
| `YUE2_ROOT` | Standalone YuE checkout (default `C:\Users\ianjm\Development\YuE2`) |
|
||
| `YUE2_PYTHON` | Isolated worker Python (`YUE2_ROOT\.venv\Scripts\python.exe`) |
|
||
| `YUE2_JOBS_DIR` | Per-job logs, request.json, intermediates, WAV |
|
||
| `YUE2_MODEL` | Default `m-a-p/YuE2-3B` |
|
||
| `YUE2_VAE` | Default `m-a-p/YuE2-Vae` |
|
||
|
||
App still uses `COMFY_CONTROL_URL` + token. YuE2 shares the same GPU reservation: Comfy is stopped while a YuE2 job owns the card. One song at a time. No `keep_model_loaded`. OOM is a visible error; the app does not fall back to `yue` or ACE.
|
||
|
||
## Call path
|
||
|
||
```json
|
||
{"engine":"yue2","duration":60,"seed":42,"tags":"pop, warm vocals","lyrics":"[Verse 1]\nYour lyrics here","folderId":"<library-folder>","name":"My song"}
|
||
```
|
||
|
||
`startMusicJob` → `startYue2Job` → `POST <COMFY_CONTROL_URL>/yue2/jobs` → `yue2-worker.py` (`plan` → `generate_semantic` → `synthesize` → `decode`) → library WAV.
|
||
|
||
Duration clamp is 30–150 seconds (default 60), same as Yue field validation: one non-empty lyric section. Upstream song length follows the plan; duration is validated and stored for library metadata.
|
||
|
||
## Success check
|
||
|
||
1. Music page Model picker shows YuE2.
|
||
2. POST with `engine:"yue2"` hits `/yue2/jobs` (host log), not Comfy queue.
|
||
3. `engine:"yue"` still hits `/yuegp/jobs`.
|
||
4. ACE graphs unchanged.
|
||
5. After a real 5080 run: WAV in library, worker unloaded, Comfy can wake again.
|
||
|
||
This agent session did not generate a song.
|
||
|
||
## Remove old YuE / YuEGP data only (host checklist)
|
||
|
||
Run these **only after** YuE2 has produced a real song and you no longer need YuEGP. Named folders only. Do **not** recursively delete Comfy `models/` or `custom_nodes/`.
|
||
|
||
Adjust paths if your `YUEGP_ROOT` / HF cache differ. Review each path before deleting.
|
||
|
||
```powershell
|
||
# --- YuEGP checkout weights / jobs (not the aigen scripts) ---
|
||
# Example defaults from setup-yuegp.ps1 / docs/yuegp-replacement.md
|
||
$YueGpRoot = 'C:\Users\ianjm\Development\YuEGP'
|
||
$YueGpJobs = Join-Path $YueGpRoot 'aigen-jobs'
|
||
$YueModels = Join-Path $env:LOCALAPPDATA 'Comfy-Desktop\ComfyUI-Shared\models\yue'
|
||
|
||
# Stage 1 / Stage 2 local folders (only if clearly YuE v1)
|
||
Remove-Item -LiteralPath (Join-Path $YueModels 'YuE-s1-7B-anneal-en-cot') -Recurse -Force -ErrorAction SilentlyContinue
|
||
Remove-Item -LiteralPath (Join-Path $YueModels 'YuE-s2-1B-general') -Recurse -Force -ErrorAction SilentlyContinue
|
||
|
||
# Old codec under YuEGP inference (YuEGP-only copy)
|
||
Remove-Item -LiteralPath (Join-Path $YueGpRoot 'inference\xcodec_mini_infer') -Recurse -Force -ErrorAction SilentlyContinue
|
||
|
||
# Old YuEGP song dumps
|
||
Remove-Item -LiteralPath $YueGpJobs -Recurse -Force -ErrorAction SilentlyContinue
|
||
|
||
# Hugging Face cache folders that are clearly YuE v1 / old codec only
|
||
$Hf = Join-Path $env:USERPROFILE '.cache\huggingface\hub'
|
||
Remove-Item -LiteralPath (Join-Path $Hf 'models--m-a-p--YuE-s1-7B-anneal-en-cot') -Recurse -Force -ErrorAction SilentlyContinue
|
||
Remove-Item -LiteralPath (Join-Path $Hf 'models--m-a-p--YuE-s2-1B-general') -Recurse -Force -ErrorAction SilentlyContinue
|
||
Remove-Item -LiteralPath (Join-Path $Hf 'models--m-a-p--xcodec_mini_infer') -Recurse -Force -ErrorAction SilentlyContinue
|
||
|
||
# Leftover Comfy YuE Stage A/B weight copies (YuE-only names). Leave ACE / H3 / Klein / Krea / LTX / upscale alone.
|
||
$ComfyYue = Join-Path $env:LOCALAPPDATA 'Comfy-Desktop\ComfyUI-Shared\models'
|
||
Get-ChildItem -LiteralPath $ComfyYue -Recurse -ErrorAction SilentlyContinue |
|
||
Where-Object { $_.Name -match 'YUE_Stage_|YuE-s1-|YuE-s2-|yue_stage' } |
|
||
ForEach-Object { Remove-Item -LiteralPath $_.FullName -Recurse -Force -ErrorAction SilentlyContinue }
|
||
```
|
||
|
||
**Do not delete:** ACE-Step weights, ACE workflow JSON, MiniMax H3 / LTX / Flux Klein / Krea assets, LoRAs, `LIBRARY_DIR`, Studio 2 graphs, or `scripts/yuegp-*` / `server/utils/yueGp.ts` until YuE2 has proven a real song. Data off disk ≠ rip out the `engine:"yue"` code path.
|