79 lines
3.5 KiB
Markdown
79 lines
3.5 KiB
Markdown
# YuE2 music engine
|
||
|
||
`engine: "yue2"` is standalone [YuE2](https://github.com/multimodal-art-projection/YuE) (`m-a-p/YuE2-3B` + `m-a-p/YuE2-Vae`). It never submits a Comfy graph. `engine: "yue"` remains YuEGP in code. ACE / ACE 1.5 keep their Comfy workflows.
|
||
|
||
## What this host already has (5080 desktop)
|
||
|
||
Ran on this machine:
|
||
|
||
1. `scripts/setup-yue2.ps1` → checkout + venv + CUDA torch + weight download
|
||
2. `scripts/cleanup-yue-v1.ps1` → named YuE v1 / YuEGP weight folders removed
|
||
3. Host agent code mounts `POST/GET /yue2/jobs` (same pattern as `/yuegp/jobs`)
|
||
|
||
Paths written:
|
||
|
||
| Item | Path |
|
||
| --- | --- |
|
||
| `YUE2_ROOT` | `C:\Users\ianjm\Development\YuE2` |
|
||
| Python | `C:\Users\ianjm\Development\YuE2\.venv\Scripts\python.exe` |
|
||
| Model | `C:\Users\ianjm\Development\YuE2\models\YuE2-3B` (~7.3 GB) |
|
||
| VAE | `C:\Users\ianjm\Development\YuE2\models\YuE2-Vae` (~531 MB) |
|
||
| Ready file | `C:\Users\ianjm\Development\YuE2\aigen-ready.json` |
|
||
| Jobs dir default | `C:\Users\ianjm\Development\YuE2\aigen-jobs` |
|
||
|
||
`aigen-ready.json` points `model` / `vae` at those local folders so Generate does not need a first-click HF pull. CUDA check: `torch 2.11.0+cu128` on `NVIDIA GeForce RTX 5080`.
|
||
|
||
After deploying this branch, restart the **desktop host connector** once so it loads `yue2-host.mjs`. Do not restart Coolify from the agent; the user triggers the existing xAIGen-dev Coolify rebuild.
|
||
|
||
## Env vars
|
||
|
||
| Variable | Purpose |
|
||
| --- | --- |
|
||
| `YUE2_ROOT` | Standalone YuE checkout |
|
||
| `YUE2_PYTHON` | Isolated worker Python |
|
||
| `YUE2_JOBS_DIR` | Per-job logs / request / WAV |
|
||
| `YUE2_MODEL` | Local model dir or HF id |
|
||
| `YUE2_VAE` | Local VAE dir or HF id |
|
||
|
||
App uses existing `COMFY_CONTROL_URL` + token. One GPU owner via the shared reservation (Comfy stopped while YuE2 runs). One song. Unload after decode. OOM is visible; no fallback to `yue`.
|
||
|
||
## Call path
|
||
|
||
```json
|
||
{"engine":"yue2","duration":60,"seed":42,"tags":"pop, warm vocals","lyrics":"[Verse 1]\nYour lyrics here","folderId":"<library-folder>","name":"My song"}
|
||
```
|
||
|
||
Picker → `POST /api/generate/music` → `startYue2Job` → `POST <COMFY_CONTROL_URL>/yue2/jobs` → `yue2-worker.py` (`plan` → `generate_semantic` → `synthesize` → `decode`) → library WAV.
|
||
|
||
Duration 30–150 s (default 60). One non-empty lyric section (same validation as Yue).
|
||
|
||
## Old YuE / YuEGP data deleted on this host
|
||
|
||
`scripts/cleanup-yue-v1.ps1` logged deletions to `scripts/cleanup-yue-v1.log`. Deleted:
|
||
|
||
- `...\ComfyUI-Shared\models\yue\YuE-s1-7B-anneal-en-cot`
|
||
- `...\ComfyUI-Shared\models\yue\YuE-s2-1B-general`
|
||
- `...\ComfyUI-Shared\models\yue\xcodec_mini_infer`
|
||
- `...\ComfyUI-Shared\models\yue\ckpt_00360000.pth`
|
||
- `...\ComfyUI-Shared\models\yue\decoder_131000.pth`
|
||
- `...\ComfyUI-Shared\models\yue\decoder_151000.pth`
|
||
- `C:\Users\ianjm\Development\YuEGP\inference\xcodec_mini_infer`
|
||
- `C:\Users\ianjm\Development\YuEGP\aigen-jobs`
|
||
|
||
HF cache YuE-s1 / YuE-s2 / xcodec entries were already absent (skipped). ACE checkpoints (`ace_step_v1_3.5b.safetensors`, `ace_step_1.5_turbo_aio.safetensors`) and other Comfy model trees were left alone. `scripts/yuegp-*` and `server/utils/yueGp.ts` remain in git.
|
||
|
||
## Re-run setup (only if rebuilding the machine)
|
||
|
||
```powershell
|
||
.\scripts\setup-yue2.ps1
|
||
.\scripts\cleanup-yue-v1.ps1
|
||
```
|
||
|
||
Setup reuses existing `models\YuE2-3B` / `YuE2-Vae` when present, then force-reinstalls CUDA torch after the editable package install.
|
||
|
||
## Validation
|
||
|
||
- JS: `node --test tests/yue2.test.mjs tests/yuegp.test.mjs`
|
||
- Python: `py -3 -m unittest tests.test_yue2_worker tests.test_yuegp_worker`
|
||
- No full song was generated in this agent session.
|