101 lines
4.3 KiB
Markdown
101 lines
4.3 KiB
Markdown
# YuE2 music engine
|
||
|
||
`engine: "yue2"` is the only music engine in the app. Standalone [YuE2](https://github.com/multimodal-art-projection/YuE) (`m-a-p/YuE2-3B` + `m-a-p/YuE2-Vae`). It never submits a Comfy graph. ACE / YuEGP are removed from the picker and generate API; leave ACE Comfy checkpoints on disk.
|
||
|
||
## Desktop agent at boot (one click)
|
||
|
||
Wrong agent on 8199 → `/yue2/status` returns `{"ok":false,"error":"not found"}` while Comfy images still work.
|
||
|
||
Fix once from this clone:
|
||
|
||
```powershell
|
||
.\scripts\install-comfy-host-agent-launchers.ps1
|
||
```
|
||
|
||
That rewrites Desktop, Start Menu, and **Windows Startup** shortcuts to this clone, and retargets the `AIGen Comfy Host Agent` scheduled task when it exists. Each launch **kills** any old `comfy-host-agent` / listener on 8199, then starts `scripts/comfy-host-agent.mjs` (imports `yue2-host.mjs`).
|
||
|
||
After install: double-click Desktop **Start Comfy Host Agent**, or reboot. Confirm:
|
||
|
||
```text
|
||
GET http://127.0.0.1:8199/yue2/status → 200 { configured, busy, backend: "yue2" }
|
||
```
|
||
|
||
Do not guess which `node.exe` to kill — the launcher replaces it.
|
||
|
||
## What this host already has (5080 desktop)
|
||
|
||
Ran on this machine:
|
||
|
||
1. `scripts/setup-yue2.ps1` → checkout + venv + CUDA torch + weight download
|
||
2. `scripts/cleanup-yue-v1.ps1` → named YuE v1 / YuEGP weight folders removed
|
||
3. Host agent mounts `POST/GET /yue2/jobs`
|
||
|
||
Paths written:
|
||
|
||
| Item | Path |
|
||
| --- | --- |
|
||
| `YUE2_ROOT` | `C:\Users\ianjm\Development\YuE2` |
|
||
| Python | `C:\Users\ianjm\Development\YuE2\.venv\Scripts\python.exe` |
|
||
| Model | `C:\Users\ianjm\Development\YuE2\models\YuE2-3B` (~7.3 GB) |
|
||
| VAE | `C:\Users\ianjm\Development\YuE2\models\YuE2-Vae` (~531 MB) |
|
||
| Ready file | `C:\Users\ianjm\Development\YuE2\aigen-ready.json` |
|
||
| Jobs dir default | `C:\Users\ianjm\Development\YuE2\aigen-jobs` |
|
||
|
||
`aigen-ready.json` points `model` / `vae` at those local folders so Generate does not need a first-click HF pull. CUDA check: `torch 2.11.0+cu128` on `NVIDIA GeForce RTX 5080`.
|
||
|
||
After deploying this branch, run the installer above (or the Desktop shortcut) once so 8199 serves YuE2. Do not restart Coolify from the agent; live apps that track `main` rebuild on push.
|
||
|
||
## Env vars
|
||
|
||
| Variable | Purpose |
|
||
| --- | --- |
|
||
| `YUE2_ROOT` | Standalone YuE checkout |
|
||
| `YUE2_PYTHON` | Isolated worker Python |
|
||
| `YUE2_JOBS_DIR` | Per-job logs / request / WAV |
|
||
| `YUE2_MODEL` | Local model dir or HF id |
|
||
| `YUE2_VAE` | Local VAE dir or HF id |
|
||
|
||
App uses existing `COMFY_CONTROL_URL` + token. One GPU owner via the shared reservation (Comfy stopped while YuE2 runs). One song. Unload after decode. OOM is visible; no ACE/YuEGP fallback.
|
||
|
||
## Call path
|
||
|
||
```json
|
||
{"engine":"yue2","duration":60,"seed":42,"tags":"pop, warm vocals","lyrics":"[Verse 1]\nYour lyrics here","folderId":"<library-folder>","name":"My song"}
|
||
```
|
||
|
||
Picker → `POST /api/generate/music` → `startYue2Job` → `POST <COMFY_CONTROL_URL>/yue2/jobs` → `yue2-worker.py` (`plan` → `generate_semantic` → `synthesize` → `decode`) → library WAV.
|
||
|
||
Duration 30–150 s (default 60). One non-empty lyric section (same validation as Yue).
|
||
|
||
## Old YuE / YuEGP data deleted on this host
|
||
|
||
`scripts/cleanup-yue-v1.ps1` logged deletions to `scripts/cleanup-yue-v1.log`. Deleted:
|
||
|
||
- `...\ComfyUI-Shared\models\yue\YuE-s1-7B-anneal-en-cot`
|
||
- `...\ComfyUI-Shared\models\yue\YuE-s2-1B-general`
|
||
- `...\ComfyUI-Shared\models\yue\xcodec_mini_infer`
|
||
- `...\ComfyUI-Shared\models\yue\ckpt_00360000.pth`
|
||
- `...\ComfyUI-Shared\models\yue\decoder_131000.pth`
|
||
- `...\ComfyUI-Shared\models\yue\decoder_151000.pth`
|
||
- `C:\Users\ianjm\Development\YuEGP\inference\xcodec_mini_infer`
|
||
- `C:\Users\ianjm\Development\YuEGP\aigen-jobs`
|
||
|
||
HF cache YuE-s1 / YuE-s2 / xcodec entries were already absent (skipped). ACE checkpoints (`ace_step_v1_3.5b.safetensors`, `ace_step_1.5_turbo_aio.safetensors`) and other Comfy model trees were left alone on disk.
|
||
|
||
## Re-run setup (only if rebuilding the machine)
|
||
|
||
```powershell
|
||
.\scripts\setup-yue2.ps1
|
||
.\scripts\cleanup-yue-v1.ps1
|
||
.\scripts\install-comfy-host-agent-launchers.ps1
|
||
```
|
||
|
||
Setup reuses existing `models\YuE2-3B` / `YuE2-Vae` when present, then force-reinstalls CUDA torch after the editable package install.
|
||
|
||
## Validation
|
||
|
||
- JS: `node --test tests/yue2.test.mjs`
|
||
- Python: `py -3 -m unittest tests.test_yue2_worker`
|
||
- After agent restart: `GET /yue2/status` returns 200.
|
||
- Do not claim a song generated unless one ran after the agent restart.
|