Compare commits

..
54 Commits
Author SHA1 Message Date
TowstyandCursor 1b311d6529 Add Studio 2 caption jobs via Qwen2.5-VL GGUF on the host agent.
Exclusive llama-server load/unload on the 5080, queued Describe UI with resultText, Copy, and Use as prompt.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-30 22:06:10 -05:00
TowstyandCursor 1f10082087 Fix Qwen 2.1 GGUF load: promote Q8 norms to F32 and wire TextEncode latent.
abenzerps Q8_0 ships 1D RMSNorms as packed Q8 (136 vs 128), which breaks
Comfy rms_rope; tagger now dequantizes small tensors and the graph uses
TextEncodeQwenImage21's 64-ch latent plus AuraFlow shift.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-20 16:22:55 -05:00
TowstyandCursor 4e61ee17b6 Wire Qwen 2.1 as Generate-only engine with 25/1 sampler defaults.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-20 15:53:44 -05:00
TowstyandCursor b95e8b2a87 Add studio2_qwen21_t2i Comfy graph for GGUF Qwen Image 2.1 T2I.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-20 15:53:44 -05:00
TowstyandCursor 277b49403c Add Qwen 2.1 host setup script and document Shared model paths.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-20 15:53:43 -05:00
TowstyandCursor 4831b32d20 Replace LoRA Thumbwheel spectrograms with range and numeric strengths.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-18 18:57:12 -05:00
TowstyandCursor eba701a82c Tune Studio display tokens for rails, radius, and amber CTA.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-18 13:12:05 -05:00
TowstyandCursor bdc1ab6c2d Restyle Studio task rail and bind Classic task to the same mode ref.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-18 13:10:44 -05:00
TowstyandCursor d9743dd8c9 Bind Classic model select and Studio engine rail to the same engine ref.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-18 13:10:13 -05:00
TowstyandCursor 1b29452455 Move Studio locks to slots and Classic hide to header checkboxes.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-18 13:09:29 -05:00
TowstyandCursor 292d53a20e Add Classic | Studio displayMode preference and Settings cards.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-18 13:07:13 -05:00
TowstyandCursor b194b3cefa Always live-scan Studio 2 LoRAs on load and Refresh like Settings.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-18 12:59:06 -05:00
TowstyandCursor 7221b6a8cf Refresh Studio 2 LoRAs from live Comfy object_info on fresh=1.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-18 12:45:48 -05:00
TowstyandCursor 36d24feb9c Allow Studio 2 MiniMax text-to-video with empty start when locks are off.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-16 14:56:16 -05:00
TowstyandCursor 5694104226 Keep YuE2 jobs live on the bench and wire target length to semantic tokens.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-15 20:53:07 -05:00
TowstyandCursor 2ebc3066e6 Reject instrumental music edits and force YuE2 on queued track patches.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-15 18:20:11 -05:00
TowstyandCursor c868605d8a Drop ACE and YuEGP defaults from studio music queue and library labels.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-15 18:19:46 -05:00
TowstyandCursor 1db62e9187 Make Studio 2 the home bench with YuE2 music, LTX, and per-preview hide.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-15 18:19:34 -05:00
TowstyandCursor a216afc4ce Remove YuEGP handlers from the Comfy host agent.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-15 18:17:59 -05:00
TowstyandCursor cce1faa9b1 Route all music jobs through YuE2 only.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-15 18:16:45 -05:00
TowstyandCursor c50bde723f Accept only YuE2 in the music generate API.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-15 18:16:45 -05:00
TowstyandCursor 17cc07e3bc Accept only YuE2 as the music engine type.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-15 18:16:44 -05:00
TowstyandCursor df885d5732 Replace stale host agent on boot and document one-click Startup install.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-15 18:15:19 -05:00
TowstyandCursor 840440d7ef Remove YuE2 process memory fraction cap on the 5080.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 20:14:44 -05:00
TowstyandCursor ee84a99035 Tighten YuE2 VRAM use on the 5080 and empty GPU before launch.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 20:01:06 -05:00
TowstyandCursor 62671d05da Allow multi-section YuE2 lyrics and force non-flash attention on Windows.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 19:28:43 -05:00
TowstyandCursor fae3b23e37 Finish YuE2 host setup, weight download paths, and v1 cleanup.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 18:46:35 -05:00
TowstyandCursor d1ff05089f Document YuE2 install, weight cleanup, and regression tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 18:46:35 -05:00
TowstyandCursor 71a61b1765 Add YuE2 to the music engine picker and generate API.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 18:46:35 -05:00
TowstyandCursor ec5791c595 Mount YuE2 on the host agent and share GPU ownership.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 18:46:35 -05:00
TowstyandCursor 5f68205f01 Route engine yue2 through a standalone job helper.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 18:45:57 -05:00
TowstyandCursor 85ef087364 Add YuE2 host, worker, and setup scripts.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 18:45:57 -05:00
Towsty 064f1db933 Read production LoRA names in Studio 2 2026-09-12 21:05:11 -05:00
Towsty fb87a50a39 Show each LoRA dropdown label once 2026-09-12 20:45:19 -05:00
Towsty 401e6c26a2 Remember Studio 2 LoRA stacks for each engine 2026-09-12 20:17:58 -05:00
Towsty 6a6931f04e Filter LoRA picker options by engine tokens 2026-09-12 20:17:58 -05:00
Towsty e190496eb4 Restore Klein empty-latent reference sampling 2026-09-12 12:18:14 -05:00
Towsty 7bb4b16c1e Sample Klein hero identity from image latents 2026-09-12 11:46:56 -05:00
Towsty 3da3656140 Show pending Studio 2 jobs and freeze finished timers 2026-09-12 10:39:33 -05:00
Towsty 3e90d2ff45 Show clear GPU status and move upscale progress into the queue 2026-09-12 10:17:03 -05:00
Towsty a9551911f2 Encode Unicode video titles in playback response headers 2026-09-12 10:17:02 -05:00
TowstyandCursor d8f42ecc68 Tighten needsFit to exact canvas size, guard against empty renders
needsFit only compared aspect ratio, so a same-aspect-but-larger still
(e.g. a 4032x2268 phone photo against a 1344x768 canvas) would skip
fitting and hand MiniMaxH3AddGuide a still that does not match the
job's own latent size. Require an exact width+height match instead;
anything else (including unreadable dimensions) still fits.

Also guard the video save path: an empty Comfy download, an empty
stitched assembly, or a stitched result with no readable duration now
fails the job with a clear error instead of silently saving/attaching
a broken zero-length clip.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-11 23:11:02 -05:00
TowstyandCursor 65c69b779a Stop resampling the extend start frame every hop
upload() ran fitStill (ffmpeg scale+pad) on every image except hero,
including the start/handoff still. Four chained extends resampled the
anchor frame four times and destroyed identity even with the hero
reference and lock text intact.

- upload(): only fit when the still's aspect truly differs from the
  job canvas (needsFit); never fit hero or start regardless.
- Track handoffSource (anchor|extract) and whether the start frame was
  fitted; surface both in job metadata and Details.
- extendSelected(): re-binding an already-selected clip on task switch
  must not stomp the user's overridden locks/settings back to the
  clip's saved values (regression introduced by the last hotfix push).
- Add tests/studio2-extend-fit.test.mjs: two extends at identical
  width/height reach Comfy with the anchor PNG byte-identical, no
  ffmpeg fit.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-11 21:12:24 -05:00
towsty c6451b10ce FIX EFFING EXTEND FILE 2026-09-12 00:30:50 +00:00
towsty 937e4d120c Upload files to "components/studio2"
FIX EFFING EXTEND BUTTON
2026-09-12 00:30:19 +00:00
Towsty ec0b6f10ab Bind the selected output when restoring complete input settings 2026-09-11 19:06:17 -05:00
Towsty ad135ae19c Simplify Studio 2 sources and persist private previews and LoRA lists 2026-09-11 19:04:03 -05:00
Towsty 86ffa9ba04 Compile image style chips and restore complete input metadata 2026-09-11 19:04:03 -05:00
Towsty b5a4882a9c Keep Studio 2 family tiles tied to existing library files 2026-09-11 19:04:03 -05:00
Towsty b1ffcd6975 Retry confirmed Studio 2 host cleanup after failures 2026-09-11 18:43:25 -05:00
Towsty 28b919ecee Sample Klein edits from source and bind Studio 2 extensions correctly 2026-09-11 12:23:34 -05:00
Towsty 8fe69272c3 Use desktop width and protect the Studio 2 prompt column 2026-09-11 09:33:05 -05:00
Towsty c719e12709 Select and confirm uploaded Studio 2 source and hero stills 2026-09-11 09:28:00 -05:00
Towsty bf58732690 Separate Studio 2 source and canvas and apply size and seed choices 2026-09-11 07:07:26 -05:00
104 changed files with 13241 additions and 9247 deletions
+118
View File
@@ -0,0 +1,118 @@
/* Studio 2 display skins — Classic | Studio. Layout tokens only; palette matches main.css. */
:root,
[data-display='classic'],
[data-display='studio'] {
--display-bg: #09090b;
--display-fg: #e4e4e7;
--display-muted: #a1a1aa;
--display-border: rgba(255, 255, 255, 0.1);
--display-amber: #fbbf24;
--display-amber-fg: #09090b;
--display-radius: 8px;
--display-focus: rgba(252, 211, 77, 0.5);
}
[data-display='classic'] {
--chrome-gap: 0.5rem;
--rail-h: 2.25rem;
--card-pad: 0.75rem;
--display-type-title: 1.75rem;
--display-type-body: 0.875rem;
--display-viewer-radius: 0.75rem;
}
[data-display='studio'] {
--chrome-gap: 1rem;
--rail-h: 3rem;
--card-pad: 1rem;
--display-type-title: 1.875rem;
--display-type-body: 0.9375rem;
--display-viewer-radius: 8px;
}
[data-display] .display-title {
font-size: var(--display-type-title);
}
[data-display] .display-body {
font-size: var(--display-type-body);
}
[data-display] .display-stack {
display: flex;
flex-direction: column;
gap: var(--chrome-gap);
}
[data-display] .display-card {
border-radius: var(--display-radius);
border: 1px solid var(--display-border);
padding: var(--card-pad);
}
[data-display] .display-viewer {
border-radius: var(--display-viewer-radius);
}
[data-display] .display-rail {
display: flex;
flex-wrap: wrap;
gap: 0.5rem;
min-height: var(--rail-h);
}
[data-display] .display-rail__seg {
min-height: var(--rail-h);
border-radius: var(--display-radius);
border: 1px solid var(--display-border);
padding: 0.5rem 1rem;
font-size: var(--display-type-body);
color: var(--display-fg);
background: transparent;
outline: none;
transition: border-color 0.15s ease, background-color 0.15s ease, color 0.15s ease;
}
[data-display] .display-rail__seg:focus-visible {
box-shadow: 0 0 0 2px var(--display-focus);
}
[data-display] .display-rail__seg[aria-pressed='true'],
[data-display] .display-rail__seg[data-active='true'] {
border-color: var(--display-amber);
background: var(--display-amber);
color: var(--display-amber-fg);
}
[data-display] .display-rail__seg:disabled,
[data-display] .display-rail__seg[aria-disabled='true'] {
opacity: 0.55;
cursor: not-allowed;
}
[data-display='studio'] .display-rail__seg {
font-weight: 600;
padding: 0.625rem 1.25rem;
}
[data-display='classic'] .display-rail__seg {
min-height: 2.25rem;
padding: 0.375rem 0.75rem;
font-weight: 500;
}
[data-display] .display-cta {
border-radius: var(--display-radius);
background: var(--display-amber);
color: var(--display-amber-fg);
font-weight: 600;
}
[data-display] .display-cta:focus-visible {
box-shadow: 0 0 0 2px var(--display-focus);
}
[data-display] .display-control {
border-radius: var(--display-radius);
}
+166 -53
View File
@@ -6,22 +6,21 @@
<li <li
v-for="(item, index) in model" v-for="(item, index) in model"
:key="item.name" :key="item.name"
class="rounded-xl border bg-zinc-950/80" class="rounded-lg border border-[#2A2A2E] bg-[#1A1A1D] p-2.5"
:class="[ :class="[
compact ? 'px-2 py-1.5' : 'px-3 py-2', dragOverIndex === index ? 'border-amber-300/50' : '',
dragOverIndex === index ? 'border-amber-300/50' : 'border-white/10',
draggingIndex === index ? 'opacity-60' : '' draggingIndex === index ? 'opacity-60' : ''
]" ]"
@dragover.prevent="onDragOver(index)" @dragover.prevent="onDragOver(index)"
@drop.prevent="onDrop(index)" @drop.prevent="onDrop(index)"
@dragleave="onDragLeave(index)" @dragleave="onDragLeave(index)"
> >
<div class="flex items-center justify-between gap-2"> <div class="flex items-start justify-between gap-2">
<div class="flex min-w-0 items-center gap-2"> <div class="flex min-w-0 flex-1 items-start gap-2">
<button <button
v-if="model.length > 1" v-if="model.length > 1"
type="button" type="button"
class="shrink-0 cursor-grab touch-none text-zinc-500 hover:text-zinc-300 active:cursor-grabbing disabled:cursor-not-allowed disabled:opacity-40" class="mt-0.5 shrink-0 cursor-grab touch-none text-zinc-500 hover:text-zinc-300 active:cursor-grabbing disabled:cursor-not-allowed disabled:opacity-40"
:disabled="disabled" :disabled="disabled"
draggable="true" draggable="true"
title="Drag to reorder" title="Drag to reorder"
@@ -31,61 +30,91 @@
> >
<span class="block leading-none" aria-hidden="true">⋮⋮</span> <span class="block leading-none" aria-hidden="true">⋮⋮</span>
</button> </button>
<span class="min-w-0 truncate text-zinc-200">{{ labelFor(item.name) }}</span> <span class="min-w-0 flex-1">
<LoraTriggerTip <span class="flex min-w-0 items-center gap-1.5">
:text="triggerFor(item.name)" <span class="block truncate font-medium text-[#F3F1EA]" :title="labelFor(item.name)">{{ labelFor(item.name) }}</span>
:aria-label="'Activation for ' + labelFor(item.name)" <LoraTriggerTip
@insert="emit('insert-trigger', $event)" :text="triggerFor(item.name)"
@save="saveTrigger(item.name, $event)" :aria-label="'Activation for ' + labelFor(item.name)"
/> @insert="emit('insert-trigger', $event)"
@save="saveTrigger(item.name, $event)"
/>
</span>
<span v-if="filenameSubtitle" class="mt-0.5 block truncate text-[11px] text-[#9A9588]" :title="item.name">{{ stemFor(item.name) }}</span>
</span>
</div> </div>
<button <button
type="button" type="button"
class="shrink-0 text-[11px] text-zinc-500 hover:text-red-300 disabled:opacity-50" class="shrink-0 text-[11px] text-[#9A9588] hover:text-red-300 disabled:opacity-50"
:disabled="disabled" :disabled="disabled"
@click="removeAt(index)" @click="removeAt(index)"
> >
Remove Remove
</button> </button>
</div> </div>
<div class="mt-1.5 flex flex-wrap gap-3">
<Thumbwheel <div class="mt-2 space-y-1.5">
:model-value="item.strengthModel" <label class="grid grid-cols-[3.25rem_minmax(0,1fr)_2.75rem] items-center gap-2 text-[11px] text-[#9A9588]">
label="Model" <span>{{ clipStrength ? 'Model' : 'Strength' }}</span>
:lock-key="`lora-model:${item.name}`" <input
:min="LORA_STRENGTH_MIN" type="range"
:max="LORA_STRENGTH_MAX" class="lora-strength-range min-w-0"
:step="LORA_STRENGTH_STEP" :min="0"
:fine-step="LORA_STRENGTH_STEP" :max="2"
:coarse-step="0.25" :step="LORA_STRENGTH_STEP"
:digits="2" :disabled="disabled"
compact :value="displayStrength(item.strengthModel)"
:disabled="disabled" :aria-label="(clipStrength ? 'Model' : 'Strength') + ' for ' + labelFor(item.name)"
@update:model-value="setStrength(index, 'strengthModel', $event)" @input="onRange(index, 'strengthModel', $event)"
/> >
<Thumbwheel <input
:model-value="item.strengthClip" type="number"
label="CLIP" class="lora-strength-num w-full rounded border border-[#2A2A2E] bg-[#0B0B0D] px-1 py-0.5 text-right font-mono text-[11px] text-[#F3F1EA] outline-none focus:border-[#F5A524]"
:lock-key="`lora-clip:${item.name}`" :min="0"
:min="LORA_STRENGTH_MIN" :max="2"
:max="LORA_STRENGTH_MAX" :step="LORA_STRENGTH_STEP"
:step="LORA_STRENGTH_STEP" :disabled="disabled"
:fine-step="LORA_STRENGTH_STEP" :value="displayStrength(item.strengthModel).toFixed(2)"
:coarse-step="0.25" :aria-label="(clipStrength ? 'Model' : 'Strength') + ' value'"
:digits="2" @change="onNumber(index, 'strengthModel', $event)"
compact >
:disabled="disabled || !clipStrength" </label>
@update:model-value="setStrength(index, 'strengthClip', $event)" <label
/> v-if="clipStrength"
class="grid grid-cols-[3.25rem_minmax(0,1fr)_2.75rem] items-center gap-2 text-[11px] text-[#9A9588]"
>
<span>CLIP</span>
<input
type="range"
class="lora-strength-range min-w-0"
:min="0"
:max="2"
:step="LORA_STRENGTH_STEP"
:disabled="disabled"
:value="displayStrength(item.strengthClip)"
:aria-label="'CLIP for ' + labelFor(item.name)"
@input="onRange(index, 'strengthClip', $event)"
>
<input
type="number"
class="lora-strength-num w-full rounded border border-[#2A2A2E] bg-[#0B0B0D] px-1 py-0.5 text-right font-mono text-[11px] text-[#F3F1EA] outline-none focus:border-[#F5A524]"
:min="0"
:max="2"
:step="LORA_STRENGTH_STEP"
:disabled="disabled"
:value="displayStrength(item.strengthClip).toFixed(2)"
:aria-label="'CLIP value'"
@change="onNumber(index, 'strengthClip', $event)"
>
</label>
</div> </div>
</li> </li>
</ul> </ul>
<p v-if="!clipStrength && model.length" class="mt-1 text-[11px] text-zinc-500">CLIP unused on this graph — model strength only.</p>
<select <select
v-if="model.length < LORA_STACK_MAX" v-if="model.length < LORA_STACK_MAX"
class="w-full rounded-xl border border-white/10 bg-zinc-950 outline-none ring-amber-300/40 focus:ring-2 disabled:opacity-50" class="w-full rounded-lg border border-[#2A2A2E] bg-[#0B0B0D] text-[#F3F1EA] outline-none ring-[#F5A524]/40 focus:ring-2 disabled:opacity-50"
:class="[ :class="[
compact ? 'px-2 py-1 text-xs' : 'px-3 py-2 text-sm', compact ? 'px-2 py-1.5 text-xs' : 'px-3 py-2 text-sm',
model.length ? 'mt-2' : '' model.length ? 'mt-2' : ''
]" ]"
:disabled="disabled || !availableNames.length" :disabled="disabled || !availableNames.length"
@@ -97,28 +126,77 @@
v-for="name in availableNames" v-for="name in availableNames"
:key="name" :key="name"
:value="name" :value="name"
:title="name"
>{{ labelFor(name) }}</option> >{{ labelFor(name) }}</option>
</select> </select>
</div> </div>
</template> </template>
<style scoped>
.lora-strength-range {
-webkit-appearance: none;
appearance: none;
height: 6px;
border-radius: 999px;
background: #2a2a2e;
outline: none;
}
.lora-strength-range::-webkit-slider-thumb {
-webkit-appearance: none;
appearance: none;
width: 14px;
height: 14px;
border-radius: 999px;
background: #f5a524;
border: 0;
cursor: pointer;
}
.lora-strength-range::-moz-range-track {
height: 6px;
border-radius: 999px;
background: #2a2a2e;
}
.lora-strength-range::-moz-range-thumb {
width: 14px;
height: 14px;
border-radius: 999px;
background: #f5a524;
border: 0;
cursor: pointer;
}
.lora-strength-range:disabled {
opacity: 0.5;
cursor: not-allowed;
}
.lora-strength-range::-webkit-slider-runnable-track {
height: 6px;
border-radius: 999px;
background: #2a2a2e;
}
</style>
<script setup lang="ts"> <script setup lang="ts">
import { import {
LORA_STACK_MAX, LORA_STACK_MAX,
LORA_STRENGTH_MAX, LORA_STRENGTH_DEFAULT,
LORA_STRENGTH_MIN,
LORA_STRENGTH_STEP, LORA_STRENGTH_STEP,
clampLoraStrength, loraDisplayName,
loraFriendlyName, loraFriendlyName,
loraIdentityKey, loraIdentityKey,
loraMatchesImageEngine, loraMatchesImageEngine,
loraMatchesUse,
type LoraUse,
loraTriggerFor, loraTriggerFor,
makeLoraStackItem, makeLoraStackItem,
type LoraStackItem type LoraStackItem
} from '~/utils/loras' } from '~/utils/loras'
const UI_MIN = 0
const UI_MAX = 2
const props = withDefaults(defineProps<{ const props = withDefaults(defineProps<{
names: string[] names: string[]
filenameSubtitle?: boolean
label?: string label?: string
emptyLabel?: string emptyLabel?: string
addLabel?: string addLabel?: string
@@ -129,8 +207,10 @@ const props = withDefaults(defineProps<{
triggers?: Record<string, string> triggers?: Record<string, string>
aliases?: Record<string, string> aliases?: Record<string, string>
engine?: 'flux' | 'krea' | '' engine?: 'flux' | 'krea' | ''
use?: LoraUse
}>(), { }>(), {
label: 'LoRA', label: 'LoRA',
filenameSubtitle: false,
emptyLabel: 'None', emptyLabel: 'None',
addLabel: 'Add LoRA', addLabel: 'Add LoRA',
missingLabel: 'Wake GPU to load LoRAs', missingLabel: 'Wake GPU to load LoRAs',
@@ -157,7 +237,7 @@ const availableNames = computed(() => {
const engine = props.engine === 'flux' || props.engine === 'krea' ? props.engine : null const engine = props.engine === 'flux' || props.engine === 'krea' ? props.engine : null
return props.names return props.names
.filter(name => !taken.has(loraIdentityKey(name))) .filter(name => !taken.has(loraIdentityKey(name)))
.filter(name => !engine || loraMatchesImageEngine(name, engine)) .filter(name => props.use ? loraMatchesUse(name, props.use) : !engine || loraMatchesImageEngine(name, engine))
.slice() .slice()
.sort((a, b) => labelFor(a).localeCompare(labelFor(b), undefined, { sensitivity: 'base' })) .sort((a, b) => labelFor(a).localeCompare(labelFor(b), undefined, { sensitivity: 'base' }))
}) })
@@ -169,6 +249,17 @@ const selectPlaceholder = computed(() => {
return model.value.length ? props.addLabel : props.emptyLabel return model.value.length ? props.addLabel : props.emptyLabel
}) })
function clampUi(raw: unknown) {
const value = Number(raw)
if (!Number.isFinite(value)) return LORA_STRENGTH_DEFAULT
const snapped = Math.round(value / LORA_STRENGTH_STEP) * LORA_STRENGTH_STEP
return Math.min(UI_MAX, Math.max(UI_MIN, Math.round(snapped * 100) / 100))
}
function displayStrength(value: unknown) {
return clampUi(value ?? LORA_STRENGTH_DEFAULT)
}
function onAdd(event: Event) { function onAdd(event: Event) {
const name = (event.target as HTMLSelectElement).value const name = (event.target as HTMLSelectElement).value
;(event.target as HTMLSelectElement).value = '' ;(event.target as HTMLSelectElement).value = ''
@@ -180,7 +271,11 @@ function addLora(name: string) {
if (!item || model.value.length >= LORA_STACK_MAX) return if (!item || model.value.length >= LORA_STACK_MAX) return
const taken = new Set(model.value.map(entry => loraIdentityKey(entry.name))) const taken = new Set(model.value.map(entry => loraIdentityKey(entry.name)))
if (taken.has(loraIdentityKey(item.name))) return if (taken.has(loraIdentityKey(item.name))) return
model.value = [...model.value, item] model.value = [...model.value, {
...item,
strengthModel: clampUi(item.strengthModel ?? LORA_STRENGTH_DEFAULT),
strengthClip: clampUi(item.strengthClip ?? LORA_STRENGTH_DEFAULT)
}]
} }
function removeAt(index: number) { function removeAt(index: number) {
@@ -228,8 +323,22 @@ function onDragEnd() {
} }
function setStrength(index: number, key: 'strengthModel' | 'strengthClip', value: number) { function setStrength(index: number, key: 'strengthModel' | 'strengthClip', value: number) {
const next = clampLoraStrength(value) const next = clampUi(value)
model.value = model.value.map((item, i) => i === index ? { ...item, [key]: next } : item) model.value = model.value.map((item, i) => {
if (i !== index) return item
if (!props.clipStrength) {
return { ...item, strengthModel: next, strengthClip: next }
}
return { ...item, [key]: next }
})
}
function onRange(index: number, key: 'strengthModel' | 'strengthClip', event: Event) {
setStrength(index, key, Number((event.target as HTMLInputElement).value))
}
function onNumber(index: number, key: 'strengthModel' | 'strengthClip', event: Event) {
setStrength(index, key, Number((event.target as HTMLInputElement).value))
} }
function triggerFor(name: string) { function triggerFor(name: string) {
@@ -240,6 +349,10 @@ function labelFor(name: string) {
return loraFriendlyName(name, props.aliases) return loraFriendlyName(name, props.aliases)
} }
function stemFor(name: string) {
return loraDisplayName(name)
}
function saveTrigger(name: string, trigger: string) { function saveTrigger(name: string, trigger: string) {
emit('update-trigger', name, trigger) emit('update-trigger', name, trigger)
} }
+35 -2
View File
@@ -20,6 +20,37 @@
:class="noticeError ? 'border-red-400/30 bg-red-950/40 text-red-100' : 'border-emerald-400/20 bg-emerald-950/30 text-emerald-100'" :class="noticeError ? 'border-red-400/30 bg-red-950/40 text-red-100' : 'border-emerald-400/20 bg-emerald-950/30 text-emerald-100'"
>{{ notice }}</p> >{{ notice }}</p>
<section v-if="activePanel === 'display'" class="glass rounded-3xl p-5 sm:p-6">
<h2 class="font-display text-xl font-bold">Display</h2>
<p class="mt-1 text-sm text-zinc-400">Layout only. Jobs, engines, and library IDs stay the same.</p>
<div class="mt-4 grid gap-3 sm:grid-cols-2">
<button
type="button"
class="rounded-2xl border p-4 text-left outline-none ring-amber-300/40 focus-visible:ring-2"
:class="storedDisplayMode === 'classic'
? 'border-amber-300/60 bg-amber-400/10'
: 'border-white/10 bg-zinc-950/50 hover:border-white/25'"
:aria-pressed="storedDisplayMode === 'classic'"
@click="setDisplayMode('classic')"
>
<span class="block font-display text-lg font-semibold text-zinc-50">Classic</span>
<span class="mt-1 block text-xs text-zinc-400">Packed header, compact selects, tight gaps.</span>
</button>
<button
type="button"
class="rounded-2xl border p-4 text-left outline-none ring-amber-300/40 focus-visible:ring-2"
:class="storedDisplayMode === 'studio'
? 'border-amber-300/60 bg-amber-400/10'
: 'border-white/10 bg-zinc-950/50 hover:border-white/25'"
:aria-pressed="storedDisplayMode === 'studio'"
@click="setDisplayMode('studio')"
>
<span class="block font-display text-lg font-semibold text-zinc-50">Studio</span>
<span class="mt-1 block text-xs text-zinc-400">Task and model rails, locks by slots, eye toggles.</span>
</button>
</div>
</section>
<section v-if="activePanel === 'gpu'" class="glass rounded-3xl p-5 sm:p-6"> <section v-if="activePanel === 'gpu'" class="glass rounded-3xl p-5 sm:p-6">
<h2 class="font-display text-xl font-bold">GPU & defaults</h2> <h2 class="font-display text-xl font-bold">GPU & defaults</h2>
<div class="mt-4 space-y-3 rounded-2xl border border-white/10 bg-zinc-950/50 px-4 py-3"> <div class="mt-4 space-y-3 rounded-2xl border border-white/10 bg-zinc-950/50 px-4 py-3">
@@ -422,11 +453,12 @@ export interface SettingsFolder {
unlocked: boolean unlocked: boolean
} }
type SettingsPanel = 'gpu' | 'storage' | 'login' | 'folders' | 'loras' | 'keeps' | 'prompts' type SettingsPanel = 'display' | 'gpu' | 'storage' | 'login' | 'folders' | 'loras' | 'keeps' | 'prompts'
type LoraKindFilter = 'all' | 'image' | 'video' type LoraKindFilter = 'all' | 'image' | 'video'
const route = useRoute() const route = useRoute()
const router = useRouter() const router = useRouter()
const { storedDisplayMode, setDisplayMode } = useDisplayMode()
const folders = ref<SettingsFolder[]>([]) const folders = ref<SettingsFolder[]>([])
const authMode = ref('') const authMode = ref('')
@@ -435,7 +467,7 @@ const notice = ref('')
const noticeError = ref(false) const noticeError = ref(false)
let noticeTimer: ReturnType<typeof setTimeout> | null = null let noticeTimer: ReturnType<typeof setTimeout> | null = null
const panelIds: SettingsPanel[] = ['gpu', 'storage', 'login', 'folders', 'loras', 'keeps', 'prompts'] const panelIds: SettingsPanel[] = ['display', 'gpu', 'storage', 'login', 'folders', 'loras', 'keeps', 'prompts']
function parsePanel(raw: unknown): SettingsPanel { function parsePanel(raw: unknown): SettingsPanel {
const value = String(raw || '') const value = String(raw || '')
@@ -456,6 +488,7 @@ const activePanel = computed({
const navItems = computed(() => { const navItems = computed(() => {
const items: Array<{ id: SettingsPanel; label: string }> = [ const items: Array<{ id: SettingsPanel; label: string }> = [
{ id: 'display', label: 'Display' },
{ id: 'gpu', label: 'GPU & defaults' }, { id: 'gpu', label: 'GPU & defaults' },
{ id: 'storage', label: 'Storage' } { id: 'storage', label: 'Storage' }
] ]
+5 -33
View File
@@ -1,7 +1,7 @@
<template> <template>
<nav v-if="!advanced" aria-label="Generation type" class="grid grid-cols-3 gap-2 rounded-2xl border border-white/10 bg-zinc-950/50 p-2"> <nav v-if="!advanced" aria-label="Generation type" class="grid grid-cols-3 gap-2 rounded-2xl border border-white/10 bg-zinc-950/50 p-2">
<button v-for="kind in ['image', 'video'] as const" :key="kind" type="button" :aria-pressed="current === kind" class="rounded-xl px-3 py-3 font-semibold capitalize" :class="current === kind ? 'bg-amber-400 text-zinc-950' : 'text-zinc-300 hover:bg-white/5'" @click="emit('select', kind)">{{ kind }}</button> <button v-for="kind in ['image', 'video'] as const" :key="kind" type="button" :aria-pressed="current === kind" class="rounded-xl px-3 py-3 font-semibold capitalize" :class="current === kind ? 'bg-amber-400 text-zinc-950' : 'text-zinc-300 hover:bg-white/5'" @click="emit('select', kind)">{{ kind }}</button>
<NuxtLink to="/music" class="rounded-xl px-3 py-3 text-center font-semibold" :aria-current="current === 'music' ? 'page' : undefined" :class="current === 'music' ? 'bg-amber-400 text-zinc-950' : 'text-zinc-300 hover:bg-white/5'">Music</NuxtLink> <NuxtLink to="/music?engine=yue2" class="rounded-xl px-3 py-3 text-center font-semibold" :aria-current="current === 'music' ? 'page' : undefined" :class="current === 'music' ? 'bg-amber-400 text-zinc-950' : 'text-zinc-300 hover:bg-white/5'">Music</NuxtLink>
</nav> </nav>
<div v-else class="mb-4 grid grid-cols-3 gap-3"> <div v-else class="mb-4 grid grid-cols-3 gap-3">
<div <div
@@ -89,7 +89,7 @@
:class="current === 'music' ? 'border-amber-300 bg-amber-400/10 text-amber-50' : 'border-white/10 text-zinc-400'" :class="current === 'music' ? 'border-amber-300 bg-amber-400/10 text-amber-50' : 'border-white/10 text-zinc-400'"
> >
<NuxtLink <NuxtLink
to="/music" to="/music?engine=yue2"
class="block w-full px-4 pb-3 pt-4 text-left transition" class="block w-full px-4 pb-3 pt-4 text-left transition"
:class="current === 'music' ? '' : 'hover:text-zinc-200'" :class="current === 'music' ? '' : 'hover:text-zinc-200'"
> >
@@ -99,26 +99,7 @@
</svg> </svg>
<span class="mt-3 block font-display text-lg font-bold text-zinc-100">Music</span> <span class="mt-3 block font-display text-lg font-bold text-zinc-100">Music</span>
</NuxtLink> </NuxtLink>
<div class="grid grid-cols-2 border-t border-white/10"> <p class="border-t border-white/10 px-4 py-2 text-xs text-zinc-400">YuE2 · 3B lyrics-to-song</p>
<button
type="button"
class="border-r border-white/10 px-3 py-2.5 text-left text-sm transition"
:class="engineBtnClass(current === 'music' && musicEngine === 'ace')"
@click="pickMusic('ace')"
>
<span class="block font-semibold">ACE</span>
<span class="text-xs opacity-80">Step</span>
</button>
<button
type="button"
class="px-3 py-2.5 text-left text-sm transition"
:class="engineBtnClass(current === 'music' && musicEngine === 'yue')"
@click="pickMusic('yue')"
>
<span class="block font-semibold">YuE</span>
<span class="text-xs opacity-80">Lyrics</span>
</button>
</div>
</div> </div>
</div> </div>
</template> </template>
@@ -131,20 +112,19 @@ const props = withDefaults(defineProps<{
videoEngine?: 'minimax' | 'ltx' videoEngine?: 'minimax' | 'ltx'
videoStart?: 'still' | 'text' videoStart?: 'still' | 'text'
imageEngine?: 'flux' | 'krea' imageEngine?: 'flux' | 'krea'
musicEngine?: 'ace' | 'yue' musicEngine?: 'yue2'
}>(), { }>(), {
ltxEnabled: false, ltxEnabled: false,
videoEngine: 'minimax', videoEngine: 'minimax',
videoStart: 'still', videoStart: 'still',
imageEngine: 'flux', imageEngine: 'flux',
musicEngine: 'ace' musicEngine: 'yue2'
}) })
const emit = defineEmits<{ const emit = defineEmits<{
select: [kind: 'video' | 'image'] select: [kind: 'video' | 'image']
'video-engine': [engine: 'minimax' | 'ltx'] 'video-engine': [engine: 'minimax' | 'ltx']
'image-engine': [engine: 'flux' | 'krea'] 'image-engine': [engine: 'flux' | 'krea']
'music-engine': [engine: 'ace' | 'yue']
}>() }>()
function engineBtnClass(active: boolean) { function engineBtnClass(active: boolean) {
@@ -161,12 +141,4 @@ function pickImage(engine: 'flux' | 'krea') {
emit('image-engine', engine) emit('image-engine', engine)
if (props.current !== 'image') emit('select', 'image') if (props.current !== 'image') emit('select', 'image')
} }
function pickMusic(engine: 'ace' | 'yue') {
if (props.current === 'music') {
emit('music-engine', engine)
return
}
void navigateTo({ path: '/music', query: { engine } })
}
</script> </script>
+3 -3
View File
@@ -1,8 +1,8 @@
<template> <template>
<div v-if="clipId" class="space-y-2"> <div v-if="clipId" class="space-y-2">
<button type="button" class="rounded-2xl border border-amber-300/40 px-4 py-2 text-sm text-amber-100 disabled:opacity-50" :disabled="busy || loading || !state" @click="openDialog">Upscale</button> <button type="button" class="rounded-2xl border border-amber-300/40 px-4 py-2 text-sm text-amber-100 disabled:opacity-50" :disabled="busy || loading || !state" @click="openDialog">Upscale</button>
<NuxtLink v-if="state?.source.upscaledFromClipId" :to="`/?clip=${state.source.upscaledFromClipId}`" class="block text-xs text-amber-200 underline">Upscaled copy · Open original</NuxtLink> <NuxtLink v-if="state?.source.upscaledFromClipId && !queueOnly" :to="`/?clip=${state.source.upscaledFromClipId}`" class="block text-xs text-amber-200 underline">Upscaled copy · Open original</NuxtLink>
<div v-if="state?.job" class="max-w-sm text-xs text-zinc-400" aria-live="polite"> <div v-if="state?.job && !queueOnly" class="max-w-sm text-xs text-zinc-400" aria-live="polite">
<span class="capitalize">Upscale {{ state.job.status }}</span> <span class="capitalize">Upscale {{ state.job.status }}</span>
<span v-if="state.job.width"> · {{ state.job.width }} × {{ state.job.height }}</span> <span v-if="state.job.width"> · {{ state.job.width }} × {{ state.job.height }}</span>
<span v-if="state.job.elapsedMs"> · {{ Math.round(state.job.elapsedMs / 1000) }}s</span> <span v-if="state.job.elapsedMs"> · {{ Math.round(state.job.elapsedMs / 1000) }}s</span>
@@ -36,7 +36,7 @@
</template> </template>
<script setup lang="ts"> <script setup lang="ts">
import { upscaleOptions, upscaleDimensions, upscaleName } from '~/shared/video-upscale.mjs' import { upscaleOptions, upscaleDimensions, upscaleName } from '~/shared/video-upscale.mjs'
const props = defineProps<{ clipId: string; generationBusy?: boolean }>() const props = defineProps<{ clipId: string; generationBusy?: boolean; queueOnly?: boolean }>()
const emit = defineEmits<{ complete: [] }>() const emit = defineEmits<{ complete: [] }>()
const dialog = ref<HTMLDialogElement>(), dialogOpen = ref(false), state = ref<any>(null), error = ref(''), loading = ref(true), submitting = ref(false) const dialog = ref<HTMLDialogElement>(), dialogOpen = ref(false), state = ref<any>(null), error = ref(''), loading = ref(true), submitting = ref(false)
const form = reactive(upscaleOptions()) const form = reactive(upscaleOptions())
+8
View File
@@ -0,0 +1,8 @@
<template>
<div class="w-36 shrink-0 space-y-2"><span class="text-xs text-zinc-400">{{ label }}</span><button class="flex h-24 w-full items-center justify-center overflow-hidden rounded-xl border border-white/15 bg-black" @click="$emit('pick')"><span v-if="asset && hidden" class="p-2 text-xs text-zinc-400">Still loaded · preview hidden</span><img v-else-if="asset" :src="asset.kind==='clip'?`/api/library/clips/${asset.id}/thumb`:`/api/library/stills/${asset.id}`" class="h-full w-full object-contain" :alt="label"><span v-else class="text-xs text-zinc-400">Choose {{ label.toLowerCase() }}</span></button><p v-if="asset" class="truncate text-xs" :title="assetLabel(asset)">{{ assetLabel(asset) }}</p><label v-if="upload" class="block cursor-pointer text-xs text-amber-200">Upload {{ label.toLowerCase() }}<input type="file" accept="image/*" class="sr-only" :disabled="disabled" @change="$emit('upload',$event)"></label></div>
</template>
<script setup lang="ts">
import {assetLabel} from '~/shared/studio2/viewer.mjs'
defineProps<{label:string;asset?:any;hidden?:boolean;upload?:boolean;disabled?:boolean}>()
defineEmits(['pick','upload'])
</script>
+552 -73
View File
@@ -1,98 +1,577 @@
<template> <template>
<main class="mx-auto max-w-7xl space-y-5 p-4 md:p-8"> <main class="display-stack w-full min-w-0 p-4 md:p-6 xl:p-8" :data-display="displayMode">
<header class="flex flex-wrap items-center justify-between gap-4"> <header class="flex flex-wrap items-end justify-between gap-4">
<div><h1 class="font-display text-3xl font-bold">Studio 2</h1><p class="text-sm text-zinc-400">One hero. Every shot.</p></div> <div>
<h1 class="display-title font-display font-bold tracking-tight text-zinc-50">Studio 2</h1>
<p class="display-body mt-1 text-zinc-400">One hero. Every shot.</p>
</div>
<div class="flex flex-wrap items-center gap-3"> <div class="flex flex-wrap items-center gap-3">
<label class="text-sm">Folder <select v-model="form.folderId" class="rounded-lg bg-zinc-800 p-2"><option v-for="f in library.folders" :value="f.id" :key="f.id">{{ f.name }}</option></select></label> <label class="display-body text-zinc-300">Folder
<label class="text-sm">Hero identity <select v-model="form.identityStillId" class="max-w-52 rounded-lg bg-zinc-800 p-2"><option value="">Choose hero</option><option v-for="s in stills" :key="s.id" :value="s.id">{{ s.name || s.filename }}</option></select></label> <select v-model="form.folderId" class="display-control ml-2 border border-white/10 bg-zinc-900 px-3 py-2 outline-none ring-amber-300/40 focus:ring-2">
<label class="text-sm"><input v-model="form.lockFace" type="checkbox"> Lock face <span class="block text-xs text-zinc-500">Face, hair, glasses, body</span></label><label class="text-sm"><input v-model="form.lockOutfit" type="checkbox"> Lock outfit <span class="block text-xs text-zinc-500">Wardrobe from hero</span></label><span v-if="form.identityStillId && form.hideInput" class="text-xs text-zinc-400">Still loaded · preview hidden</span> <option v-for="f in library.folders" :key="f.id" :value="f.id">{{ f.name }}</option>
<img v-if="form.identityStillId && !form.hideInput" :src="stillUrl(form.identityStillId)" class="h-12 w-12 rounded-lg object-cover" alt="Sticky hero identity"> </select>
<NuxtLink to="/library" class="text-sm text-amber-200">Library</NuxtLink><NuxtLink to="/" class="text-sm text-zinc-400">Studio 1</NuxtLink> </label>
<div v-if="isClassic && form.mode!=='music' && form.mode!=='caption'" class="flex flex-wrap items-center gap-3 text-sm text-zinc-300" aria-label="Identity locks">
<label class="inline-flex items-center gap-1.5"><input v-model="form.lockFace" type="checkbox" class="accent-amber-400">Lock face</label>
<label class="inline-flex items-center gap-1.5"><input v-model="form.lockOutfit" type="checkbox" class="accent-amber-400">Lock outfit</label>
</div>
<div v-if="isClassic && xaigen" class="flex flex-wrap items-center gap-3 text-sm text-zinc-300" aria-label="Preview hide">
<label class="inline-flex items-center gap-1.5"><input v-model="classicHideInput" type="checkbox" class="accent-amber-400">Hide input</label>
<label class="inline-flex items-center gap-1.5"><input v-model="classicHideOutput" type="checkbox" class="accent-amber-400">Hide output</label>
</div>
<nav class="flex flex-wrap items-center gap-3 text-sm" aria-label="Studio links">
<Studio2GpuStatus :status="gpu" />
<NuxtLink to="/queue" class="text-zinc-300 outline-none ring-amber-300/40 focus-visible:ring-2">Queue</NuxtLink>
<NuxtLink to="/library" class="text-amber-200 outline-none ring-amber-300/40 focus-visible:ring-2">Library</NuxtLink>
<NuxtLink to="/settings" class="text-zinc-300 outline-none ring-amber-300/40 focus-visible:ring-2">Settings</NuxtLink>
</nav>
</div> </div>
</header> </header>
<div class="flex flex-wrap justify-between gap-3 rounded-xl border border-white/10 px-4 py-2 text-sm"><span>GPU · {{ gpuLabel }}</span><span>{{ activeJob ? `${activeJob.request.promptSections.action.slice(0,60)} · ${activeJob.state}` : 'No active Studio 2 job' }}</span><label><input v-model="form.hideInput" type="checkbox"> Hide input stills</label></div>
<nav class="flex flex-wrap gap-2"><button v-for="task in tasks" :key="task" class="rounded-full border px-4 py-2 capitalize" :class="form.mode===task?'border-amber-300 bg-amber-300 text-black':'border-white/15'" @click="chooseTask(task)">{{ task }}</button></nav> <div class="display-stack min-w-0">
<div class="grid gap-6 lg:grid-cols-[1.3fr_1fr]"> <label v-if="isClassic" class="display-body block text-zinc-300">Task
<section class="space-y-4"> <select :value="form.mode" class="display-control mt-1 w-full max-w-xs border border-white/10 bg-zinc-900 px-3 py-2 capitalize outline-none ring-amber-300/40 focus:ring-2" @change="onTaskSelect">
<div class="flex min-h-80 items-center justify-center overflow-hidden rounded-3xl border border-white/10 bg-black"> <option v-for="task in tasks" :key="task" :value="task" class="capitalize">{{ task }}</option>
<video v-if="selected?.kind==='clip'" :src="`/api/library/clips/${selected.id}/video`" controls class="max-h-[65vh] w-full" /> </select>
<img v-else-if="selected" :src="stillUrl(selected.id)" class="max-h-[65vh] object-contain" alt="Current output"> </label>
<p v-else-if="form.imageAId && form.hideInput" class="text-zinc-400">Still loaded · preview hidden</p> <nav v-else class="display-rail" aria-label="Tasks">
<img v-else-if="form.imageAId" :src="stillUrl(form.imageAId)" class="max-h-[65vh] object-contain" alt="Source composition"> <button
<p v-else class="text-zinc-500">Your next scene starts here</p> v-for="task in tasks"
:key="task"
type="button"
class="display-rail__seg capitalize"
:aria-pressed="form.mode===task"
@click="chooseTask(task)"
>{{ task }}</button>
</nav>
<div v-if="form.mode!=='music' && form.mode!=='caption'" class="min-w-0">
<label v-if="isClassic" class="display-body block text-zinc-300">Model
<select v-model="form.engine" class="display-control mt-1 w-full max-w-md border border-white/10 bg-zinc-900 px-3 py-2 outline-none ring-amber-300/40 focus:ring-2">
<option v-for="opt in engineOptions" :key="opt.value" :value="opt.value" :disabled="opt.disabled">{{ opt.label }}{{ opt.hint ? ' — '+opt.hint : '' }}</option>
</select>
</label>
<div v-else class="display-rail" role="group" aria-label="Model">
<button
v-for="opt in engineOptions"
:key="opt.value"
type="button"
class="display-rail__seg"
:aria-pressed="form.engine===opt.value"
:aria-disabled="opt.disabled || undefined"
:disabled="opt.disabled"
:title="opt.hint || undefined"
@click="!opt.disabled && (form.engine=opt.value)"
>
<span class="block">{{ opt.label }}</span>
<span v-if="opt.hint" class="mt-0.5 block text-[11px] font-normal opacity-80">{{ opt.hint }}</span>
</button>
</div> </div>
<div v-if="selected" class="flex gap-3"><button class="rounded-full border border-white/20 px-4 py-2 text-sm" @click="restore(selected)">Use as input</button><button class="rounded-full border border-white/20 px-4 py-2 text-sm" @click="detailsOpen=true">Details</button><VideoUpscaleControl v-if="selected.kind==='clip'" :clip-id="selected.id" /></div> </div>
<div class="flex gap-2 overflow-x-auto" aria-label="Current family outputs"><button v-for="o in family" :key="o.id" @click="selectOutput(o)" class="shrink-0 rounded-lg border p-1" :class="selected?.id===o.id?'border-amber-300':'border-white/10'"><img :src="o.kind==='clip'?`/api/library/clips/${o.id}/thumb`:stillUrl(o.id)" class="h-20 w-24 rounded object-cover" alt="Family output"></button></div> <p v-else-if="form.mode==='music'" class="display-body text-zinc-400">Model · YuE2</p>
<div class="grid gap-3 sm:grid-cols-2"> <p v-else class="display-body text-zinc-400">Model · Qwen2.5-VL caption</p>
<label>Source / start still<select v-model="form.imageAId" class="mt-1 w-full rounded-xl bg-zinc-800 p-2"><option value="">None</option><option v-for="s in stills" :key="s.id" :value="s.id">{{ s.name || s.filename }}</option></select></label> </div>
<label v-if="form.mode==='compose'">Second still<select v-model="form.imageBId" class="mt-1 w-full rounded-xl bg-zinc-800 p-2"><option value="">Required</option><option v-for="s in stills" :key="s.id" :value="s.id">{{ s.name || s.filename }}</option></select></label>
<label v-if="form.mode==='refine'">Mask<select v-model="form.maskId" class="mt-1 w-full rounded-xl bg-zinc-800 p-2"><option value="">Required: white area to refine</option><option v-for="s in stills" :key="s.id" :value="s.id">{{ s.name || s.filename }}</option></select></label> <div class="grid min-w-0 gap-6 xl:grid-cols-[minmax(0,3fr)_minmax(24rem,2fr)] 2xl:gap-8">
<label v-if="form.mode==='extend'">Previous clip<select :value="form.startClipId" @change="chooseClip(($event.target as HTMLSelectElement).value)" class="mt-1 w-full rounded-xl bg-zinc-800 p-2"><option value="">Choose a clip</option><option v-for="c in clips" :key="c.id" :value="c.id">{{ c.name || c.prompt?.slice(0,50) }}</option></select></label> <section class="min-w-0 space-y-4">
<label v-if="video">Optional end still<select v-model="form.endStillId" class="mt-1 w-full rounded-xl bg-zinc-800 p-2"><option value="">None</option><option v-for="s in stills" :key="s.id" :value="s.id">{{ s.name || s.filename }}</option></select></label> <div class="display-viewer relative flex min-h-80 items-center justify-center overflow-hidden border border-white/10 bg-black">
<label class="text-sm">Add still to folder<input type="file" accept="image/*" class="mt-2 block text-xs" @change="upload"></label> <button
v-if="selected && xaigen && isStudio"
type="button"
class="absolute right-3 top-3 z-10 rounded-full border border-white/20 bg-zinc-950/80 p-2 text-zinc-200 outline-none ring-amber-300/50 hover:border-amber-300/50 focus-visible:ring-2"
:aria-pressed="!isHidden(selected)"
:aria-label="isHidden(selected) ? 'Show preview' : 'Hide preview'"
@click.stop="toggleReveal(selected.id)"
>
<svg class="h-5 w-5" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.75" aria-hidden="true">
<path v-if="isHidden(selected)" d="M3 3l18 18M10.5 10.6a2.5 2.5 0 0 0 3 3M9.4 5.1A10.4 10.4 0 0 1 12 4.5c5 0 9.3 3.1 11 7.5a11.6 11.6 0 0 1-4.2 5.1M6.2 6.2A11.5 11.5 0 0 0 1 12c1.2 2.7 3.3 4.9 5.9 6.3A10.5 10.5 0 0 0 12 19.5c1.2 0 2.4-.2 3.5-.6" stroke-linecap="round"/>
<template v-else>
<path d="M2 12s3.5-7.5 10-7.5S22 12 22 12s-3.5 7.5-10 7.5S2 12 2 12Z"/>
<circle cx="12" cy="12" r="3"/>
</template>
</svg>
</button>
<p v-if="selected && isHidden(selected)" class="text-zinc-400">{{ selected.kind==='track' ? 'Track loaded · preview hidden' : selected.role==='input' ? 'Still loaded · preview hidden' : 'Output hidden' }}</p>
<div v-else-if="selected?.kind==='track'" class="w-full max-w-xl px-6 py-4">
<AudioPlayer :src="`/api/library/tracks/${selected.id}/audio`" :download-name="`${selected.name || 'track'}.${selected.ext || 'wav'}`" :label="selected.name || 'Track'" />
</div>
<video v-else-if="selected?.kind==='clip'" :key="selected.id" :src="`/api/library/clips/${selected.id}/video`" controls class="max-h-[65vh] w-full object-contain" />
<img v-else-if="selected" :src="stillUrl(selected.id)" class="max-h-[65vh] max-w-full object-contain" alt="Current output">
<p v-else class="text-zinc-500">Select a result to view it here</p>
</div> </div>
<details v-if="video" class="rounded-xl border border-white/10 p-3"><summary>Add Guide stills</summary><div v-for="(g,i) in form.guides" :key="i" class="mt-2 flex gap-2"><select v-model="g.stillId" class="min-w-0 flex-1 bg-zinc-800 p-2"><option value="">Choose guide</option><option v-for="s in stills" :key="s.id" :value="s.id">{{ s.name || s.filename }}</option></select><input v-model.number="g.frame" type="number" min="1" aria-label="Guide frame" class="w-20 bg-zinc-800 p-2"><button @click="form.guides.splice(i,1)">Remove</button></div><button class="mt-3 text-amber-200" @click="form.guides.push({stillId:'',frame:24})">Add guide</button></details>
<div v-if="selected" class="flex flex-wrap items-start gap-3">
<button v-if="selected.kind!=='track'" class="rounded-full border border-white/20 px-4 py-2 text-sm outline-none ring-amber-300/40 focus-visible:ring-2" @click="restore(selected)">Use as input</button>
<button v-if="selected.kind==='still'" class="rounded-full border border-white/20 px-4 py-2 text-sm outline-none ring-amber-300/40 focus-visible:ring-2" @click="Object.assign(form,useAsSource(form,selected))">Use as source still</button>
<button v-if="selected.kind==='still'" class="rounded-full border border-white/20 px-4 py-2 text-sm outline-none ring-amber-300/40 focus-visible:ring-2" @click="Object.assign(form,useAsHero(form,selected))">Use as hero</button>
<button v-if="selected.kind==='still' || (selected.kind!=='track' && selected.kind!=='clip' && selected.id)" class="rounded-full border border-amber-300/40 px-4 py-2 text-sm text-amber-100 outline-none ring-amber-300/40 focus-visible:ring-2" :disabled="submitting" @click="describeSelected">Describe</button>
<button v-if="selected.kind!=='track'" class="rounded-full border border-white/20 px-4 py-2 text-sm outline-none ring-amber-300/40 focus-visible:ring-2" @click="detailsOpen=true">Details</button>
<button class="rounded-full border border-red-400/40 px-4 py-2 text-sm text-red-200 outline-none ring-amber-300/40 focus-visible:ring-2" @click="deleteOutput">Delete</button>
<VideoUpscaleControl v-if="selected.kind==='clip'" :clip-id="selected.id" queue-only @complete="refresh" />
</div>
<div class="flex min-w-0 max-w-full gap-2 overflow-x-auto" aria-label="Current family outputs">
<div
v-for="o in family"
:key="o.id"
class="relative shrink-0 rounded-lg border p-1"
:class="selected?.id===o.id ? 'border-amber-300' : 'border-white/10'"
>
<button type="button" class="block outline-none ring-amber-300/40 focus-visible:ring-2" @click="selectOutput(o)">
<span v-if="isHidden(o)" class="flex h-20 w-24 items-center justify-center p-1 text-xs text-zinc-400">{{ o.kind==='track' ? 'Audio hidden' : o.role==='input' ? 'Still loaded · preview hidden' : 'Output hidden' }}</span>
<span v-else-if="o.kind==='track'" class="flex h-20 w-24 items-center justify-center rounded bg-zinc-900 text-xs text-amber-100">Audio</span>
<img v-else :src="o.kind==='clip' ? `/api/library/clips/${o.id}/thumb` : stillUrl(o.id)" class="h-20 w-24 rounded object-cover" alt="Family output">
</button>
<button
v-if="xaigen && isStudio"
type="button"
class="absolute right-1 top-1 rounded-md border border-white/20 bg-zinc-950/80 p-1 text-zinc-200 outline-none ring-amber-300/50 focus-visible:ring-2"
:aria-label="isHidden(o) ? 'Show tile' : 'Hide tile'"
:aria-pressed="!isHidden(o)"
@click.stop="toggleReveal(o.id)"
>
<svg class="h-3.5 w-3.5" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.75" aria-hidden="true">
<path v-if="isHidden(o)" d="M3 3l18 18M10.5 10.6a2.5 2.5 0 0 0 3 3M9.4 5.1A10.4 10.4 0 0 1 12 4.5c5 0 9.3 3.1 11 7.5a11.6 11.6 0 0 1-4.2 5.1M6.2 6.2A11.5 11.5 0 0 0 1 12c1.2 2.7 3.3 4.9 5.9 6.3A10.5 10.5 0 0 0 12 19.5c1.2 0 2.4-.2 3.5-.6" stroke-linecap="round"/>
<template v-else>
<path d="M2 12s3.5-7.5 10-7.5S22 12 22 12s-3.5 7.5-10 7.5S2 12 2 12Z"/>
<circle cx="12" cy="12" r="3"/>
</template>
</svg>
</button>
</div>
</div>
<section v-if="form.mode!=='music' && form.mode!=='caption'" aria-label="Source" class="display-card">
<div class="mb-3 flex flex-wrap items-start justify-between gap-3">
<h2 class="text-xs uppercase tracking-wider text-zinc-400">Source</h2>
<div v-if="isStudio" class="flex flex-wrap gap-4 text-sm text-zinc-300" aria-label="Identity locks">
<template v-if="form.engine!=='qwen21'">
<label class="flex items-start gap-2"><input v-model="form.lockFace" type="checkbox" class="mt-1 accent-amber-400"><span>Lock face<span class="block text-xs text-zinc-500">Hero encode + lock on Klein / Krea stills</span></span></label>
<label class="flex items-start gap-2"><input v-model="form.lockOutfit" type="checkbox" class="mt-1 accent-amber-400"><span>Lock outfit<span class="block text-xs text-zinc-500">Hero encode + lock on Klein / Krea stills</span></span></label>
</template>
</div>
</div>
<div class="flex flex-wrap gap-4">
<Studio2AssetSlot label="Hero" :asset="stills.find((a:any)=>a.id===form.identityStillId)" :hidden="slotHidden(form.identityStillId)" upload :disabled="uploading" @pick="picker='identityStillId'" @upload="upload($event,'hero')" />
<Studio2AssetSlot :label="form.mode==='extend'?'Previous clip':'Start still'" :asset="sourceSlot" :hidden="slotHidden(form.mode==='extend'?form.startClipId:form.imageAId)" upload :disabled="uploading" @pick="picker=form.mode==='extend'?'startClipId':'imageAId'" @upload="upload($event,'source')" />
<Studio2AssetSlot v-if="form.mode==='compose'" label="Second still" :asset="stills.find((a:any)=>a.id===form.imageBId)" :hidden="slotHidden(form.imageBId)" @pick="picker='imageBId'" />
<Studio2AssetSlot v-if="form.mode==='refine'" label="Mask" :asset="stills.find((a:any)=>a.id===form.maskId)" :hidden="slotHidden(form.maskId)" @pick="picker='maskId'" />
<Studio2AssetSlot v-if="video" label="End still" :asset="stills.find((a:any)=>a.id===form.endStillId)" :hidden="slotHidden(form.endStillId)" @pick="picker='endStillId'" />
</div>
<label v-if="form.mode==='refine'" class="mt-3 block text-sm">Mask still (white = change). On-canvas mask comes later.
<select v-model="form.maskId" class="mt-1 w-full rounded-xl bg-zinc-800 p-2"><option value="">Choose mask still</option><option v-for="a in stills" :key="a.id" :value="a.id">{{ assetLabel(a) }}</option></select>
</label>
<p v-if="uploading || uploadMessage" role="status" class="mt-2 text-sm text-amber-200">{{ uploading?'Uploading image…':uploadMessage }}</p>
</section>
<details v-if="video" class="rounded-xl border border-white/10 p-3"><summary>Add Guide stills</summary>
<div v-for="(g,i) in form.guides" :key="i" class="mt-2 flex gap-2">
<Studio2AssetSlot label="Guide" :asset="stills.find((a:any)=>a.id===g.stillId)" :hidden="slotHidden(g.stillId)" @pick="picker='guide:'+i" />
<input v-model.number="g.frame" type="number" min="1" aria-label="Guide frame" class="w-20 bg-zinc-800 p-2">
<button type="button" @click="form.guides.splice(i,1)">Remove</button>
</div>
<button type="button" class="mt-3 text-amber-200" @click="form.guides.push({stillId:'',frame:24})">Add guide</button>
</details>
</section> </section>
<section class="space-y-4">
<div class="flex justify-between"><h2 class="text-xl font-semibold">Prompt</h2><button class="text-sm text-amber-200" @click="advanced=!advanced">{{ advanced?'Simple':'Advanced' }}</button></div> <section class="min-w-0 space-y-4">
<label class="block">Action<textarea v-model="form.promptSections.action" rows="4" class="mt-1 w-full rounded-xl bg-zinc-900 p-3" placeholder="What happens in this shot?" /></label> <template v-if="form.mode==='music'">
<Studio2PromptControls v-if="video" v-model="form.promptSections" /> <p class="text-sm text-zinc-400">YuE2 · lyrics-to-song · plan → synthesize → decode</p>
<label class="block">Locks<textarea v-model="form.promptSections.locks" rows="2" class="mt-1 w-full rounded-xl bg-zinc-900 p-3" /></label> <label class="block text-sm">Genre / style tags
<details class="rounded-xl border border-white/10 p-3"><summary>{{ video?'More shots':'Iteration prompts' }}</summary><textarea v-model="batchText" rows="5" class="mt-2 w-full rounded-xl bg-zinc-900 p-3" placeholder="One action prompt per line. Hero stays attached to every output." /></details> <textarea v-model="music.tags" rows="3" class="mt-1 w-full rounded-xl bg-zinc-900 p-3 outline-none ring-amber-300/40 focus:ring-2" :placeholder="tagsPlaceholder" />
<button class="text-sm text-amber-200" @click="settingsOpen=true">Generation settings</button> </label>
<label class="block text-sm">Lyrics
<textarea v-model="music.lyrics" rows="8" class="mt-1 w-full rounded-xl bg-zinc-900 p-3 outline-none ring-amber-300/40 focus:ring-2" :placeholder="lyricsPlaceholder" />
</label>
<div class="grid grid-cols-2 gap-3">
<label class="block text-sm">Target length (sec)
<input v-model.number="music.duration" type="number" min="15" max="150" class="mt-1 w-full rounded-xl bg-zinc-900 p-3 outline-none ring-amber-300/40 focus:ring-2">
<span class="mt-1 block text-[11px] text-zinc-500">Caps semantic tokens (~25/s). Actual length follows the model end token — not an FFmpeg trim.</span>
</label>
<label class="block text-sm">Seed
<input v-model="music.seed" placeholder="random" class="mt-1 w-full rounded-xl bg-zinc-900 p-3 outline-none ring-amber-300/40 focus:ring-2">
</label>
</div>
<label class="block text-sm">Track name (optional)
<input v-model="music.name" class="mt-1 w-full rounded-xl bg-zinc-900 p-3 outline-none ring-amber-300/40 focus:ring-2" placeholder="Optional track name">
</label>
</template>
<template v-else-if="form.mode==='caption'">
<p class="text-sm text-zinc-400">Qwen2.5-VL · image-to-text · exclusive GPU · unloads after one shot</p>
<div class="display-card space-y-3 p-3">
<Studio2AssetSlot label="Still to describe" :asset="captionStill" :hidden="slotHidden(captionStillId)" upload :disabled="uploading" @pick="picker='captionStillId'" @upload="uploadCaption($event)" />
<label class="block text-sm text-zinc-300">Caption style
<select v-model="captionStyle" class="mt-1 w-full rounded-xl border border-white/10 bg-zinc-900 p-3 outline-none ring-amber-300/40 focus:ring-2">
<option v-for="style in captionStyles" :key="style" :value="style">{{ style }}</option>
</select>
</label>
<p v-if="captionStyle==='delta'" class="text-xs text-zinc-500">Delta describes only changes from the locked look (bob / pink tank / blue shorts / Slytherin robe / barefoot).</p>
</div>
</template>
<template v-else>
<label class="block">Prompt<textarea v-model="form.promptSections.action" rows="4" class="mt-1 w-full rounded-xl bg-zinc-900 p-3 outline-none ring-amber-300/40 focus:ring-2" placeholder="Describe the image or the shot." /></label>
<Studio2PromptControls v-if="video" v-model="form.promptSections" />
<Studio2StyleChips v-else v-model="form.imageStyles" />
<label v-if="video" class="block">Extra constraints (optional) — do not repeat the Prompt. Example: keep the black glasses, no extra people.
<textarea v-model="form.promptSections.locks" rows="2" class="mt-1 w-full rounded-xl bg-zinc-900 p-3 outline-none ring-amber-300/40 focus:ring-2" />
</label>
<details v-if="video || form.mode==='edit'" class="rounded-xl border border-white/10 p-3">
<summary>{{ video?'Queue more shots':'Queue extra edit passes' }}</summary>
<p class="mt-2 text-xs text-zinc-400">Each blank-line-separated block is another job. Hero and settings stay. Press Enter twice between shots. A wrapped sentence is still one shot.{{ video?'':' Each extra pass edits the previous result with Klein.' }}</p>
<textarea v-model="batchText" rows="5" class="mt-2 w-full rounded-xl bg-zinc-900 p-3 outline-none ring-amber-300/40 focus:ring-2" placeholder="First additional prompt&#10;&#10;Next additional prompt" />
</details>
<button type="button" class="text-sm text-amber-200 outline-none ring-amber-300/40 focus-visible:ring-2" @click="settingsOpen=true">Generation settings</button>
<p v-if="!video && form.engine==='krea'" class="text-sm text-zinc-400">Krea does not attach the hero identity reference.</p>
<p v-if="!video && form.engine==='qwen21'" class="text-sm text-zinc-400">Qwen 2.1 is T2I — no hero, locks, or LoRA stack in this build.</p>
<p v-if="form.mode==='extend'" class="text-sm text-amber-200">{{ form.startClipId ? 'Extend · '+(clips.find((c:any)=>c.id===form.startClipId)?.name || form.startClipId)+' · last frame' : 'Select the clip in the family strip or Previous clip.' }}</p>
</template>
<p v-if="error" role="alert" class="text-sm text-red-300">{{ error }}</p> <p v-if="error" role="alert" class="text-sm text-red-300">{{ error }}</p>
<button :disabled="submitting" @click="queue" class="w-full rounded-2xl bg-amber-300 p-4 font-semibold text-black disabled:opacity-50">{{ submitting?'Queuing…':'Queue' }}</button> <button type="button" :disabled="submitting" class="display-cta w-full p-4 outline-none disabled:opacity-50" @click="queue">{{ submitting?'Queuing…':'Queue' }}</button>
<Studio2JobProgress v-for="job in folderJobs.slice(0,6)" :key="job.id" :job="job" :now="now" @select="focusJob(job)" @cancel="cancel(job)" />
<details v-if="advanced"><summary>Compiled prompt</summary><pre class="whitespace-pre-wrap text-xs text-zinc-400">{{ compiled }}</pre></details> <h2 class="text-sm font-semibold text-zinc-300">Job queue</h2>
<div v-for="job in upscales.filter(j=>j.folderId===form.folderId && ['queued','running'].includes(j.status))" :key="job.id" class="space-y-2 rounded-xl border border-amber-300/25 bg-zinc-900 p-4 text-sm" role="status">
<div class="flex justify-between gap-3"><strong>Video upscale</strong><span class="capitalize text-amber-200">{{ job.status }}</span></div>
<p class="truncate text-zinc-300">{{ job.name }}</p>
<p class="text-xs text-zinc-400">{{ job.width?job.width+' × '+job.height+' · ':'' }}{{ Math.round((job.elapsedMs||0)/1000) }}s elapsed</p>
<progress class="h-2 w-full accent-amber-300" max="100" :value="job.progress || 0" />
<p class="text-xs text-zinc-400">{{ job.message }}</p>
</div>
<div v-for="job in captionJobs" :key="job.id" class="space-y-2 rounded-xl border border-amber-300/25 bg-zinc-900 p-3 text-sm" aria-live="polite" role="status">
<div class="flex justify-between gap-3">
<div class="flex min-w-0 items-start gap-3">
<img v-if="job.request?.stillId || job.request?.imageAId" :src="stillUrl(job.request.stillId || job.request.imageAId)" class="h-14 w-14 shrink-0 rounded object-cover" alt="Caption source">
<div class="min-w-0">
<strong class="block truncate">Describe · {{ job.request?.captionStyle || 'caption' }}</strong>
<span class="capitalize text-amber-200">{{ job.state==='waiting'?'queued':job.state }}</span>
</div>
</div>
<button v-if="!['complete','failed','cancelled'].includes(job.state)" type="button" class="shrink-0 text-zinc-300" @click="cancel(job)">Cancel</button>
</div>
<p class="text-xs text-zinc-400">{{ elapsedCaption(job) }}</p>
<p v-if="job.resultText" class="whitespace-pre-wrap text-zinc-200">{{ job.resultText }}</p>
<div v-if="job.resultText" class="flex flex-wrap gap-2">
<button type="button" class="rounded-full border border-white/20 px-3 py-1.5 text-xs outline-none ring-amber-300/40 focus-visible:ring-2" @click="copyCaption(job.resultText)">Copy</button>
<button type="button" class="rounded-full border border-amber-300/40 px-3 py-1.5 text-xs text-amber-100 outline-none ring-amber-300/40 focus-visible:ring-2" @click="useCaptionAsPrompt(job.resultText)">Use as prompt</button>
</div>
<p v-if="job.error" role="alert" class="text-red-300">{{ job.error }}</p>
</div>
<div v-for="job in musicJobs" :key="job.id" class="space-y-2 rounded-xl border border-amber-300/25 bg-zinc-900 p-3 text-sm" aria-live="polite" role="status">
<div class="flex justify-between gap-3">
<strong class="truncate">{{ job.name || 'YuE2' }}</strong>
<span class="capitalize text-amber-200">{{ job.live?.status || job.status }}</span>
</div>
<p class="text-xs text-zinc-400">YuE2 · target {{ job.duration || '—' }}s<span v-if="job.live?.message"> · {{ job.live.message }}</span></p>
<progress v-if="job.live?.progress" class="h-2 w-full accent-amber-300" max="100" :value="job.live.progress || 0" />
<p v-if="job.lastError || job.live?.error" role="alert" class="text-red-300">{{ job.lastError || job.live?.error }}</p>
</div>
<Studio2JobProgress v-for="job in benchJobs" :key="job.id" :job="job" :now="now" @select="focusJob(job)" @cancel="cancel(job)" />
<button v-if="folderJobs.some(j=>['complete','failed','cancelled'].includes(j.state)) || musicJobs.some(j=>['complete','done','error','cancelled'].includes(j.status)) || captionJobs.some(j=>['complete','failed','cancelled'].includes(j.state))" type="button" class="text-sm text-zinc-400 outline-none ring-amber-300/40 focus-visible:ring-2" @click="clearFinished">Clear finished</button>
</section> </section>
</div> </div>
<div v-if="settingsOpen || detailsOpen" class="fixed inset-0 z-50 bg-black/70" @click.self="settingsOpen=false;detailsOpen=false"> <div v-if="settingsOpen || detailsOpen" class="fixed inset-0 z-50 bg-black/70" @click.self="settingsOpen=false;detailsOpen=false">
<aside class="absolute inset-y-0 right-0 w-full max-w-md overflow-auto border-l border-white/15 bg-zinc-950 p-6"><button class="float-right" @click="settingsOpen=false;detailsOpen=false">Close</button> <aside class="absolute inset-y-0 right-0 w-full max-w-md overflow-auto border-l border-white/15 bg-zinc-950 p-6">
<template v-if="settingsOpen"><h2 class="mb-6 text-xl font-semibold">Generation settings</h2><label class="mb-4 block">Engine<select v-model="form.engine" class="mt-1 w-full rounded bg-zinc-800 p-2"><template v-if="video"><option value="minimax">MiniMax H3</option></template><template v-else><option value="flux">Flux Klein 9B Base</option><option value="krea" :disabled="['edit','compose','iterate'].includes(form.mode)">Krea 2 (Generate / Refine)</option></template></select></label><AspectRatioPick v-model="form.settings.aspect" heading="Aspect" :options="aspectOptions" /><p class="my-3 text-sm text-zinc-400">{{ resolvedSize.width }} × {{ resolvedSize.height }} · fit with padding, never stretch</p><label v-for="key in ['steps','cfg','seed',...(video?['duration','fps']:[])]" :key="key" class="mb-4 block capitalize">{{ key }}<input v-model.number="form.settings[key]" type="number" class="mt-1 w-full rounded bg-zinc-800 p-2"></label><label v-if="video"><input v-model="form.settings.turbo" type="checkbox"> Turbo</label><LoraSelect v-model="form.settings.loraStack" class="mt-5" :names="video?loraNames.video:loraNames.image" :engine="video?'':form.engine" :clip-strength="true" @insert-trigger="form.promptSections.action += ' '+$event" /><button class="mt-2 text-sm text-amber-200" @click="loadLoras">Refresh LoRAs</button></template> <button type="button" class="float-right outline-none ring-amber-300/40 focus-visible:ring-2" @click="settingsOpen=false;detailsOpen=false">Close</button>
<template v-else><h2 class="mb-6 text-xl font-semibold">Output details</h2><Studio2Details :metadata="selected?.studio2" /></template> <template v-if="settingsOpen">
<h2 class="mb-6 text-xl font-semibold">Generation settings</h2>
<AspectRatioPick v-model="form.settings.aspect" heading="Aspect" :options="aspectOptions" />
<p class="my-3 text-sm text-zinc-400">{{ resolvedSize.width }} × {{ resolvedSize.height }} · fit with padding, never stretch</p>
<label v-for="key in ['steps','cfg',...(video?['duration','fps']:[])]" :key="key" class="mb-4 block capitalize">{{ key }}
<input v-model.number="form.settings[key]" type="number" class="mt-1 w-full rounded bg-zinc-800 p-2 outline-none ring-amber-300/40 focus:ring-2">
</label>
<label class="mb-4 block">Seed · {{ form.settings.seedMode==='locked'?'Locked':'Random each job' }}
<input :value="form.settings.seed" type="number" min="0" max="2147483647" class="mt-1 w-full rounded bg-zinc-800 p-2 outline-none ring-amber-300/40 focus:ring-2" @input="form.settings.seed=Number(($event.target as HTMLInputElement).value);form.settings.seedMode='locked'">
</label>
<button type="button" class="mb-4 text-sm text-amber-200 outline-none ring-amber-300/40 focus-visible:ring-2" @click="form.settings.seedMode='random';form.settings.seed=randomSeed()">Random</button>
<label v-if="video"><input v-model="form.settings.turbo" type="checkbox" class="accent-amber-400"> Turbo</label>
<template v-if="form.engine!=='qwen21'">
<LoraSelect v-model="form.settings.loraStack" class="mt-5" :names="pickerLoras" :use="currentLoraUse" :clip-strength="!video" filename-subtitle :aliases="{...loraNames.aliases,...library.loraAliases}" :triggers="{...loraNames.triggers,...library.loraTriggers}" missing-label="No cached LoRAs yet — Refresh to check Comfy" @insert-trigger="form.promptSections.action += ' '+$event" />
<button type="button" class="mt-2 text-sm text-amber-200 outline-none ring-amber-300/40 focus-visible:ring-2" @click="loadLoras(true)">Refresh LoRAs</button>
</template>
<details class="mt-6"><summary>Compiled prompt</summary>
<pre class="whitespace-pre-wrap text-xs text-zinc-400">{{ compiled }}</pre>
<p v-if="!video" class="mt-3 text-xs text-zinc-400">Negative: {{ stylePrompt(form.imageStyles,true) || 'None' }}</p>
</details>
</template>
<template v-else>
<h2 class="mb-6 text-xl font-semibold">Output details</h2>
<Studio2Details :metadata="selected?.studio2" />
</template>
</aside> </aside>
</div> </div>
<div v-if="picker" class="fixed inset-0 z-50 flex items-center justify-center bg-black/80 p-6" @click.self="picker=''">
<section role="dialog" aria-modal="true" aria-label="Choose source asset" class="max-h-[80vh] w-full max-w-4xl overflow-auto rounded-2xl border border-white/15 bg-zinc-950 p-5">
<div class="mb-4 flex justify-between"><h2>Choose {{ picker==='identityStillId'?'hero':'source' }}</h2><button type="button" @click="picker=''">Close</button></div>
<button type="button" class="mb-4 text-sm text-zinc-400" @click="pickAsset(null)">Clear slot</button>
<div class="grid grid-cols-2 gap-3 sm:grid-cols-4 lg:grid-cols-5">
<button v-for="a in pickerAssets" :key="a.id" type="button" class="rounded-xl border border-white/15 p-2 text-left outline-none ring-amber-300/40 focus-visible:ring-2" @click="pickAsset(a)">
<span v-if="isHidden(a)" class="flex h-24 items-center text-xs text-zinc-400">Still loaded · preview hidden</span>
<img v-else :src="a.kind==='clip'?'/api/library/clips/'+a.id+'/thumb':stillUrl(a.id)" class="h-24 w-full object-contain" alt="Source option">
<span class="mt-2 block truncate text-xs">{{ assetLabel(a) }}</span>
</button>
</div>
</section>
</div>
</main> </main>
</template> </template>
<script setup lang="ts"> <script setup lang="ts">
import {createLoraMemory,loraUse} from '~/shared/studio2/lora-memory.mjs'
import {filterLorasForStudio,filterLorasForUse} from '~/utils/loras'
import {compilePrompt,sectionsFrom} from '~/shared/studio2/contracts.mjs' import {compilePrompt,sectionsFrom} from '~/shared/studio2/contracts.mjs'
import {aspectOptions,resolveSize,bindClip,restoreAsset,extendSelected} from '~/shared/studio2/input.mjs' import {CAPTION_STYLES} from '~/shared/studio2/caption.mjs'
const loraNames=ref<any>({image:[],video:[]}),now=ref(Date.now()),gpuLabel=ref('Checking…') import {aspectOptions,resolveSize,bindClip,changeSource,restoreAll,useAsHero,useAsSource} from '~/shared/studio2/input.mjs'
const library=ref<any>({folders:[],stills:[],clips:[]}),jobs=ref<any[]>([]),selected=ref<any>(null),family=ref<any[]>([]) import {folderAssets,newestOutput,assetLabel,reconcileSelection,previewHidden} from '~/shared/studio2/viewer.mjs'
const form=reactive<any>({mode:'generate',engine:'flux',lockFace:true,lockOutfit:true,hideInput:false,folderId:'',identityStillId:'',imageAId:'',imageBId:'',maskId:'',startClipId:'',endStillId:'',guides:[],promptSections:sectionsFrom({}),settings:{aspect:'auto',fps:24,width:1344,height:768,steps:24,cfg:4,seed:1,duration:5,turbo:true,loraStack:[]}}) import {queueJobs,elapsedSeconds} from '~/shared/studio2/queue-view.mjs'
import {gpuStatus} from '~/shared/studio2/status.mjs'
import {applyUploadedStill} from '~/shared/studio2/upload.mjs'
import {parseBatch,stylePrompt} from '~/shared/studio2/styles.mjs'
import {randomSeed} from '~/shared/studio2/seed.mjs'
import {DEFAULT_MUSIC_LYRICS,DEFAULT_MUSIC_TAGS} from '~/utils/music'
const xaigen=useRuntimeConfig().public.studio==='xaigen'
const {displayMode,isStudio,isClassic}=useDisplayMode()
const classicHideInput=ref(true)
const classicHideOutput=ref(true)
const picker=ref('')
const uploading=ref(false),uploadMessage=ref('')
const revealed=ref<Set<string>>(new Set())
const measured=reactive<Record<string,{width:number;height:number}>>({}),seenOutputs=new Set<string>()
const loraNames=ref<any>({image:[],video:[]}),now=ref(Date.now()),healthState=ref<any>(null),upscales=ref<any[]>([])
const library=ref<any>({folders:[],stills:[],clips:[],tracks:[]}),jobs=ref<any[]>([]),selected=ref<any>(null)
const studioQueue=ref<any[]>([])
const liveMusic=ref<Record<string,any>>({})
const form=reactive<any>({mode:'generate',engine:'flux',imageStyles:{positive:[],negative:[]},lockFace:true,lockOutfit:true,hideInput:xaigen,folderId:'',identityStillId:'',imageAId:'',imageBId:'',maskId:'',startClipId:'',endStillId:'',guides:[],promptSections:sectionsFrom({}),settings:{aspect:'auto',fps:24,steps:24,cfg:4,seed:null,seedMode:'random',duration:5,turbo:true,loraStack:[]}})
const music=reactive({tags:DEFAULT_MUSIC_TAGS,lyrics:DEFAULT_MUSIC_LYRICS,duration:60,seed:'',name:''})
const tagsPlaceholder=DEFAULT_MUSIC_TAGS,lyricsPlaceholder=DEFAULT_MUSIC_LYRICS
const captionStyles=CAPTION_STYLES
const captionStyle=ref('descriptive')
const captionStillId=ref('')
let loraMemory=createLoraMemory(xaigen)
const currentLoraUse=computed(()=>loraUse(form))
const pickerLoras=computed(()=>!xaigen && currentLoraUse.value==='ltx'?[]:filterLorasForUse(filterLorasForStudio([...loraNames.value.image,...loraNames.value.video],xaigen),currentLoraUse.value))
watch(()=>[currentLoraUse.value,form.settings.loraStack],()=>loraMemory.sync(form),{deep:true,flush:'sync'})
const route=useRoute() const route=useRoute()
const tasks=['generate','edit','compose','refine','iterate','video','extend'],advanced=ref(false),settingsOpen=ref(false),detailsOpen=ref(false),submitting=ref(false),error=ref(''),batchText=ref('') const tasks=['generate','edit','compose','refine','video','extend','music','caption']
const video=computed(()=>['video','extend'].includes(form.mode)),stills=computed(()=>library.value.stills.filter((s:any)=>s.folderId===form.folderId)),clips=computed(()=>library.value.clips.filter((s:any)=>s.folderId===form.folderId)),compiled=computed(()=>compilePrompt(form.promptSections,video.value,form)) const settingsOpen=ref(false),detailsOpen=ref(false),submitting=ref(false),error=ref(''),batchText=ref('')
const video=computed(()=>['video','extend'].includes(form.mode))
const kreaSticky=computed(()=>['edit','compose','iterate'].includes(form.mode))
const qwenOnlyGenerate=computed(()=>form.mode!=='generate')
const engineOptions=computed(()=>{
if(video.value){
return [
{value:'minimax',label:'MiniMax H3',disabled:false,hint:''},
{value:'ltx',label:'LTX',disabled:!xaigen,hint:xaigen?'':'xAIGen only'}
]
}
return [
{value:'flux',label:'Klein',disabled:false,hint:''},
{value:'krea',label:'Krea 2',disabled:kreaSticky.value,hint:kreaSticky.value?'sticky hero = Klein':''},
{value:'qwen21',label:'Qwen 2.1',disabled:qwenOnlyGenerate.value,hint:qwenOnlyGenerate.value?'Qwen 2.1 is T2I in this build':''}
]
})
watch(engineOptions,opts=>{
const current=opts.find(o=>o.value===form.engine)
if(!current || current.disabled)form.engine=opts.find(o=>!o.disabled)?.value || form.engine
})
watch(()=>form.engine,(engine,prev)=>{
if(engine==='qwen21' && prev!=='qwen21'){
form.settings.steps=25
form.settings.cfg=1
form.lockFace=false
form.lockOutfit=false
form.settings.loraStack=[]
}
})
const stills=computed(()=>library.value.stills.filter((s:any)=>s.folderId===form.folderId))
const clips=computed(()=>library.value.clips.filter((s:any)=>s.folderId===form.folderId))
const compiled=computed(()=>compilePrompt(form.promptSections,video.value,form))
const stillUrl=(id:string)=>`/api/library/stills/${id}` const stillUrl=(id:string)=>`/api/library/stills/${id}`
const folderJobs=computed(()=>jobs.value.filter(j=>j.request.folderId===form.folderId)) const folderJobs=computed(()=>jobs.value.filter(j=>j.request.folderId===form.folderId))
const activeJob=computed(()=>folderJobs.value.find(j=>!['complete','failed','cancelled'].includes(j.state))) const captionJobs=computed(()=>folderJobs.value.filter(j=>j.request?.mode==='caption'))
const sourceSize=computed(()=>form.startClipId?clips.value.find((c:any)=>c.id===form.startClipId):stills.value.find((s:any)=>s.id===(form.imageAId||form.identityStillId))) const benchJobs=computed(()=>queueJobs(folderJobs.value.filter(j=>j.request?.mode!=='caption')))
const resolvedSize=computed(()=>resolveSize(form.settings.aspect,sourceSize.value,form.settings)) const musicJobs=computed(()=>studioQueue.value
function chooseTask(task:string){if(task==='extend'){Object.assign(form,extendSelected(form,selected.value?.kind==='clip'?selected.value:null));return}const was=video.value;form.mode=task;form.startClipId='';if(video.value!==was){form.engine=video.value?'minimax':'flux';form.settings.steps=video.value?8:24;form.settings.cfg=video.value?1.5:4}} .filter((j:any)=>j.kind==='music' && j.folderId===form.folderId && !['complete','done'].includes(j.status))
function chooseClip(id:string){if(!id){form.startClipId='';return}const clip=clips.value.find((c:any)=>c.id===id);if(clip)Object.assign(form,bindClip(form,clip))} .map((j:any)=>({...j,live:j.liveJobId?liveMusic.value[j.liveJobId]:null})))
function selectOutput(item:any){selected.value=item;if(item?.kind==='clip' && form.startClipId!==item.id){Object.assign(form,bindClip(form,item));form.mode='extend'}} const gpu=computed(()=>gpuStatus(jobs.value,healthState.value,upscales.value,musicJobs.value,captionJobs.value))
function restore(item:any){Object.assign(form,restoreAsset(form,item));form.guides=form.guides||[];batchText.value=''} const captionStill=computed(()=>stills.value.find((s:any)=>s.id===captionStillId.value))
function focusJob(job:any){family.value=job.outputs||[];selectOutput(family.value.at(-1)||null)} const sourceSlot=computed(()=>{const clip=clips.value.find((a:any)=>a.id===form.startClipId);return form.mode==='extend'?(clip?{...clip,kind:'clip'}:null):inputStill.value})
async function loadLoras(){loraNames.value=await $fetch('/api/loras').catch(()=>({image:[],video:[]}))} const pickerAssets=computed(()=>picker.value==='startClipId'?clips.value.map((a:any)=>({...a,kind:'clip'})):stills.value.map((a:any)=>({...a,kind:'still'})))
async function queue(){submitting.value=true;error.value='';try{await $fetch('/api/studio-2/jobs',{method:'POST',body:{...form,settings:{...form.settings,...resolvedSize.value},batch:batchText.value.split('\n').map(s=>s.trim()).filter(Boolean)}});await refresh()}catch(e:any){error.value=e.data?.statusMessage||e.message}finally{submitting.value=false}} const inputStill=computed(()=>stills.value.find((s:any)=>s.id===form.imageAId))
async function cancel(job:any){await $fetch('/api/studio-2/cancel',{method:'POST',body:{id:job.id}});await refresh()} const family=computed(()=>folderAssets(library.value,jobs.value,form.folderId))
async function upload(event:Event){const file=(event.target as HTMLInputElement).files?.[0];if(!file)return;const body=new FormData();body.append('image',file);body.append('folderId',form.folderId);try{library.value=await $fetch('/api/library/stills',{method:'POST',body})}catch(e:any){error.value=e.data?.statusMessage||e.message}} const sourceAsset=computed(()=>form.startClipId?clips.value.find((c:any)=>c.id===form.startClipId):stills.value.find((s:any)=>s.id===(form.imageAId||form.identityStillId)))
async function refresh(){ const sourceSize=computed(()=>measured[sourceAsset.value?.id] || sourceAsset.value)
jobs.value=await $fetch<any[]>('/api/studio-2/jobs') const resolvedSize=computed(()=>resolveSize(form.settings.aspect,sourceSize.value,undefined,form.engine))
const latest=jobs.value.find(j=>j.request.folderId===form.folderId&&j.outputs?.length)
if(latest&&!selected.value){library.value=await $fetch('/api/library?light=1');focusJob(latest);return} function elapsedCaption(job:any){
const current=jobs.value.find(j=>j.outputs?.some((o:any)=>family.value.some(f=>f.id===o.id))) const seconds=elapsedSeconds(job,now.value)
if(current&&current.outputs.some((o:any)=>!family.value.some(f=>f.id===o.id))){ return `${Math.floor(seconds/60)}m ${seconds%60}s ${job.startedAt||job.state==='rendering'||job.state==='saving'?'elapsed':'waiting'}`
const following=selected.value?.id===family.value.at(-1)?.id
family.value=current.outputs
if(following)selectOutput(family.value.at(-1))
library.value=await $fetch('/api/library?light=1')
}
} }
watch(()=>form.folderId,id=>{if(import.meta.client)form.identityStillId=localStorage.getItem(`studio2-hero:${id}`)||''},{flush:'sync'}) async function copyCaption(text:string){
try{await navigator.clipboard.writeText(text)}catch{error.value='Could not copy caption.'}
}
function useCaptionAsPrompt(text:string){
const current=String(form.promptSections.action||'').trim()
form.promptSections.action=current?`${current}\n\n${text}`:text
if(form.mode==='caption' || form.mode==='music')form.mode='generate'
}
async function queueCaption(stillId:string){
if(!stillId){error.value='Choose a still to describe.';return false}
try{
await $fetch('/api/studio-2/caption',{method:'POST',body:{folderId:form.folderId,stillId,sourcePath:stillId,captionStyle:captionStyle.value}})
await refresh()
return true
}catch(e:any){error.value=e.data?.statusMessage||e.message;return false}
}
function describeSelected(){
const id=selected.value?.kind==='still'?selected.value.id:(selected.value?.kind==='clip'? '' : (selected.value?.id || form.imageAId || form.identityStillId))
if(!id){error.value='Select a still to describe.';return}
captionStillId.value=id
submitting.value=true;error.value=''
void queueCaption(id).finally(()=>{submitting.value=false})
}
async function uploadCaption(event:Event){
const input=event.target as HTMLInputElement,file=input.files?.[0];if(!file)return
uploading.value=true;uploadMessage.value='';error.value=''
const body=new FormData();body.append('image',file);body.append('folderId',form.folderId);body.append('hideInput',String(xaigen))
try{const response=await $fetch<any>('/api/studio-2/stills',{method:'POST',body});library.value=response;captionStillId.value=response.still.id;measured[response.still.id]={width:response.still.width,height:response.still.height};uploadMessage.value=`Loaded ${response.still.name || file.name} for caption.`}
catch(e:any){error.value=e.data?.statusMessage||e.message}
finally{uploading.value=false;input.value=''}
}
function isHidden(asset:any){
if(!xaigen || !asset?.id)return false
if(isClassic.value){
const input=asset.role==='input'
return input?classicHideInput.value:classicHideOutput.value
}
return previewHidden(asset,revealed.value,xaigen)
}
function slotHidden(id:string){
if(!xaigen || !id)return false
if(isClassic.value)return classicHideInput.value
return !revealed.value.has(id)
}
function toggleReveal(id:string){
if(!id)return
const next=new Set(revealed.value)
if(next.has(id))next.delete(id);else next.add(id)
revealed.value=next
if(import.meta.client)localStorage.setItem(`studio2-revealed:${form.folderId}`,JSON.stringify([...next]))
}
function loadRevealed(folderId:string){
if(!import.meta.client||!xaigen){revealed.value=new Set();return}
try{revealed.value=new Set(JSON.parse(localStorage.getItem(`studio2-revealed:${folderId}`)||'[]'))}
catch{revealed.value=new Set()}
}
watch(()=>sourceAsset.value?.id,id=>{const asset=sourceAsset.value;if(!import.meta.client||!asset||measured[id])return;const media=asset.id===form.startClipId?document.createElement('video'):new Image();if(media.tagName==='VIDEO')(media as HTMLVideoElement).preload='metadata';const done=()=>{measured[asset.id]={width:(media as HTMLVideoElement).videoWidth||(media as HTMLImageElement).naturalWidth,height:(media as HTMLVideoElement).videoHeight||(media as HTMLImageElement).naturalHeight}};media.addEventListener(media.tagName==='VIDEO'?'loadedmetadata':'load',done,{once:true});media.src=media.tagName==='VIDEO'?`/api/library/clips/${asset.id}/video`:stillUrl(asset.id)})
function chooseTask(task:string){
if(task==='music'){form.mode='music';return}
if(task==='caption'){
form.mode='caption'
captionStillId.value=selected.value?.kind==='still'?selected.value.id:form.imageAId||form.identityStillId||captionStillId.value
return
}
if(task==='extend'){
form.mode='extend'
form.engine=xaigen && form.engine==='ltx'?'ltx':'minimax'
const clip=selected.value?.kind==='clip'?selected.value:clips.value.find((c:any)=>c.id===form.startClipId)
if(clip?.id){
try{Object.assign(form,bindClip(form,clip,xaigen,loraMemory))}
catch(e:any){error.value=e?.message||'Could not copy clip settings. Previous clip is still bound.';form.startClipId=clip.id}
}
return
}
const was=video.value;form.mode=task;form.startClipId='';if(video.value!==was){form.engine=video.value?(xaigen && form.engine==='ltx'?'ltx':'minimax'):'flux';form.settings.steps=video.value?8:24;form.settings.cfg=video.value?1.5:4}
}
function onTaskSelect(event:Event){chooseTask((event.target as HTMLSelectElement).value)}
function chooseClip(id:string){if(!id){form.startClipId='';return}const clip=clips.value.find((c:any)=>c.id===id);if(clip){try{Object.assign(form,bindClip(form,clip,xaigen,loraMemory))}catch(e:any){error.value=e?.message||'Could not copy clip settings.';form.startClipId=clip.id;form.mode='extend';form.engine=xaigen && form.engine==='ltx'?'ltx':'minimax'}}}
function selectOutput(item:any){selected.value=item;if(form.mode==='extend' && item?.kind==='clip')chooseClip(item.id)}
function restore(item:any){Object.assign(form,restoreAll(form,item,xaigen,loraMemory));form.guides=form.guides||[];batchText.value=''}
function focusJob(job:any){selectOutput(job.outputs?.at(-1)||null)}
async function loadLoras(fresh=false){try{loraNames.value=await $fetch('/api/studio-2/loras'+(fresh?'?fresh=1':''))}catch{error.value='Could not refresh LoRAs. The last loaded list is still available.'}}
function pickAsset(asset:any){const key=picker.value;if(key.startsWith('guide:'))form.guides[Number(key.split(':')[1])].stillId=asset?.id||'';else if(key==='startClipId')chooseClip(asset?.id||'');else if(key==='captionStillId')captionStillId.value=asset?.id||'';else form[key]=asset?.id||'';picker.value=''}
async function clearFinished(){await $fetch('/api/studio-2/clear-finished',{method:'POST'});await refresh()}
async function deleteOutput(){
if(!selected.value)return
try{
const kind=selected.value.kind==='clip'?'clips/':selected.value.kind==='track'?'tracks/':'stills/'
await $fetch('/api/library/'+kind+selected.value.id,{method:'DELETE'})
await refresh()
}catch(e:any){error.value=e.data?.statusMessage||e.message}
}
async function queue(){
submitting.value=true;error.value=''
try{
if(form.mode==='music'){
await $fetch('/api/generate/music',{method:'POST',body:{engine:'yue2',tags:music.tags,lyrics:music.lyrics,duration:music.duration,seed:music.seed||'random',folderId:form.folderId,name:music.name}})
await refresh()
return
}
if(form.mode==='caption'){
await queueCaption(captionStillId.value || form.imageAId || form.identityStillId)
return
}
if(form.settings.seedMode!=='locked')form.settings.seed=randomSeed()
const queued=await $fetch<any>('/api/studio-2/jobs',{method:'POST',body:{...form,mode:form.mode==='edit' && parseBatch(batchText.value).length?'iterate':form.mode,hideInput:xaigen,hideOutput:false,promptSections:video.value?form.promptSections:{action:form.promptSections.action},settings:{...form.settings,...resolvedSize.value},batch:video.value || form.mode==='edit'?parseBatch(batchText.value):[]}})
form.settings.seed=queued.seed;form.settings.width=queued.width;form.settings.height=queued.height
await refresh()
}catch(e:any){error.value=e.data?.statusMessage||e.message}
finally{submitting.value=false}
}
async function cancel(job:any){await $fetch('/api/studio-2/cancel',{method:'POST',body:{id:job.id}});await refresh()}
async function upload(event:Event,target:'source'|'hero'){
const input=event.target as HTMLInputElement,file=input.files?.[0];if(!file)return
uploading.value=true;uploadMessage.value='';error.value=''
const body=new FormData();body.append('image',file);body.append('folderId',form.folderId);body.append('hideInput',String(xaigen))
try{const response=await $fetch<any>('/api/studio-2/stills',{method:'POST',body});library.value=response;Object.assign(form,applyUploadedStill(form,response,target));measured[response.still.id]={width:response.still.width,height:response.still.height};uploadMessage.value=`Loaded ${response.still.name || file.name} as ${target==='hero'?'hero identity':'start still'}.`}
catch(e:any){error.value=e.data?.statusMessage||e.message}
finally{uploading.value=false;input.value=''}
}
async function refresh(){
const [nextJobs,nextLibrary,nextUpscales,queue]=await Promise.all([
$fetch<any[]>('/api/studio-2/jobs'),
$fetch<any>('/api/studio-2/library'),
$fetch<any[]>('/api/studio-2/upscales'),
$fetch<any>('/api/studio-queue?full=1').catch(()=>({jobs:[]}))
])
jobs.value=nextJobs;library.value=nextLibrary;upscales.value=nextUpscales
studioQueue.value=Array.isArray(queue)?queue:(queue?.jobs||[])
const musicRows=studioQueue.value.filter((j:any)=>j.kind==='music' && j.liveJobId && !['complete','done'].includes(j.status))
const liveEntries=await Promise.all(musicRows.map(async(j:any)=>{
try{return [j.liveJobId,await $fetch<any>(`/api/generate/${j.liveJobId}`)] as const}
catch{return [j.liveJobId,null] as const}
}))
const nextLive:Record<string,any>={}
for(const [id,snap] of liveEntries){if(snap)nextLive[id]=snap}
liveMusic.value=nextLive
const fresh=newestOutput(nextJobs,seenOutputs,form.folderId)
for(const job of nextJobs)for(const output of job.outputs||[])seenOutputs.add(output.id)
const existingFresh=family.value.find(a=>a.id===fresh?.id)
const newestTrack=(library.value.tracks||[]).filter((t:any)=>t.folderId===form.folderId).sort((a:any,b:any)=>(b.createdAt||0)-(a.createdAt||0))[0]
if(newestTrack && !seenOutputs.has(newestTrack.id)){seenOutputs.add(newestTrack.id);selected.value={...newestTrack,kind:'track'};return}
selected.value=existingFresh || reconcileSelection(selected.value,family.value)
}
watch(()=>form.folderId,id=>{selected.value=null;form.imageAId='';form.startClipId='';loadRevealed(id);if(import.meta.client)form.identityStillId=localStorage.getItem(`studio2-hero:${id}`)||''},{flush:'sync'})
watch(()=>form.identityStillId,id=>{if(import.meta.client&&form.folderId)localStorage.setItem(`studio2-hero:${form.folderId}`,id)}) watch(()=>form.identityStillId,id=>{if(import.meta.client&&form.folderId)localStorage.setItem(`studio2-hero:${form.folderId}`,id)})
watch(()=>form.hideInput,value=>{if(import.meta.client)localStorage.setItem('studio2-hide-input',String(value))})
let timer:ReturnType<typeof setTimeout>,clock:ReturnType<typeof setInterval>,disposed=false let timer:ReturnType<typeof setTimeout>,clock:ReturnType<typeof setInterval>,disposed=false
onMounted(async()=>{form.hideInput=localStorage.getItem('studio2-hide-input')==='true';clock=setInterval(()=>{now.value=Date.now()},1000);void loadLoras();library.value=await $fetch('/api/library?light=1');form.folderId=library.value.folders[0]?.id||'';const id=String(route.query.still||route.query.clip||'');const asset=[...library.value.stills,...library.value.clips].find((a:any)=>a.id===id);if(asset){selected.value={...asset,kind:route.query.clip?'clip':'still'};family.value=[selected.value];if(route.query.use==='1')restore(selected.value)};const poll=async()=>{try{await refresh();const health=await $fetch<any>('/api/health');gpuLabel.value=health.comfy?.busy?'Busy':health.comfy?.ok?'Ready':'Sleeping / unavailable'}catch(e:any){error.value=e.data?.statusMessage||e.message}if(!disposed)timer=setTimeout(poll,3000)};void poll()}) onMounted(async()=>{
let storage;try{storage=localStorage}catch{}
loraMemory=createLoraMemory(xaigen,storage);loraMemory.sync(form);form.settings.seed=randomSeed()
clock=setInterval(()=>{now.value=Date.now()},1000);void loadLoras(true)
library.value=await $fetch('/api/studio-2/library');form.folderId=library.value.folders[0]?.id||'';loadRevealed(form.folderId)
const id=String(route.query.still||route.query.clip||route.query.track||'')
const asset=[...library.value.stills,...library.value.clips,...(library.value.tracks||[])].find((a:any)=>a.id===id)
if(asset){selected.value={...asset,kind:route.query.clip?'clip':route.query.track?'track':'still'};if(route.query.use==='1'&&selected.value.kind!=='track')restore(selected.value);if(xaigen)toggleReveal(asset.id)}
if(route.query.task==='music')form.mode='music'
if(route.query.task==='caption')form.mode='caption'
const poll=async()=>{try{await refresh();const health=await $fetch<any>('/api/health');healthState.value=health}catch(e:any){healthState.value=null;error.value=e.data?.statusMessage||e.message}if(!disposed)timer=setTimeout(poll,3000)};void poll()
})
onBeforeUnmount(()=>{disposed=true;clearTimeout(timer);clearInterval(clock)}) onBeforeUnmount(()=>{disposed=true;clearTimeout(timer);clearInterval(clock)})
</script> </script>
+5 -1
View File
@@ -8,12 +8,16 @@ const props=defineProps<{metadata?:any}>()
const rows=computed(()=>{const m=props.metadata||{},s=m.settings||{};const time=(v:any)=>v?new Date(v).toLocaleString():null;return [ const rows=computed(()=>{const m=props.metadata||{},s=m.settings||{};const time=(v:any)=>v?new Date(v).toLocaleString():null;return [
{label:'Kind / mode',value:[m.kind,m.mode].filter(Boolean).join(' / ')}, {label:'Kind / mode',value:[m.kind,m.mode].filter(Boolean).join(' / ')},
{label:'Engine',value:m.engine},{label:'Graph ID',value:m.graphId}, {label:'Engine',value:m.engine},{label:'Graph ID',value:m.graphId},
{label:'Sample latent',value:m.sampleLatent},{label:'Sample denoise',value:m.sampleDenoise},
{label:'Source still',value:m.sourceStillId || m.imageAId || 'None'},
{label:'Hero reference attached',value:m.heroReferenceAttached==null?'Not recorded':m.heroReferenceAttached?'Yes':'No'},
{label:'Seed',value:s.seed},{label:'Steps',value:s.steps},{label:'CFG',value:s.cfg}, {label:'Seed',value:s.seed},{label:'Steps',value:s.steps},{label:'CFG',value:s.cfg},
{label:'Size',value:s.width&&s.height?`${s.width} × ${s.height}`:null},{label:'Duration',value:s.duration!=null?`${s.duration}s`:null}, {label:'Size',value:s.width&&s.height?`${s.width} × ${s.height}`:null},{label:'Duration',value:s.duration!=null?`${s.duration}s`:null},
{label:'LoRAs',value:s.loraStack?.length?s.loraStack.map((l:any)=>`${l.name} · ${l.strengthModel}`).join('\n'):'None'}, {label:'LoRAs',value:s.loraStack?.length?s.loraStack.map((l:any)=>`${l.name} · ${l.strengthModel}`).join('\n'):'None'},
{label:'Identity still',value:m.identityStillId||'None'},{label:'Face lock',value:m.lockFace!==false?'On':'Off'},{label:'Outfit lock',value:m.lockOutfit!==false?'On':'Off'}, {label:'Hero still',value:m.identityStillId||'None'},{label:'Face lock',value:m.lockFace!==false?'On':'Off'},{label:'Outfit lock',value:m.lockOutfit!==false?'On':'Off'},
{label:'Aspect',value:s.aspect||'auto'},{label:'FPS',value:s.fps||24},{label:'Hide input',value:m.hideInput?'On':'Off'}, {label:'Aspect',value:s.aspect||'auto'},{label:'FPS',value:s.fps||24},{label:'Hide input',value:m.hideInput?'On':'Off'},
{label:'Start frame',value:m.startFrameSource?JSON.stringify(m.startFrameSource):'None'}, {label:'Start frame',value:m.startFrameSource?JSON.stringify(m.startFrameSource):'None'},
{label:'Handoff source',value:m.handoffSource || 'None'},{label:'Handoff fitted (resampled)',value:m.fitted==null?'Not recorded':m.fitted?'Yes':'No'},
{label:'End still',value:m.endStillId||'None'},{label:'Guide stills',value:m.guides?.length?JSON.stringify(m.guides):'None'}, {label:'End still',value:m.endStillId||'None'},{label:'Guide stills',value:m.guides?.length?JSON.stringify(m.guides):'None'},
{label:'Compiled prompt',value:m.compiledPrompt},{label:'Queued',value:time(m.queuedAt)},{label:'Started',value:time(m.startedAt)},{label:'Saved',value:time(m.savedAt)}, {label:'Compiled prompt',value:m.compiledPrompt},{label:'Queued',value:time(m.queuedAt)},{label:'Started',value:time(m.startedAt)},{label:'Saved',value:time(m.savedAt)},
{label:'GPU seconds',value:m.gpuSeconds},{label:'Wall seconds',value:m.wallTime},{label:'Comfy prompt ID',value:m.promptId},{label:'Host cleanup',value:m.purgeResult} {label:'GPU seconds',value:m.gpuSeconds},{label:'Wall seconds',value:m.wallTime},{label:'Comfy prompt ID',value:m.promptId},{label:'Host cleanup',value:m.purgeResult}
+9
View File
@@ -0,0 +1,9 @@
<template>
<div role="status" aria-live="polite" class="flex min-w-0 flex-wrap items-center gap-3">
<span class="inline-flex shrink-0 items-center gap-2 rounded-full border px-4 py-2 text-base font-semibold" :class="status.tone==='ready'?'border-emerald-400/50 bg-emerald-400/15 text-emerald-200':status.tone==='busy'?'border-amber-300/50 bg-amber-300/15 text-amber-200':'border-zinc-600 bg-zinc-800 text-zinc-200'"><span class="h-2.5 w-2.5 rounded-full bg-current" :class="{'animate-pulse':status.tone==='busy'}" />GPU · {{ status.label }}</span>
<span class="max-w-2xl truncate text-sm text-zinc-300" :title="status.detail">{{ status.detail }}</span>
</div>
</template>
<script setup lang="ts">
defineProps<{status:{label:string;tone:string;detail:string}}>()
</script>
+3 -2
View File
@@ -1,15 +1,16 @@
<template> <template>
<div class="space-y-2 rounded-xl border border-white/10 p-3 text-sm" aria-live="polite"> <div class="space-y-2 rounded-xl border border-white/10 p-3 text-sm" aria-live="polite">
<div class="flex justify-between gap-3"><button class="text-left" @click="$emit('select')"><strong>{{ job.request.promptSections.action.slice(0,70) }}</strong><span class="block capitalize text-amber-200">{{ job.state==='waiting'?'queued':job.state }} · Shot {{ shot }}/{{ total }}</span></button><button v-if="!terminal" @click="$emit('cancel')">Cancel</button></div> <div class="flex justify-between gap-3"><button class="text-left" @click="$emit('select')"><strong>{{ job.request.promptSections.action.slice(0,70) }}</strong><span class="block capitalize text-amber-200">{{ job.state==='waiting'?'queued':job.state }} · Shot {{ shot }}/{{ total }}</span></button><button v-if="!terminal" @click="$emit('cancel')">Cancel</button></div>
<p>{{ elapsed }} elapsed · {{ job.request.settings.duration }}s target<span v-if="job.state==='rendering' && job.progress"> · Step {{ job.progress.step }}/{{ job.progress.maxStep }} ({{ job.progress.percent }}%)</span></p> <p>{{ elapsed }} {{ job.startedAt?'elapsed':'waiting' }} · {{ job.request.settings.duration }}s target<span v-if="job.state==='rendering' && job.progress"> · Step {{ job.progress.step }}/{{ job.progress.maxStep }} ({{ job.progress.percent }}%)</span></p>
<progress v-if="job.state==='rendering'" class="h-2 w-full accent-amber-300" :value="job.progress?.percent" max="100" /> <progress v-if="job.state==='rendering'" class="h-2 w-full accent-amber-300" :value="job.progress?.percent" max="100" />
<p v-if="job.error" role="alert" class="text-red-300">{{ job.error }}</p> <p v-if="job.error" role="alert" class="text-red-300">{{ job.error }}</p>
</div> </div>
</template> </template>
<script setup lang="ts"> <script setup lang="ts">
import {elapsedSeconds} from '~/shared/studio2/queue-view.mjs'
const props=defineProps<{job:any;now:number}>();defineEmits(['select','cancel']) const props=defineProps<{job:any;now:number}>();defineEmits(['select','cancel'])
const terminal=computed(()=>['complete','failed','cancelled'].includes(props.job.state)) const terminal=computed(()=>['complete','failed','cancelled'].includes(props.job.state))
const total=computed(()=>props.job.prompts?.length || (props.job.request.batch?.length || 0)+1) const total=computed(()=>props.job.prompts?.length || (props.job.request.batch?.length || 0)+1)
const shot=computed(()=>Math.min(total.value,(props.job.index || 0)+1)) const shot=computed(()=>Math.min(total.value,(props.job.index || 0)+1))
const elapsed=computed(()=>{const seconds=Math.max(0,Math.floor(((terminal.value?props.job.updatedAt:props.now)-(props.job.startedAt||props.job.queuedAt))/1000));return `${Math.floor(seconds/60)}m ${seconds%60}s`}) const elapsed=computed(()=>{const seconds=elapsedSeconds(props.job,props.now);return `${Math.floor(seconds/60)}m ${seconds%60}s`})
</script> </script>
+10
View File
@@ -0,0 +1,10 @@
<template>
<section class="space-y-3" aria-label="Image style">
<p class="text-xs text-zinc-400">Click to include · right-click to exclude. Gold = positive, red = negative.</p>
<div v-for="(chips,group) in styleGroups" :key="group"><h3 class="mb-1 text-sm text-zinc-400">{{ group }}</h3><div class="flex flex-wrap gap-2"><button v-for="chip in chips" :key="chip" type="button" class="rounded-full border px-3 py-1 text-sm" :class="model.positive?.includes(chip)?'border-amber-300 bg-amber-300/15 text-amber-200':model.negative?.includes(chip)?'border-red-400 bg-red-400/15 text-red-200':'border-white/15'" :aria-pressed="model.positive?.includes(chip)||model.negative?.includes(chip)" @click="model=toggleStyle(model,chip)" @contextmenu.prevent="model=toggleStyle(model,chip,true)">{{ model.negative?.includes(chip)?'− ':model.positive?.includes(chip)?'+ ':'' }}{{ chip }}</button></div></div>
</section>
</template>
<script setup lang="ts">
import {styleGroups,toggleStyle} from '~/shared/studio2/styles.mjs'
const model=defineModel<any>({default:()=>({positive:[],negative:[]})})
</script>
+82
View File
@@ -0,0 +1,82 @@
export type DisplayMode = 'classic' | 'studio'
const STORAGE_KEY = 'aigen.displayMode'
function parseMode(raw: unknown): DisplayMode | null {
const value = String(raw || '').trim().toLowerCase()
return value === 'classic' || value === 'studio' ? value : null
}
function readStored(): DisplayMode {
if (!import.meta.client) return 'studio'
try {
return parseMode(localStorage.getItem(STORAGE_KEY)) || 'studio'
} catch {
return 'studio'
}
}
function writeStored(mode: DisplayMode) {
if (!import.meta.client) return
try {
localStorage.setItem(STORAGE_KEY, mode)
} catch { /* ignore quota / private mode */ }
}
/** Classic | Studio chrome skin. One Bench tree; layout only. */
export function useDisplayMode() {
const route = useRoute()
const displayMode = useState<DisplayMode>('aigen.displayMode', () => 'studio')
const sessionOverride = useState<DisplayMode | null>('aigen.displayMode.session', () => null)
const hydrated = useState('aigen.displayMode.hydrated', () => false)
const resolved = computed<DisplayMode>(() => {
const fromQuery = parseMode(route.query.display)
if (fromQuery) return fromQuery
if (sessionOverride.value) return sessionOverride.value
return displayMode.value
})
function setDisplayMode(mode: DisplayMode, opts?: { persist?: boolean; session?: boolean }) {
const persist = opts?.persist !== false
if (opts?.session) sessionOverride.value = mode
else sessionOverride.value = null
displayMode.value = mode
if (persist) writeStored(mode)
}
function hydrate() {
if (hydrated.value) return
hydrated.value = true
const fromQuery = parseMode(route.query.display)
if (fromQuery) {
sessionOverride.value = fromQuery
return
}
displayMode.value = readStored()
}
if (import.meta.client) {
onMounted(hydrate)
watch(
() => route.query.display,
(raw) => {
const fromQuery = parseMode(raw)
if (fromQuery) sessionOverride.value = fromQuery
else if (sessionOverride.value && !parseMode(route.query.display)) {
// Drop session override when query cleared; keep stored preference.
sessionOverride.value = null
displayMode.value = readStored()
}
}
)
}
return {
displayMode: resolved,
storedDisplayMode: displayMode,
setDisplayMode,
isStudio: computed(() => resolved.value === 'studio'),
isClassic: computed(() => resolved.value === 'classic')
}
}
+40
View File
@@ -0,0 +1,40 @@
# Studio 2 image→text caption (Qwen2.5-VL NSFW Caption V4 GGUF)
Caption jobs run on the Windows GPU host via **llama-server** (llama.cpp vision / mmproj). Not Comfy, not Klein, not JoyCaption.
## Weights (~7GB — two files only)
```powershell
powershell -ExecutionPolicy Bypass -File scripts\setup-caption.ps1
```
Installs into the Shared models tree:
`%LOCALAPPDATA%\Comfy-Desktop\ComfyUI-Shared\models\caption\qwen25vl-7b-nsfw-v4\`
| File | Role |
| --- | --- |
| `Qwen2.5-VL-7B-NSFW-Caption-V4.Q5_K_M.gguf` | Language model |
| `Qwen2.5-VL-7B-NSFW-Caption-V4.mmproj-f16.gguf` | Vision projector |
Do **not** download the full 16.6GB safetensors repo.
Runtime needs ~8–10GB VRAM. Exclusive GPU: host stops Comfy (and waits if YuE2/upscale is busy), loads the GGUF for one shot, then kills llama-server (`keep_alive 0`).
## Host agent
Requires `llama-server` on PATH (`winget install ggml.llamacpp`) or `CAPTION_LLAMA_SERVER`.
Restart the Comfy host agent after setup. Confirm:
```text
GET http://127.0.0.1:8199/caption/status → { configured, busy, backend: "llama.cpp" }
POST /caption { imagePath, style } → load → { text } → unload
POST /caption/jobs + PUT …/input → async job used by Studio 2
```
Styles: `descriptive` | `klein_prompt` | `delta` | `tags`.
## App
`POST /api/studio-2/caption` with `{ folderId, stillId|sourcePath, captionStyle }` queues a Studio job. Bench **Describe** button and the **caption** task pill both enqueue. Result stores `resultText` on the studio2 job and optionally writes `{still}.txt` beside the library file.
+42
View File
@@ -0,0 +1,42 @@
# Qwen Image 2.1 (engine `qwen21`) — host weights + Comfy graph
## Host paths (this RTX 5080 box)
| Role | Resolved path |
|---|---|
| ComfyUI root (Klein / host agent `ComfyUI (1)`) | `C:\Users\ianjm\AppData\Local\Comfy-Desktop\ComfyUI-Installs\ComfyUI (1)\ComfyUI` |
| Models root (Desktop Shared via `shared_model_paths.yaml`) | `C:\Users\ianjm\AppData\Local\Comfy-Desktop\ComfyUI-Shared\models` |
Weights must land under Shared so every Desktop instance sees them:
- `diffusion_models\qwen-image-2.1-Q8_0.gguf` (~7.59 GiB) — from `abenzerps/Qwen-Image-2.1-Uncensored-GGUF`
- `text_encoders\qwen3vl_8b_int8_convrot.safetensors` — from `Comfy-Org/Qwen-Image-2.1` (INT8; BF16 8B VL OOMs on 16 GB)
- `vae\qwen_image_2.1_vae_bf16.safetensors` — from `Comfy-Org/Qwen-Image-2.1` (**not** the old Qwen-Image 1.0 VAE)
Custom node: `custom_nodes\ComfyUI-GGUF` (`UnetLoaderGGUF`). Do not install a second GGUF pack.
## Setup
```powershell
powershell -ExecutionPolicy Bypass -File scripts\setup-qwen21.ps1
```
DiT download uses the exact host invocation:
```powershell
hf download hf://abenzerps/Qwen-Image-2.1-Uncensored-GGUF/qwen-image-2.1-Q8_0.gguf
```
That lands in the Hugging Face hub cache; the setup script copies it into Shared `diffusion_models`. The abenzerps GGUF ships with `kv_count=0` (no `general.architecture`) **and** Q8_0-quantized 1D RMSNorm weights (logical 128 → packed 136), which breaks Comfy’s fused `rms_rope`. `scripts/tag-qwen21-gguf.py` rewrites the file with `general.architecture=qwen_image` and promotes small/1D tensors to F32 so city96 `UnetLoaderGGUF` can load it.
Restart Comfy **only when idle** (`COMFY_CONTROL_URL/status` → `gpu.busy=false`). Confirm `object_info` lists `UnetLoaderGGUF` and the three filenames.
## App
- Engine key: `qwen21` · UI label: **Qwen 2.1**
- Generate (T2I) only in this build. Edit / Compose / Iterate / Video disable with: “Qwen 2.1 is T2I in this build”.
- Sampler defaults: euler / simple / cfg **1** / steps **25** · `ModelSamplingAuraFlow` shift **3.1**
- Default canvas **1024×1024** (aspect 16:9 / 9:16 / 1:1 → long-edge square via `TextEncodeQwenImage21` resolution; multiples of 32). Drop to 768 if VRAM errors.
- No hero / locks / Klein LoRA stack on this engine.
Graph: `server/assets/studio2_qwen21_t2i.json`.
+11
View File
@@ -0,0 +1,11 @@
# Production LoRA names in Studio 2
LoRA keys are user library data, not Git branch files. Separate dev storage does not contain production aliases automatically.
The dev host runs `xaigen-dev-lora-catalog.path`, watching the production user's catalog directory. Its oneshot service runs `server/utils/studio2/export-lora-catalog.py SOURCE_CATALOG DESTINATION` on changes. The installed helper is `/usr/local/lib/xaigen-dev/export-lora-catalog.py`.
The destination is `studio2/shared-lora-catalog.cache` inside the existing dev library volume. It contains **only** `loraAliases` and `loraTriggers`. No production catalog, media, passwords, or tokens are mounted into dev. The exporter writes atomically and skips unchanged content.
Both Studio 2 library and LoRA endpoints read this file on each request, normalize identity keys, and prefer production values over local duplicates. Unmapped dev entries remain available. If the export is unavailable, local settings remain usable. `STUDIO2_LORA_CATALOG_PATH` optionally overrides the export path.
The existing library poll refreshes labels and triggers without waking Comfy or restarting a generation. A one-time settings API import seeds local fallback labels and triggers. Main and Studio 1 code are unchanged.
+100
View File
@@ -0,0 +1,100 @@
# YuE2 music engine
`engine: "yue2"` is the only music engine in the app. Standalone [YuE2](https://github.com/multimodal-art-projection/YuE) (`m-a-p/YuE2-3B` + `m-a-p/YuE2-Vae`). It never submits a Comfy graph. ACE / YuEGP are removed from the picker and generate API; leave ACE Comfy checkpoints on disk.
## Desktop agent at boot (one click)
Wrong agent on 8199 → `/yue2/status` returns `{"ok":false,"error":"not found"}` while Comfy images still work.
Fix once from this clone:
```powershell
.\scripts\install-comfy-host-agent-launchers.ps1
```
That rewrites Desktop, Start Menu, and **Windows Startup** shortcuts to this clone, and retargets the `AIGen Comfy Host Agent` scheduled task when it exists. Each launch **kills** any old `comfy-host-agent` / listener on 8199, then starts `scripts/comfy-host-agent.mjs` (imports `yue2-host.mjs`).
After install: double-click Desktop **Start Comfy Host Agent**, or reboot. Confirm:
```text
GET http://127.0.0.1:8199/yue2/status → 200 { configured, busy, backend: "yue2" }
```
Do not guess which `node.exe` to kill — the launcher replaces it.
## What this host already has (5080 desktop)
Ran on this machine:
1. `scripts/setup-yue2.ps1` → checkout + venv + CUDA torch + weight download
2. `scripts/cleanup-yue-v1.ps1` → named YuE v1 / YuEGP weight folders removed
3. Host agent mounts `POST/GET /yue2/jobs`
Paths written:
| Item | Path |
| --- | --- |
| `YUE2_ROOT` | `C:\Users\ianjm\Development\YuE2` |
| Python | `C:\Users\ianjm\Development\YuE2\.venv\Scripts\python.exe` |
| Model | `C:\Users\ianjm\Development\YuE2\models\YuE2-3B` (~7.3 GB) |
| VAE | `C:\Users\ianjm\Development\YuE2\models\YuE2-Vae` (~531 MB) |
| Ready file | `C:\Users\ianjm\Development\YuE2\aigen-ready.json` |
| Jobs dir default | `C:\Users\ianjm\Development\YuE2\aigen-jobs` |
`aigen-ready.json` points `model` / `vae` at those local folders so Generate does not need a first-click HF pull. CUDA check: `torch 2.11.0+cu128` on `NVIDIA GeForce RTX 5080`.
After deploying this branch, run the installer above (or the Desktop shortcut) once so 8199 serves YuE2. Do not restart Coolify from the agent; live apps that track `main` rebuild on push.
## Env vars
| Variable | Purpose |
| --- | --- |
| `YUE2_ROOT` | Standalone YuE checkout |
| `YUE2_PYTHON` | Isolated worker Python |
| `YUE2_JOBS_DIR` | Per-job logs / request / WAV |
| `YUE2_MODEL` | Local model dir or HF id |
| `YUE2_VAE` | Local VAE dir or HF id |
App uses existing `COMFY_CONTROL_URL` + token. One GPU owner via the shared reservation (Comfy stopped while YuE2 runs). One song. Unload after decode. OOM is visible; no ACE/YuEGP fallback.
## Call path
```json
{"engine":"yue2","duration":60,"seed":42,"tags":"pop, warm vocals","lyrics":"[Verse 1]\nYour lyrics here","folderId":"<library-folder>","name":"My song"}
```
Picker → `POST /api/generate/music` → `startYue2Job` → `POST <COMFY_CONTROL_URL>/yue2/jobs` → `yue2-worker.py` (`plan` → `generate_semantic` → `synthesize` → `decode`) → library WAV.
Duration 30–150 s (default 60). One non-empty lyric section (same validation as Yue).
## Old YuE / YuEGP data deleted on this host
`scripts/cleanup-yue-v1.ps1` logged deletions to `scripts/cleanup-yue-v1.log`. Deleted:
- `...\ComfyUI-Shared\models\yue\YuE-s1-7B-anneal-en-cot`
- `...\ComfyUI-Shared\models\yue\YuE-s2-1B-general`
- `...\ComfyUI-Shared\models\yue\xcodec_mini_infer`
- `...\ComfyUI-Shared\models\yue\ckpt_00360000.pth`
- `...\ComfyUI-Shared\models\yue\decoder_131000.pth`
- `...\ComfyUI-Shared\models\yue\decoder_151000.pth`
- `C:\Users\ianjm\Development\YuEGP\inference\xcodec_mini_infer`
- `C:\Users\ianjm\Development\YuEGP\aigen-jobs`
HF cache YuE-s1 / YuE-s2 / xcodec entries were already absent (skipped). ACE checkpoints (`ace_step_v1_3.5b.safetensors`, `ace_step_1.5_turbo_aio.safetensors`) and other Comfy model trees were left alone on disk.
## Re-run setup (only if rebuilding the machine)
```powershell
.\scripts\setup-yue2.ps1
.\scripts\cleanup-yue-v1.ps1
.\scripts\install-comfy-host-agent-launchers.ps1
```
Setup reuses existing `models\YuE2-3B` / `YuE2-Vae` when present, then force-reinstalls CUDA torch after the editable package install.
## Validation
- JS: `node --test tests/yue2.test.mjs`
- Python: `py -3 -m unittest tests.test_yue2_worker`
- After agent restart: `GET /yue2/status` returns 200.
- Do not claim a song generated unless one ran after the agent restart.
+1 -1
View File
@@ -4,7 +4,7 @@ const xaigenStudio = /^xaigen$/i.test(String(process.env.NUXT_PUBLIC_STUDIO || '
export default defineNuxtConfig({ export default defineNuxtConfig({
compatibilityDate: '2025-08-01', compatibilityDate: '2025-08-01',
modules: ['@nuxtjs/tailwindcss'], modules: ['@nuxtjs/tailwindcss'],
css: ['~/assets/css/main.css'], css: ['~/assets/css/main.css', '~/assets/css/display.css'],
app: { app: {
head: { head: {
title: 'AIGen — MiniMax H3', title: 'AIGen — MiniMax H3',
+1 -8190
View File
File diff suppressed because it is too large Load Diff
+24 -164
View File
@@ -1,7 +1,7 @@
<template> <template>
<div class="min-h-dvh min-w-0"> <div class="min-h-dvh min-w-0">
<AppHeader <AppHeader
:subtitle="advanced ? headerSubtitle : 'Create music from text'" :subtitle="advanced ? 'YuE2 · lyrics-to-song' : 'Create music from text'"
current="music" current="music"
:queue-count="queueCount" :queue-count="queueCount"
/> />
@@ -10,9 +10,8 @@
<StudioKindCards <StudioKindCards
:advanced="advanced" :advanced="advanced"
current="music" current="music"
:music-engine="engineFamily" music-engine="yue2"
@select="goStudio" @select="goStudio"
@music-engine="selectEngineFamily"
/> />
<div class="flex items-center justify-between gap-3"><span class="rounded-full border border-amber-300/60 px-4 py-2 text-sm text-amber-100">From text</span><label class="flex items-center gap-2 text-sm text-zinc-300"><input v-model="advanced" type="checkbox" class="accent-amber-400">Advanced mode</label></div> <div class="flex items-center justify-between gap-3"><span class="rounded-full border border-amber-300/60 px-4 py-2 text-sm text-amber-100">From text</span><label class="flex items-center gap-2 text-sm text-zinc-300"><input v-model="advanced" type="checkbox" class="accent-amber-400">Advanced mode</label></div>
<div class="grid min-w-0 gap-6 lg:grid-cols-[minmax(0,1fr)_minmax(0,1fr)] lg:items-stretch"> <div class="grid min-w-0 gap-6 lg:grid-cols-[minmax(0,1fr)_minmax(0,1fr)] lg:items-stretch">
@@ -20,42 +19,15 @@
<div> <div>
<h2 class="font-display text-2xl font-bold">Music Generator</h2> <h2 class="font-display text-2xl font-bold">Music Generator</h2>
<p class="mt-1 text-sm text-zinc-400"> <p class="mt-1 text-sm text-zinc-400">
Tags describe style and voice. Lyrics use structure chips. Instrumental skips the vocal line. Tags describe style and voice. Lyrics use structure chips. Multiple sections are OK.
</p> </p>
</div> </div>
<GenerationSettings :advanced="advanced"> <GenerationSettings :advanced="advanced">
<label class="block space-y-2 text-sm"><span>Model</span><select :value="engineFamily" class="w-full rounded-xl border border-white/15 bg-zinc-900 p-3" @change="selectEngineFamily(($event.target as HTMLSelectElement).value as 'ace' | 'yue')"><option value="ace">ACE-Step</option><option value="yue">YuE</option></select></label> <p class="text-xs text-zinc-400">YuE2-3B · lyrics with section tags · plan → synthesize → decode · no score editor</p>
<label v-if="engineFamily === 'yue'" class="block space-y-2 text-sm">
<span>YuEGP profile</span>
<select v-model.number="yueProfile" class="w-full rounded-xl border border-white/15 bg-zinc-900 p-3">
<option :value="1">1 · Full quality · 16GB (default)</option>
<option :value="3">3 · Lower memory · Manual fallback</option>
</select>
<span class="block text-xs text-zinc-400">One lyric section. Compile off. Profile 3 is never selected automatically.</span>
</label>
<div v-if="!advanced" class="space-y-4"> <div v-if="!advanced" class="space-y-4">
<label v-if="engineFamily !== 'yue'" class="block text-sm">Steps<input v-model.number="steps" type="number" :min="stepsMin" :max="stepsMax" class="mt-2 w-full rounded-xl border border-white/15 bg-zinc-900 p-3"></label>
<label class="block text-sm">Seed<input v-model="seed" placeholder="random" class="mt-2 w-full rounded-xl border border-white/15 bg-zinc-900 p-3"></label> <label class="block text-sm">Seed<input v-model="seed" placeholder="random" class="mt-2 w-full rounded-xl border border-white/15 bg-zinc-900 p-3"></label>
</div> </div>
<div v-if="engineFamily === 'ace'" class="flex items-center justify-between gap-3 rounded-2xl border border-white/10 bg-zinc-950/40 px-4 py-3 text-sm">
<span class="min-w-0">
<span class="flex items-center gap-2">
<span class="block font-medium text-zinc-200">ACE-Step 1.5</span>
<InfoTip
aria-label="What ACE-Step 1.5 does"
:text="ace15Info"
/>
</span>
<span class="mt-0.5 block text-xs text-zinc-500">Turbo AIO checkpoint. Different graph than v1, usually 8 steps.</span>
</span>
<label class="relative inline-flex h-6 w-11 shrink-0 cursor-pointer items-center" title="Toggle ACE-Step 1.5">
<input v-model="ace15" type="checkbox" class="peer sr-only" role="switch">
<span class="absolute inset-0 rounded-full bg-zinc-700 transition peer-checked:bg-amber-400" />
<span class="absolute left-0.5 top-0.5 h-5 w-5 rounded-full bg-white transition peer-checked:translate-x-5" />
</label>
</div>
</GenerationSettings> </GenerationSettings>
<p v-if="error" class="rounded-2xl border border-red-400/30 bg-red-400/10 px-4 py-3 text-sm text-red-200">{{ error }}</p> <p v-if="error" class="rounded-2xl border border-red-400/30 bg-red-400/10 px-4 py-3 text-sm text-red-200">{{ error }}</p>
@@ -72,18 +44,13 @@
<div> <div>
<div class="mb-1 flex flex-wrap items-center justify-between gap-2"> <div class="mb-1 flex flex-wrap items-center justify-between gap-2">
<span class="text-sm font-medium text-zinc-300">Lyrics</span> <span class="text-sm font-medium text-zinc-300">Lyrics</span>
<label class="flex cursor-pointer items-center gap-2 text-xs text-zinc-400">
<input v-model="instrumental" :disabled="engineFamily === 'yue'" type="checkbox" class="accent-amber-400">
Instrumental
</label>
</div> </div>
<div class="mb-2 flex flex-wrap gap-1.5"> <div class="mb-2 flex flex-wrap gap-1.5">
<button <button
v-for="chip in chips" v-for="chip in chips"
:key="chip" :key="chip"
type="button" type="button"
class="rounded-full border border-white/10 px-2.5 py-1 text-[11px] text-zinc-300 hover:border-amber-300/50 hover:text-white disabled:opacity-40" class="rounded-full border border-white/10 px-2.5 py-1 text-[11px] text-zinc-300 hover:border-amber-300/50 hover:text-white"
:disabled="instrumental"
@click="insertChip(chip)" @click="insertChip(chip)"
>{{ chip }}</button> >{{ chip }}</button>
</div> </div>
@@ -91,8 +58,7 @@
ref="lyricsEl" ref="lyricsEl"
v-model="lyrics" v-model="lyrics"
rows="12" rows="12"
class="max-h-[min(28rem,50vh)] w-full overflow-y-auto rounded-2xl border border-white/10 bg-zinc-950/70 px-4 py-3 text-sm outline-none ring-amber-300/40 placeholder:text-zinc-600 [overflow-wrap:anywhere] focus:ring-2 disabled:opacity-50" class="max-h-[min(28rem,50vh)] w-full overflow-y-auto rounded-2xl border border-white/10 bg-zinc-950/70 px-4 py-3 text-sm outline-none ring-amber-300/40 placeholder:text-zinc-600 [overflow-wrap:anywhere] focus:ring-2"
:disabled="instrumental"
:placeholder="lyricsPlaceholder" :placeholder="lyricsPlaceholder"
/> />
</div> </div>
@@ -108,19 +74,7 @@
:max="durationMax" :max="durationMax"
step="5" step="5"
> >
<span class="mt-1 block text-[11px] text-zinc-500">{{ engineFamily === 'yue' ? 'Approximate length · one lyric section' : `${durationMin}–${durationMax} seconds` }}</span> <span class="mt-1 block text-[11px] text-zinc-500">Approximate length · multiple sections OK</span>
</label>
<label v-if="advanced && engineFamily !== 'yue'" class="block text-sm">
<span class="mb-1 block font-medium text-zinc-300">Steps · {{ steps }}</span>
<input
v-model.number="steps"
type="range"
class="aigen-slider w-full"
:min="stepsMin"
:max="stepsMax"
step="1"
>
<span class="mt-1 block text-[11px] text-zinc-500">{{ ace15 ? '8 is the usual 1.5 turbo range' : '30–50 is the usual range' }}</span>
</label> </label>
</div> </div>
@@ -208,7 +162,7 @@
</div> </div>
<p v-if="busy && !queued && jobId" class="text-xs text-zinc-400"> <p v-if="busy && !queued && jobId" class="text-xs text-zinc-400">
{{ musicElapsed }} elapsed · {{ musicActivityText }} {{ musicElapsed }} elapsed · {{ musicActivityText }}
<span v-if="activeMusicEngine === 'yue' && !indeterminateMusic" class="mt-1 block">Progress is for the current stage.</span> <span v-if="!indeterminateMusic" class="mt-1 block">Progress is for the current stage.</span>
<span v-if="indeterminateMusic" class="mt-1 block">Waiting for a progress report.</span> <span v-if="indeterminateMusic" class="mt-1 block">Waiting for a progress report.</span>
</p> </p>
<AudioPlayer <AudioPlayer
@@ -235,18 +189,10 @@ const { advanced } = useStudioLayout()
import { import {
DEFAULT_MUSIC_LYRICS, DEFAULT_MUSIC_LYRICS,
DEFAULT_MUSIC_TAGS, DEFAULT_MUSIC_TAGS,
MUSIC_CFG_DEFAULT,
MUSIC_CFG_DEFAULT_15,
MUSIC_DURATION_DEFAULT, MUSIC_DURATION_DEFAULT,
MUSIC_DURATION_MAX, MUSIC_DURATION_MAX,
MUSIC_DURATION_MIN, MUSIC_DURATION_MIN,
MUSIC_STEPS_DEFAULT,
MUSIC_STEPS_DEFAULT_15,
MUSIC_STEPS_MAX,
MUSIC_STEPS_MIN,
MUSIC_STRUCTURE_CHIPS, MUSIC_STRUCTURE_CHIPS,
parseMusicEngine,
type MusicEngine,
yueLyricsProblem, yueLyricsProblem,
} from '~/utils/music' } from '~/utils/music'
import { ensureGenerationNotifyPermission, notifyGenerationComplete } from '~/utils/generationNotify' import { ensureGenerationNotifyPermission, notifyGenerationComplete } from '~/utils/generationNotify'
@@ -258,25 +204,16 @@ const tagsPlaceholder = DEFAULT_MUSIC_TAGS
const lyricsPlaceholder = DEFAULT_MUSIC_LYRICS const lyricsPlaceholder = DEFAULT_MUSIC_LYRICS
const chips = MUSIC_STRUCTURE_CHIPS const chips = MUSIC_STRUCTURE_CHIPS
const durationMin = MUSIC_DURATION_MIN const durationMin = MUSIC_DURATION_MIN
const durationMax = computed(() => engineFamily.value === 'yue' ? 150 : MUSIC_DURATION_MAX) const durationMax = MUSIC_DURATION_MAX
const stepsMin = MUSIC_STEPS_MIN
const stepsMax = MUSIC_STEPS_MAX
const ace15Info = 'Turns on ACE-Step 1.5 Turbo, the all-in-one checkpoint already on Beast. 1.5 uses a different Comfy graph than v1: a 1.5 text encoder (tags, BPM, key, duration) and EmptyAceStep1.5LatentAudio, usually with 8 sampler steps and CFG 1. Songs often come back faster and cleaner. Lyrics-strength from v1 does not apply. Leave this off to stay on ACE-Step v1 3.5B, which is the graph that already generated on this GPU.'
const tags = ref(DEFAULT_MUSIC_TAGS) const tags = ref(DEFAULT_MUSIC_TAGS)
const lyrics = ref(DEFAULT_MUSIC_LYRICS) const lyrics = ref(DEFAULT_MUSIC_LYRICS)
const instrumental = ref(false)
const duration = ref(MUSIC_DURATION_DEFAULT) const duration = ref(MUSIC_DURATION_DEFAULT)
const steps = ref(MUSIC_STEPS_DEFAULT)
const seed = ref('') const seed = ref('')
const name = ref('') const name = ref('')
const folderId = ref('') const folderId = ref('')
const folders = ref<LibraryFolder[]>([]) const folders = ref<LibraryFolder[]>([])
const lyricsEl = ref<HTMLTextAreaElement | null>(null) const lyricsEl = ref<HTMLTextAreaElement | null>(null)
const ace15 = ref(false)
const engineFamily = ref<'ace' | 'yue'>('ace')
const yueProfile = ref<1 | 3>(1)
const busy = ref(false) const busy = ref(false)
const forceClearing = ref(false) const forceClearing = ref(false)
const error = ref('') const error = ref('')
@@ -284,10 +221,9 @@ const status = ref('')
const progress = ref(0) const progress = ref(0)
const musicClock = ref(Date.now()) const musicClock = ref(Date.now())
const musicStartedAt = ref(0) const musicStartedAt = ref(0)
const activeMusicEngine = ref('')
const musicActivity = ref<{ checkedAt: number; running: boolean } | null>(null) const musicActivity = ref<{ checkedAt: number; running: boolean } | null>(null)
let musicClockTimer: ReturnType<typeof setInterval> | null = null let musicClockTimer: ReturnType<typeof setInterval> | null = null
const indeterminateMusic = computed(() => !queued.value && activeMusicEngine.value === 'yue' && progress.value < 98 && !musicActivity.value?.running) const indeterminateMusic = computed(() => !queued.value && progress.value < 98 && !musicActivity.value?.running)
const recentMusicCheck = computed(() => Boolean(musicActivity.value?.running && musicClock.value - musicActivity.value.checkedAt < 15000)) const recentMusicCheck = computed(() => Boolean(musicActivity.value?.running && musicClock.value - musicActivity.value.checkedAt < 15000))
const musicElapsed = computed(() => { const musicElapsed = computed(() => {
const seconds = musicStartedAt.value ? Math.max(0, Math.floor((musicClock.value - musicStartedAt.value) / 1000)) : 0 const seconds = musicStartedAt.value ? Math.max(0, Math.floor((musicClock.value - musicStartedAt.value) / 1000)) : 0
@@ -317,56 +253,19 @@ let queuePoll: ReturnType<typeof setInterval> | null = null
let recoverPoll: ReturnType<typeof setInterval> | null = null let recoverPoll: ReturnType<typeof setInterval> | null = null
const blocked = computed(() => Boolean(blockReason.value)) const blocked = computed(() => Boolean(blockReason.value))
const headerSubtitle = computed(() => {
if (engineFamily.value === 'yue') return 'YuE · lyrics-to-song'
return ace15.value ? 'ACE-Step 1.5 · vocals' : 'ACE-Step · vocals'
})
const selectedEngine = computed<MusicEngine>(() => {
if (engineFamily.value === 'yue') return 'yue'
return ace15.value ? 'ace-step-1.5' : 'ace-step'
})
const blockReason = computed(() => { const blockReason = computed(() => {
if (!folderId.value) return 'Choose a library folder before generating.' if (!folderId.value) return 'Choose a library folder before generating.'
if (engineFamily.value === 'yue' && instrumental.value) return 'YuEGP requires lyrics. Use ACE for instrumental music.' if (duration.value > 150) return 'YuE2 supports up to 150 seconds.'
if (engineFamily.value === 'yue' && duration.value > 150) return 'YuEGP supports up to 150 seconds per section.'
if (!tags.value.trim()) return 'Add genre and style tags.' if (!tags.value.trim()) return 'Add genre and style tags.'
if (engineFamily.value === 'yue') { const problem = yueLyricsProblem(lyrics.value); if (problem) return problem } const problem = yueLyricsProblem(lyrics.value)
if (!instrumental.value && !lyrics.value.trim()) return 'Write lyrics, or turn on Instrumental.' if (problem) return problem
return '' return ''
}) })
function selectEngineFamily(family: 'ace' | 'yue') {
engineFamily.value = family
if (family === 'yue') { instrumental.value = false; duration.value = Math.min(150, duration.value) }
if (family === 'yue' && lyrics.value === DEFAULT_MUSIC_LYRICS) lyrics.value = '[Verse 1]\n'
if (family === 'ace' && ace15.value && steps.value === MUSIC_STEPS_DEFAULT) {
steps.value = MUSIC_STEPS_DEFAULT_15
}
if (family === 'ace' && !ace15.value && steps.value === MUSIC_STEPS_DEFAULT_15) {
steps.value = MUSIC_STEPS_DEFAULT
}
}
function goStudio(kind: 'video' | 'image') { function goStudio(kind: 'video' | 'image') {
void navigateTo({ path: '/', query: { studio: kind } }) void navigateTo({ path: '/', query: { studio: kind } })
} }
function applyEngine(engine: MusicEngine) {
if (engine === 'yue') {
engineFamily.value = 'yue'
ace15.value = false
return
}
engineFamily.value = 'ace'
ace15.value = engine === 'ace-step-1.5'
}
watch(ace15, (on) => {
if (engineFamily.value !== 'ace') return
if (on && (steps.value === MUSIC_STEPS_DEFAULT || steps.value >= 30)) steps.value = MUSIC_STEPS_DEFAULT_15
if (!on && steps.value === MUSIC_STEPS_DEFAULT_15) steps.value = MUSIC_STEPS_DEFAULT
})
function insertChip(chip: string) { function insertChip(chip: string) {
const el = lyricsEl.value const el = lyricsEl.value
const token = `${chip}\n` const token = `${chip}\n`
@@ -452,9 +351,7 @@ async function attachLiveMusic(opts: {
if (typeof opts.progress === 'number') progress.value = opts.progress if (typeof opts.progress === 'number') progress.value = opts.progress
if (opts.message) status.value = opts.message if (opts.message) status.value = opts.message
else if (!status.value || /queueing|waiting for a generate|waiting in the job queue|waiting for gpu/i.test(status.value)) { else if (!status.value || /queueing|waiting for a generate|waiting in the job queue|waiting for gpu/i.test(status.value)) {
status.value = selectedEngine.value === 'yue' status.value = 'YuE2 is generating your song'
? 'YuEGP is generating your song'
: 'Generating…'
} }
if (jobId.value !== liveId) { if (jobId.value !== liveId) {
musicActivity.value = null musicActivity.value = null
@@ -497,9 +394,7 @@ async function refreshStudioQueue() {
busy.value = true busy.value = true
status.value = musicRow.status === 'held' status.value = musicRow.status === 'held'
? 'Paused in the job queue' ? 'Paused in the job queue'
: (selectedEngine.value === 'yue' : 'Waiting for the GPU to run YuE2'
? 'Waiting for the GPU to run YuEGP'
: 'Waiting in the job queue…')
} }
if (!studioJobId.value) return if (!studioJobId.value) return
@@ -509,9 +404,7 @@ async function refreshStudioQueue() {
queued.value = false queued.value = false
} }
if (jobId.value && /waiting in the job queue/i.test(status.value)) { if (jobId.value && /waiting in the job queue/i.test(status.value)) {
status.value = selectedEngine.value === 'yue' status.value = 'YuE2 is generating your song'
? 'YuEGP is generating your song'
: 'Generating…'
} }
return return
} }
@@ -520,9 +413,7 @@ async function refreshStudioQueue() {
busy.value = true busy.value = true
status.value = row.status === 'held' status.value = row.status === 'held'
? 'Paused in the job queue' ? 'Paused in the job queue'
: (selectedEngine.value === 'yue' : 'Waiting for the GPU to run YuE2'
? 'Waiting for the GPU to run YuEGP'
: 'Waiting in the job queue…')
return return
} }
if (row.status === 'running' && row.liveJobId) { if (row.status === 'running' && row.liveJobId) {
@@ -557,7 +448,6 @@ async function resumeActiveMusic() {
function applyEvent(payload: Record<string, any>) { function applyEvent(payload: Record<string, any>) {
if (typeof payload.elapsedMs === 'number') musicStartedAt.value = Date.now() - payload.elapsedMs if (typeof payload.elapsedMs === 'number') musicStartedAt.value = Date.now() - payload.elapsedMs
if (payload.engine) activeMusicEngine.value = payload.engine
if (payload.musicActivity) musicActivity.value = payload.musicActivity if (payload.musicActivity) musicActivity.value = payload.musicActivity
if (payload.message) status.value = payload.message if (payload.message) status.value = payload.message
if (typeof payload.progress === 'number') progress.value = payload.progress if (typeof payload.progress === 'number') progress.value = payload.progress
@@ -665,11 +555,7 @@ async function generate() {
trackId.value = '' trackId.value = ''
jobId.value = '' jobId.value = ''
studioJobId.value = '' studioJobId.value = ''
status.value = engineFamily.value === 'yue' status.value = 'Queueing YuE2…'
? 'Queueing YuE…'
: ace15.value
? 'Queueing ACE-Step 1.5…'
: 'Queueing ACE-Step…'
try { try {
const started = await $fetch<{ jobId: string; studioJobId?: string; queued?: boolean; name?: string }>( const started = await $fetch<{ jobId: string; studioJobId?: string; queued?: boolean; name?: string }>(
'/api/generate/music', '/api/generate/music',
@@ -679,14 +565,10 @@ async function generate() {
tags: tags.value, tags: tags.value,
lyrics: lyrics.value, lyrics: lyrics.value,
duration: duration.value, duration: duration.value,
steps: steps.value,
seed: seed.value || 'random', seed: seed.value || 'random',
cfg: selectedEngine.value === 'ace-step-1.5' ? MUSIC_CFG_DEFAULT_15 : MUSIC_CFG_DEFAULT,
instrumental: instrumental.value,
folderId: folderId.value, folderId: folderId.value,
name: name.value.trim(), name: name.value.trim(),
engine: selectedEngine.value, engine: 'yue2'
...(engineFamily.value === 'yue' ? { yueProfile: yueProfile.value } : {})
} }
} }
) )
@@ -712,10 +594,6 @@ async function generate() {
} }
} }
watch(instrumental, (on) => {
if (on && !lyrics.value.trim()) lyrics.value = DEFAULT_MUSIC_LYRICS
})
function applyIncomingMusic() { function applyIncomingMusic() {
try { try {
const genRaw = sessionStorage.getItem(LOAD_GENERATION_PAYLOAD_KEY) const genRaw = sessionStorage.getItem(LOAD_GENERATION_PAYLOAD_KEY)
@@ -726,14 +604,10 @@ function applyIncomingMusic() {
const payload = loaded.payload const payload = loaded.payload
if (typeof payload.prompt === 'string') tags.value = payload.prompt if (typeof payload.prompt === 'string') tags.value = payload.prompt
if (typeof payload.lyrics === 'string') lyrics.value = payload.lyrics if (typeof payload.lyrics === 'string') lyrics.value = payload.lyrics
instrumental.value = payload.instrumental === true if (typeof payload.duration === 'number') duration.value = Math.min(MUSIC_DURATION_MAX, payload.duration)
if (typeof payload.duration === 'number') duration.value = payload.duration
if (typeof payload.steps === 'number') steps.value = payload.steps
if (payload.seed != null) seed.value = String(payload.seed) if (payload.seed != null) seed.value = String(payload.seed)
if (typeof payload.name === 'string') name.value = payload.name if (typeof payload.name === 'string') name.value = payload.name
if (payload.folderId && folders.value.some(folder => folder.id === payload.folderId)) folderId.value = payload.folderId if (payload.folderId && folders.value.some(folder => folder.id === payload.folderId)) folderId.value = payload.folderId
if (payload.yueProfile === 1 || payload.yueProfile === 3) yueProfile.value = payload.yueProfile
if (payload.musicEngine || payload.engine) applyEngine(parseMusicEngine(payload.musicEngine || payload.engine))
status.value = 'Loaded from generation log' status.value = 'Loaded from generation log'
return return
} }
@@ -748,24 +622,16 @@ function applyIncomingMusic() {
name?: string name?: string
tags?: string tags?: string
lyrics?: string lyrics?: string
instrumental?: boolean
duration?: number duration?: number
steps?: number
seed?: string seed?: string
folderId?: string folderId?: string
engine?: string
yueProfile?: 1 | 3
} }
if (typeof loaded.tags === 'string') tags.value = loaded.tags if (typeof loaded.tags === 'string') tags.value = loaded.tags
if (typeof loaded.lyrics === 'string') lyrics.value = loaded.lyrics if (typeof loaded.lyrics === 'string') lyrics.value = loaded.lyrics
instrumental.value = loaded.instrumental === true if (typeof loaded.duration === 'number') duration.value = Math.min(MUSIC_DURATION_MAX, loaded.duration)
if (typeof loaded.duration === 'number') duration.value = loaded.duration
if (typeof loaded.steps === 'number') steps.value = loaded.steps
if (typeof loaded.seed === 'string') seed.value = loaded.seed if (typeof loaded.seed === 'string') seed.value = loaded.seed
if (typeof loaded.name === 'string') name.value = loaded.name if (typeof loaded.name === 'string') name.value = loaded.name
if (loaded.folderId && folders.value.some(folder => folder.id === loaded.folderId)) folderId.value = loaded.folderId if (loaded.folderId && folders.value.some(folder => folder.id === loaded.folderId)) folderId.value = loaded.folderId
if (loaded.engine) applyEngine(parseMusicEngine(loaded.engine))
yueProfile.value = loaded.yueProfile === 3 ? 3 : 1
status.value = 'Loaded from the queue' status.value = 'Loaded from the queue'
} catch { /* ignore */ } } catch { /* ignore */ }
} }
@@ -781,12 +647,8 @@ function applyPreset() {
const settings = preset.settings as MusicPresetSettings const settings = preset.settings as MusicPresetSettings
if (typeof settings.tags === 'string') tags.value = settings.tags if (typeof settings.tags === 'string') tags.value = settings.tags
if (typeof settings.lyrics === 'string') lyrics.value = settings.lyrics if (typeof settings.lyrics === 'string') lyrics.value = settings.lyrics
instrumental.value = settings.instrumental === true if (typeof settings.duration === 'number') duration.value = Math.min(MUSIC_DURATION_MAX, settings.duration)
if (typeof settings.duration === 'number') duration.value = settings.duration
if (typeof settings.steps === 'number') steps.value = settings.steps
if (typeof settings.seed === 'number') seed.value = String(settings.seed) if (typeof settings.seed === 'number') seed.value = String(settings.seed)
if (settings.engine) applyEngine(settings.engine)
yueProfile.value = settings.yueProfile === 3 ? 3 : 1
name.value = preset.name name.value = preset.name
} }
@@ -794,14 +656,12 @@ onMounted(async () => {
musicClockTimer = setInterval(() => { musicClock.value = Date.now() }, 1000) musicClockTimer = setInterval(() => { musicClock.value = Date.now() }, 1000)
await loadLibrary() await loadLibrary()
await loadMusicPresets() await loadMusicPresets()
const route = useRoute()
const engineQ = String(route.query.engine || '').toLowerCase()
if (engineQ === 'yue' || engineQ === 'ace') selectEngineFamily(engineQ)
applyIncomingMusic() applyIncomingMusic()
await resumeActiveMusic() await resumeActiveMusic()
await refreshStudioQueue() await refreshStudioQueue()
stopQueuePoll() stopQueuePoll()
queuePoll = setInterval(() => { void refreshStudioQueue() }, 8000) queuePoll = setInterval(() => { void refreshStudioQueue() }, 8000)
const route = useRoute()
const query = route.query const query = route.query
if (typeof query.track === 'string' && query.track) { if (typeof query.track === 'string' && query.track) {
audioUrl.value = `/api/library/tracks/${query.track}/audio` audioUrl.value = `/api/library/tracks/${query.track}/audio`
@@ -815,7 +675,7 @@ onMounted(async () => {
if (recovered?.trackId && /recovered/i.test(String(recovered.message || ''))) { if (recovered?.trackId && /recovered/i.test(String(recovered.message || ''))) {
applyEvent({ ...recovered, type: 'complete' }) applyEvent({ ...recovered, type: 'complete' })
} }
} catch { /* nothing waiting in ComfyUI */ } } catch { /* nothing waiting */ }
} }
}) })
+2 -2
View File
@@ -663,7 +663,7 @@ function progressLine(queue: Queue) {
function jobLine(job: StudioJobRow) { function jobLine(job: StudioJobRow) {
if (job.kind === 'music') { if (job.kind === 'music') {
if (job.status === 'waiting') return job.cutIn ? 'Next after the current shot · ACE-Step' : 'Waiting · ACE-Step' if (job.status === 'waiting') return job.cutIn ? 'Next after the current shot · YuE2' : 'Waiting · YuE2'
if (job.status === 'held') return 'Paused' if (job.status === 'held') return 'Paused'
if (job.status === 'error') return job.lastError || 'Failed' if (job.status === 'error') return job.lastError || 'Failed'
return 'Generating track' return 'Generating track'
@@ -1237,7 +1237,7 @@ async function clearAll() {
async function forceClearAll() { async function forceClearAll() {
if (forceClearing.value) return if (forceClearing.value) return
if (!confirm( if (!confirm(
'Force clear cancels every running and waiting job (including stuck YuE/music), wipes shot batches, and kills Comfy if it is wedged. Saved library clips stay. Continue?' 'Force clear cancels every running and waiting job (including stuck YuE2/music), wipes shot batches, and kills Comfy if it is wedged. Saved library clips stay. Continue?'
)) return )) return
forceClearing.value = true forceClearing.value = true
error.value = '' error.value = ''
+8192
View File
File diff suppressed because it is too large Load Diff
+2 -3
View File
@@ -1,7 +1,6 @@
<template> <template>
<div v-if="config.public.studio === 'xaigen'" class="border-b border-amber-400/20 bg-zinc-950 px-6 py-3 text-sm text-amber-100">xAIGen 2.0 development preview · Image and MiniMax tools are available here. <NuxtLink to="/" class="underline">Open the existing studio for LTX and music</NuxtLink>.</div> <div />
<Studio2Bench />
</template> </template>
<script setup lang="ts"> <script setup lang="ts">
const config = useRuntimeConfig() await navigateTo('/', { replace: true })
</script> </script>
+331
View File
@@ -0,0 +1,331 @@
import { spawn } from 'node:child_process'
import { createServer } from 'node:net'
import { existsSync, mkdirSync, readFileSync, writeFileSync, renameSync, appendFileSync, readdirSync, copyFileSync } from 'node:fs'
import { join, resolve, extname } from 'node:path'
import { captionStylePrompt, CAPTION_STYLES } from '../shared/studio2/caption.mjs'
const MODEL_NAME = 'Qwen2.5-VL-7B-NSFW-Caption-V4.Q5_K_M.gguf'
const MMPROJ_NAME = 'Qwen2.5-VL-7B-NSFW-Caption-V4.mmproj-f16.gguf'
function defaultModelsDir() {
const shared = process.env.COMFY_MODELS_ROOT
|| join(process.env.LOCALAPPDATA || '', 'Comfy-Desktop', 'ComfyUI-Shared', 'models')
return resolve(process.env.CAPTION_MODELS || join(shared, 'caption', 'qwen25vl-7b-nsfw-v4'))
}
function defaultLlamaServer() {
if (process.env.CAPTION_LLAMA_SERVER) return resolve(process.env.CAPTION_LLAMA_SERVER)
return 'llama-server'
}
function freePort() {
return new Promise((resolvePort, reject) => {
const server = createServer()
server.listen(0, '127.0.0.1', () => {
const address = server.address()
const port = typeof address === 'object' && address ? address.port : 0
server.close(error => error ? reject(error) : resolvePort(port))
})
server.on('error', reject)
})
}
async function waitForServer(port, signal, timeoutMs = 120_000, fetchImpl = fetch) {
const started = Date.now()
while (Date.now() - started < timeoutMs) {
if (signal?.aborted) throw new Error('Caption cancelled while loading the model.')
try {
const response = await fetchImpl(`http://127.0.0.1:${port}/health`, { signal: AbortSignal.timeout(2000) })
if (response.ok) return
} catch { /* still booting */ }
await new Promise(r => setTimeout(r, 500))
}
throw new Error('llama-server did not become ready in time.')
}
function mimeFor(path) {
const ext = extname(path).toLowerCase()
if (ext === '.jpg' || ext === '.jpeg') return 'image/jpeg'
if (ext === '.webp') return 'image/webp'
if (ext === '.gif') return 'image/gif'
return 'image/png'
}
function killTree(child) {
return new Promise(resolveKill => {
if (!child?.pid) return resolveKill()
let finished = false
const done = () => {
if (finished) return
finished = true
resolveKill()
}
child.once('close', done)
try {
if (typeof child.kill === 'function') child.kill()
} catch { /* ignore */ }
if (process.platform === 'win32' && child.pid > 0) {
try {
const killer = spawn('taskkill', ['/PID', String(child.pid), '/T', '/F'], { windowsHide: true, stdio: 'ignore' })
killer.once('close', done)
killer.once('error', done)
} catch {
done()
}
}
setTimeout(done, 3000)
})
}
export function validateCaptionHostRequest(body) {
if (!/^[a-zA-Z0-9-]{12,80}$/.test(body?.id || '')) throw new Error('Invalid caption job ID.')
const style = String(body.style || body.captionStyle || 'descriptive')
if (!CAPTION_STYLES.includes(style)) throw new Error('Unknown caption style.')
return { id: body.id, style, imagePath: body.imagePath ? String(body.imagePath) : '' }
}
/** One llama-server process per caption. Process exit is the VRAM unload boundary (keep_alive 0). */
export function createCaptionHost({
prepare,
leaseValid,
spawnProcess = spawn,
modelsDir,
llamaServer,
dataDir,
fetchImpl = fetch
} = {}) {
const models = resolve(modelsDir || defaultModelsDir())
const executable = llamaServer || defaultLlamaServer()
const data = resolve(dataDir || process.env.CAPTION_JOBS_DIR || join(models, 'aigen-jobs'))
let active = null
let holdUntil = 0
if (existsSync(data)) {
for (const id of readdirSync(data)) {
try {
const path = join(data, id, 'status.json')
if (!existsSync(path)) continue
const state = JSON.parse(readFileSync(path, 'utf8'))
if (['running', 'starting', 'cancelling'].includes(state.status)) {
holdUntil = Date.now() + 10000
state.status = 'error'
state.error = 'Caption host restarted. GPU model unloaded.'
writeFileSync(path + '.tmp', JSON.stringify(state))
renameSync(path + '.tmp', path)
}
} catch { /* ignore bad records */ }
}
}
const dir = id => {
if (!/^[a-zA-Z0-9-]{12,80}$/.test(id || '')) throw new Error('Invalid caption job ID.')
return join(data, id)
}
const persist = job => {
const target = join(dir(job.id), 'status.json')
writeFileSync(target + '.tmp', JSON.stringify(job))
renameSync(target + '.tmp', target)
}
const read = id => {
if (active?.job.id === id) return { ...active.job, text: active.job.text }
const path = join(dir(id), 'status.json')
if (!existsSync(path)) return null
return JSON.parse(readFileSync(path, 'utf8'))
}
const modelPath = () => join(models, MODEL_NAME)
const mmprojPath = () => join(models, MMPROJ_NAME)
const configured = () => {
try {
if (!existsSync(modelPath()) || !existsSync(mmprojPath())) return false
if (executable.includes('\\') || executable.includes('/')) return existsSync(executable)
return true
} catch { return false }
}
return {
busy: () => Boolean(active) || Date.now() < holdUntil,
configured,
modelsDir: () => models,
read,
async upload(id, stream) {
if (read(id)?.status === 'complete') throw new Error('This caption job has already finished.')
const jobDir = dir(id)
mkdirSync(jobDir, { recursive: true })
const dest = join(jobDir, 'input.upload')
const { createWriteStream } = await import('node:fs')
const { pipeline } = await import('node:stream/promises')
const { Transform } = await import('node:stream')
let bytes = 0
await pipeline(stream, new Transform({
transform(chunk, _, callback) {
bytes += chunk.length
callback(bytes > 40 * 1024 * 1024 ? new Error('Image exceeds the 40 MB upload limit.') : null, chunk)
}
}), createWriteStream(dest, { flags: 'w' }))
if (bytes < 32) throw new Error('Image file is empty.')
renameSync(dest, join(jobDir, 'input.png'))
},
async start(body, lease) {
const request = validateCaptionHostRequest(body)
const previous = read(request.id)
if (previous) return previous
if (active || Date.now() < holdUntil) throw Object.assign(new Error('Caption is already running or releasing VRAM.'), { statusCode: 409 })
if (!configured()) throw new Error(`Caption model is not installed. Expected ${MODEL_NAME} and ${MMPROJ_NAME} under ${models}. Run scripts/setup-caption.ps1.`)
if (!leaseValid(lease)) throw new Error('GPU reservation expired.')
let imagePath = request.imagePath
if (imagePath) {
if (!existsSync(imagePath)) throw new Error('imagePath does not exist on the GPU host.')
mkdirSync(dir(request.id), { recursive: true })
const dest = join(dir(request.id), `input${extname(imagePath) || '.png'}`)
copyFileSync(imagePath, dest)
imagePath = dest
} else {
imagePath = join(dir(request.id), 'input.png')
if (!existsSync(imagePath)) {
const alt = readdirSync(dir(request.id)).find(name => /^input\./i.test(name))
if (!alt) throw new Error('Upload an image first or pass imagePath.')
imagePath = join(dir(request.id), alt)
}
}
const job = {
id: request.id,
status: 'starting',
message: 'Preparing GPU for caption',
progress: 0,
style: request.style,
startedAt: Date.now(),
checkedAt: Date.now()
}
active = { job, child: null, cancelled: false, abort: new AbortController() }
const run = active
mkdirSync(dir(job.id), { recursive: true })
writeFileSync(join(dir(job.id), 'request.json'), JSON.stringify({ ...request, imagePath, prompt: captionStylePrompt(request.style) }))
persist(job)
try {
if (prepare) await prepare()
if (run.cancelled || !leaseValid(lease)) throw new Error('Caption start cancelled or GPU reservation expired.')
job.status = 'running'
job.message = 'Loading Qwen2.5-VL caption model'
job.progress = 5
persist(job)
const port = await freePort()
const args = [
'-m', modelPath(),
'--mmproj', mmprojPath(),
'--host', '127.0.0.1',
'--port', String(port),
'-ngl', String(process.env.CAPTION_NGL || '99'),
'-c', String(process.env.CAPTION_CTX || '4096'),
'--jinja'
]
const child = spawnProcess(executable, args, {
windowsHide: true,
shell: false,
stdio: ['ignore', 'pipe', 'pipe'],
env: { ...process.env }
})
run.child = child
const log = chunk => appendFileSync(join(dir(job.id), 'worker.log'), chunk)
child.stdout.on('data', log)
child.stderr.on('data', log)
const watchdog = setInterval(() => {
if (!leaseValid(lease)) {
job.error = 'GPU reservation expired; caption stopped.'
run.abort.abort()
void killTree(child)
}
}, 5000)
try {
await waitForServer(port, run.abort.signal, 120_000, fetchImpl)
if (run.cancelled) throw new Error('Cancelled')
job.message = 'Captioning'
job.progress = 40
persist(job)
const bytes = readFileSync(imagePath)
const dataUrl = `data:${mimeFor(imagePath)};base64,${bytes.toString('base64')}`
const timeout = AbortSignal.timeout(Number(process.env.CAPTION_TIMEOUT_MS || 180_000))
const onAbort = () => { /* combined via race below */ }
run.abort.signal.addEventListener('abort', onAbort, { once: true })
let response
try {
response = await fetchImpl(`http://127.0.0.1:${port}/v1/chat/completions`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
signal: run.abort.signal.aborted ? run.abort.signal : timeout,
body: JSON.stringify({
temperature: 0.2,
max_tokens: 512,
messages: [{
role: 'user',
content: [
{ type: 'text', text: captionStylePrompt(request.style) },
{ type: 'image_url', image_url: { url: dataUrl } }
]
}]
})
})
} finally {
run.abort.signal.removeEventListener('abort', onAbort)
}
if (run.abort.signal.aborted) throw new Error('Cancelled')
if (!response.ok) {
const detail = await response.text().catch(() => '')
throw new Error(`llama-server caption failed (${response.status}): ${detail.slice(0, 400)}`)
}
const payload = await response.json()
const text = String(payload?.choices?.[0]?.message?.content || '').trim()
if (!text) throw new Error('Caption model returned empty text.')
job.text = text
job.status = run.cancelled ? 'cancelled' : 'complete'
job.message = job.status === 'complete' ? 'Caption ready' : 'Cancelled'
job.progress = 100
job.checkedAt = Date.now()
persist(job)
return { ...job }
} finally {
clearInterval(watchdog)
await killTree(child)
// keep_alive 0: process exit unloads GGUF+mmproj from VRAM
holdUntil = Date.now() + 2000
if (active === run) active = null
}
} catch (error) {
job.status = run.cancelled ? 'cancelled' : 'error'
job.error = error.message || String(error)
job.message = job.error
job.checkedAt = Date.now()
persist(job)
if (run.child) await killTree(run.child)
if (active === run) active = null
holdUntil = Date.now() + 2000
throw error
}
},
async cancel(id) {
if (active?.job.id !== id) return read(id)
const run = active
run.cancelled = true
run.job.status = 'cancelling'
run.job.message = 'Cancelling'
persist(run.job)
run.abort.abort()
if (run.child) await killTree(run.child)
return read(id)
},
/** Sync one-shot used by POST /caption { imagePath, style }. */
async captionOnce(body, lease) {
const id = body.id || `caption-${Date.now()}-${Math.random().toString(36).slice(2, 10)}`
const result = await this.start({ ...body, id }, lease)
if (result.status === 'complete') return { text: result.text, id: result.id, style: result.style }
throw new Error(result.error || 'Caption failed.')
}
}
}
export const captionModelFiles = { model: MODEL_NAME, mmproj: MMPROJ_NAME }
+66
View File
@@ -0,0 +1,66 @@
# Delete named YuE v1 / YuEGP weight folders only. Never recurse-wipe Comfy models\ or custom_nodes\.
[CmdletBinding()]
param(
[string]$YueGpRoot = 'C:\Users\ianjm\Development\YuEGP',
[string]$YueModels = '',
[string]$LogPath = ''
)
$ErrorActionPreference = 'Continue'
if (-not $YueModels) {
$YueModels = Join-Path $env:LOCALAPPDATA 'Comfy-Desktop\ComfyUI-Shared\models\yue'
}
if (-not $LogPath) {
$LogPath = Join-Path $PSScriptRoot 'cleanup-yue-v1.log'
}
function Log([string]$Message) {
$line = "$(Get-Date -Format o) $Message"
Add-Content -LiteralPath $LogPath -Value $line
Write-Host $line
}
function RemoveNamed([string]$Path) {
if (-not $Path) { return }
if (!(Test-Path -LiteralPath $Path)) {
Log "SKIP missing $Path"
return
}
try {
Remove-Item -LiteralPath $Path -Recurse -Force
Log "DELETED $Path"
} catch {
Log "FAIL $Path :: $($_.Exception.Message)"
}
}
Log "START cleanup-yue-v1 YueGpRoot=$YueGpRoot YueModels=$YueModels"
# Stage 1 / Stage 2 + codec under the shared yue models dir (named folders only)
RemoveNamed (Join-Path $YueModels 'YuE-s1-7B-anneal-en-cot')
RemoveNamed (Join-Path $YueModels 'YuE-s2-1B-general')
RemoveNamed (Join-Path $YueModels 'xcodec_mini_infer')
# Loose codec files that live beside the stage folders in this install
foreach ($name in @('ckpt_00360000.pth', 'decoder_131000.pth', 'decoder_151000.pth')) {
RemoveNamed (Join-Path $YueModels $name)
}
# YuEGP checkout codec copy + job dumps
RemoveNamed (Join-Path $YueGpRoot 'inference\xcodec_mini_infer')
RemoveNamed (Join-Path $YueGpRoot 'aigen-jobs')
# Hugging Face cache dirs that are clearly YuE v1 / old codec
$Hf = Join-Path $env:USERPROFILE '.cache\huggingface\hub'
RemoveNamed (Join-Path $Hf 'models--m-a-p--YuE-s1-7B-anneal-en-cot')
RemoveNamed (Join-Path $Hf 'models--m-a-p--YuE-s2-1B-general')
RemoveNamed (Join-Path $Hf 'models--m-a-p--xcodec_mini_infer')
# Leftover Comfy YuE Stage A/B weight copies by name only (no blanket models\ purge)
$ComfyModels = Join-Path $env:LOCALAPPDATA 'Comfy-Desktop\ComfyUI-Shared\models'
if (Test-Path -LiteralPath $ComfyModels) {
Get-ChildItem -LiteralPath $ComfyModels -Recurse -ErrorAction SilentlyContinue |
Where-Object { $_.Name -match '^(YUE_Stage_|YuE-s1-|YuE-s2-|yue_stage)' } |
ForEach-Object { RemoveNamed $_.FullName }
} else {
Log "SKIP missing $ComfyModels"
}
Log "DONE cleanup-yue-v1 log=$LogPath"
Write-Host "Left in place: ACE / H3 / Klein / Krea / LTX / upscale / LoRAs / LIBRARY_DIR / yuegp code paths."
+72 -28
View File
@@ -3,7 +3,8 @@ import { purgeStudio2Files } from './studio2-purge.mjs'
import { stableMemoryArgs } from './comfy-memory-policy.mjs' import { stableMemoryArgs } from './comfy-memory-policy.mjs'
import { createGpuReservation } from './gpu-reservation.mjs' import { createGpuReservation } from './gpu-reservation.mjs'
import { createGpuProxy } from './gpu-proxy.mjs' import { createGpuProxy } from './gpu-proxy.mjs'
import { createYueGpHost } from './yuegp-host.mjs' import { createYue2Host } from './yue2-host.mjs'
import { createCaptionHost } from './caption-host.mjs'
import http from 'node:http' import http from 'node:http'
import net from 'node:net' import net from 'node:net'
import { execFile, spawn } from 'node:child_process' import { execFile, spawn } from 'node:child_process'
@@ -147,7 +148,7 @@ let proxyTarget = 0
function ensureProxyListening() { function ensureProxyListening() {
if (proxyServer) return if (proxyServer) return
proxyServer = createGpuProxy({ target: () => proxyTarget, reservation: gpuReservation, authorized, markWork, externalBusy: () => yueGp.busy() || upscale.busy() }) proxyServer = createGpuProxy({ target: () => proxyTarget, reservation: gpuReservation, authorized, markWork, externalBusy: () => yue2.busy() || upscale.busy() || caption.busy() })
proxyServer.on('error', (error) => { proxyServer.on('error', (error) => {
console.log(JSON.stringify({ src: 'comfy-host-agent', event: 'proxy-error', error: String(error.message || error) })) console.log(JSON.stringify({ src: 'comfy-host-agent', event: 'proxy-error', error: String(error.message || error) }))
}) })
@@ -819,7 +820,7 @@ function purgeDesktopFiles(body) {
} }
const gpuReservation = createGpuReservation({ idle: async () => { const gpuReservation = createGpuReservation({ idle: async () => {
if (yueGp.busy() || upscale.busy()) return false if (yue2.busy() || upscale.busy() || caption.busy()) return false
if ((await trainingLock()).busy) return false if ((await trainingLock()).busy) return false
const healthy = await syncProxy() const healthy = await syncProxy()
if (healthy) { if (healthy) {
@@ -830,26 +831,67 @@ const gpuReservation = createGpuReservation({ idle: async () => {
return !(await processUp()) && !(await pythonMainUp().catch(() => true)) return !(await processUp()) && !(await pythonMainUp().catch(() => true))
} }) } })
const yueGp = createYueGpHost({ async function prepareExclusiveGpu(label) {
leaseValid: lease => gpuReservation.isOwner(lease), if ((await trainingLock()).busy) throw new Error('GPU is busy with training.')
prepare: async () => { if (label !== 'YuE2' && yue2.busy()) throw new Error('YuE2 is using the GPU.')
if ((await trainingLock()).busy) throw new Error('GPU is busy with training.') if (label !== 'caption' && caption.busy()) throw new Error('Caption is using the GPU.')
const healthy = await syncProxy() if (upscale.busy()) throw new Error('Local upscale is using the GPU.')
if (healthy) { const healthy = await syncProxy()
const queue = await fetchLocalQueue(healthy) if (healthy) {
if (!queue.ok || queue.running || queue.pending) throw new Error('Comfy is busy; YuEGP cannot start.') const queue = await fetchLocalQueue(healthy)
await stopComfyProcesses() if (!queue.ok || queue.running || queue.pending) throw new Error(`Comfy is busy; ${label} cannot start.`)
markAsleep()
}
if (await pythonMainUp()) throw new Error('Comfy has not stopped; retry after the GPU is free.')
} }
}) // Stop Comfy and refuse to launch while its Python still owns VRAM.
if (healthy || await processUp() || await pythonMainUp().catch(() => false)) {
await stopComfyProcesses()
markAsleep()
}
if (await pythonMainUp()) throw new Error('Comfy has not stopped; retry after the GPU is free.')
}
const upscale = createUpscaleHost({ leaseValid: token => gpuReservation.isOwner(token) }) const upscale = createUpscaleHost({ leaseValid: token => gpuReservation.isOwner(token) })
const yue2 = createYue2Host({
leaseValid: lease => gpuReservation.isOwner(lease),
prepare: async () => prepareExclusiveGpu('YuE2')
})
const caption = createCaptionHost({
leaseValid: lease => gpuReservation.isOwner(lease),
prepare: async () => prepareExclusiveGpu('caption')
})
async function handleControl(req, res) { async function handleControl(req, res) {
if (!authorized(req)) return json(res, 401, { ok: false, error: 'unauthorized' }) if (!authorized(req)) return json(res, 401, { ok: false, error: 'unauthorized' })
const url = new URL(req.url || '/', 'http://localhost') const url = new URL(req.url || '/', 'http://localhost')
if (url.pathname.startsWith('/caption')) {
const match = url.pathname.match(/^\/caption\/jobs\/([a-zA-Z0-9-]{12,80})(\/input|\/cancel)?$/)
if (req.method === 'GET' && url.pathname === '/caption/status') {
return json(res, 200, { configured: caption.configured(), busy: caption.busy(), backend: 'llama.cpp', modelsDir: caption.modelsDir() })
}
// Sync one-shot: POST /caption { imagePath, style } → load → caption → unload → { text }
if (req.method === 'POST' && url.pathname === '/caption') {
try {
const body = await readJson(req)
return json(res, 200, await caption.captionOnce(body, String(req.headers['x-aigen-gpu-lease'] || '')))
} catch (error) {
return json(res, error.statusCode || 400, { error: error.message || 'Caption failed' })
}
}
if (req.method === 'POST' && url.pathname === '/caption/jobs') {
return json(res, 200, await caption.start(await readJson(req), String(req.headers['x-aigen-gpu-lease'] || '')))
}
if (match && req.method === 'PUT' && match[2] === '/input') {
await caption.upload(match[1], req)
return json(res, 200, { uploaded: true })
}
if (match && req.method === 'POST' && match[2] === '/cancel') return json(res, 200, await caption.cancel(match[1]))
if (match && req.method === 'GET' && !match[2]) {
const job = caption.read(match[1])
return json(res, job ? 200 : 404, job || { error: 'Caption job not found' })
}
return json(res, 404, { error: 'Unknown caption endpoint' })
}
if (url.pathname.startsWith('/upscale/')) { if (url.pathname.startsWith('/upscale/')) {
const match = url.pathname.match(/^\/upscale\/jobs\/([a-zA-Z0-9-]{12,80})(\/input|\/video|\/cancel)?$/) const match = url.pathname.match(/^\/upscale\/jobs\/([a-zA-Z0-9-]{12,80})(\/input|\/video|\/cancel)?$/)
if (req.method === 'POST' && url.pathname === '/upscale/jobs') return json(res, 200, upscale.start(await readJson(req), String(req.headers['x-aigen-gpu-lease'] || ''))) if (req.method === 'POST' && url.pathname === '/upscale/jobs') return json(res, 200, upscale.start(await readJson(req), String(req.headers['x-aigen-gpu-lease'] || '')))
@@ -859,20 +901,20 @@ async function handleControl(req, res) {
if (match && req.method === 'GET' && !match[2]) { const job = upscale.read(match[1]); return json(res, job ? 200 : 404, job || { error: 'Upscale job not found' }) } if (match && req.method === 'GET' && !match[2]) { const job = upscale.read(match[1]); return json(res, job ? 200 : 404, job || { error: 'Upscale job not found' }) }
return json(res, 404, { error: 'Unknown upscale endpoint' }) return json(res, 404, { error: 'Unknown upscale endpoint' })
} }
if (url.pathname.startsWith('/yuegp/')) { if (url.pathname.startsWith('/yue2/')) {
const match = url.pathname.match(/^\/yuegp\/jobs\/([a-zA-Z0-9-]{12,80})(\/audio|\/cancel)?$/) const match = url.pathname.match(/^\/yue2\/jobs\/([a-zA-Z0-9-]{12,80})(\/audio|\/cancel)?$/)
if (req.method === 'GET' && url.pathname === '/yuegp/status') return json(res, 200, { configured: yueGp.configured(), busy: yueGp.busy(), backend: 'yuegp' }) if (req.method === 'GET' && url.pathname === '/yue2/status') return json(res, 200, { configured: yue2.configured(), busy: yue2.busy(), backend: 'yue2' })
if (req.method === 'POST' && url.pathname === '/yuegp/jobs') return json(res, 200, await yueGp.start(await readJson(req), String(req.headers['x-aigen-gpu-lease'] || ''))) if (req.method === 'POST' && url.pathname === '/yue2/jobs') return json(res, 200, await yue2.start(await readJson(req), String(req.headers['x-aigen-gpu-lease'] || '')))
if (match && req.method === 'POST' && match[2] === '/cancel') return json(res, 200, await yueGp.cancel(match[1])) if (match && req.method === 'POST' && match[2] === '/cancel') return json(res, 200, await yue2.cancel(match[1]))
if (match && req.method === 'GET' && match[2] === '/audio') { if (match && req.method === 'GET' && match[2] === '/audio') {
const path = yueGp.audio(match[1]) const path = yue2.audio(match[1])
return path ? streamFile(res, path) : json(res, 404, { error: 'Audio not ready' }) return path ? streamFile(res, path) : json(res, 404, { error: 'Audio not ready' })
} }
if (match && req.method === 'GET' && !match[2]) { if (match && req.method === 'GET' && !match[2]) {
const job = yueGp.read(match[1]) const job = yue2.read(match[1])
return json(res, job ? 200 : 404, job || { error: 'YuEGP job not found' }) return json(res, job ? 200 : 404, job || { error: 'YuE2 job not found' })
} }
return json(res, 404, { error: 'Unknown YuEGP endpoint' }) return json(res, 404, { error: 'Unknown YuE2 endpoint' })
} }
if (req.method === 'POST' && url.pathname.startsWith('/gpu/')) { if (req.method === 'POST' && url.pathname.startsWith('/gpu/')) {
const body = await readJson(req) const body = await readJson(req)
@@ -905,13 +947,15 @@ async function handleControl(req, res) {
proxyPort, proxyPort,
gpu: gpuReservation.availability(), gpu: gpuReservation.availability(),
training: { busy: lastTraining.busy }, training: { busy: lastTraining.busy },
yuegp: { busy: yueGp.busy(), configured: yueGp.configured() }, yue2: { busy: yue2.busy(), configured: yue2.configured() },
caption: { busy: caption.busy(), configured: caption.configured() },
upscale: { busy: upscale.busy(), engine: 'realesrgan-rife', local: true } upscale: { busy: upscale.busy(), engine: 'realesrgan-rife', local: true }
}) })
} }
if (req.method === 'POST' && url.pathname === '/start') { if (req.method === 'POST' && url.pathname === '/start') {
if (upscale.busy()) return json(res, 409, { message: 'Local upscale is using the GPU.' }) if (upscale.busy()) return json(res, 409, { message: 'Local upscale is using the GPU.' })
if (yueGp.busy()) return json(res, 409, { message: 'YuEGP is using the GPU.' }) if (yue2.busy()) return json(res, 409, { message: 'YuE2 is using the GPU.' })
if (caption.busy()) return json(res, 409, { message: 'Caption is using the GPU.' })
const training = await trainingLock() const training = await trainingLock()
if (training.busy) { if (training.busy) {
return json(res, 409, { return json(res, 409, {
@@ -1013,7 +1057,7 @@ const server = http.createServer(async (req, res) => {
} else await handleControl(req, res) } else await handleControl(req, res)
} catch (error) { } catch (error) {
req.resume() req.resume()
if ((String(req.url || '').startsWith('/yuegp/') || String(req.url || '').startsWith('/upscale/')) && !res.headersSent) return json(res, error.statusCode || 400, { error: error.message || 'YuEGP request failed' }) if ((String(req.url || '').startsWith('/yue2/') || String(req.url || '').startsWith('/upscale/') || String(req.url || '').startsWith('/caption')) && !res.headersSent) return json(res, error.statusCode || 400, { error: error.message || 'Host request failed' })
if (!res.headersSent) json(res, error.statusCode || 400, { ok: false, message: error.statusCode === 409 ? 'GPU is in use. Waiting for availability.' : 'GPU coordination request failed.' }) if (!res.headersSent) json(res, error.statusCode || 400, { ok: false, message: error.statusCode === 409 ? 'GPU is in use. Waiting for availability.' : 'GPU coordination request failed.' })
} }
}) })
+1 -1
View File
@@ -55,7 +55,7 @@ export function createGpuProxy({ target, reservation, authorized = () => true, m
try { try {
if (req.method === 'GET' || req.method === 'HEAD') await forward() if (req.method === 'GET' || req.method === 'HEAD') await forward()
else { else {
if (externalBusy()) { req.resume(); return reply(res, 409, { error: { message: 'YuEGP is using the GPU.' }, code: 'GPU_BUSY' }) } if (externalBusy()) { req.resume(); return reply(res, 409, { error: { message: 'A music or upscale job is using the GPU.' }, code: 'GPU_BUSY' }) }
if (!authorized(req)) { req.resume(); return reply(res, 401, { error: 'Unauthorized' }) } if (!authorized(req)) { req.resume(); return reply(res, 401, { error: 'Unauthorized' }) }
await reservation.permit(String(req.headers['x-aigen-gpu-lease'] || ''), async () => { markWork(); await forward() }) await reservation.permit(String(req.headers['x-aigen-gpu-lease'] || ''), async () => { markWork(); await forward() })
} }
+29 -7
View File
@@ -1,26 +1,48 @@
# Installs Desktop + Start Menu shortcuts that start the Comfy host agent # Installs Desktop, Start Menu, and Windows Startup shortcuts that start THIS
# without Cursor. The scheduled task still starts the agent at logon. # clone's Comfy host agent (YuE2) and replace any stale process on 8199.
$ErrorActionPreference = 'Stop' $ErrorActionPreference = 'Stop'
$root = (Resolve-Path (Join-Path $PSScriptRoot '..')).Path $root = (Resolve-Path (Join-Path $PSScriptRoot '..')).Path
$vbs = Join-Path $root 'scripts\launch-comfy-host-agent.vbs' $vbs = Join-Path $root 'scripts\launch-comfy-host-agent.vbs'
$startPs1 = Join-Path $root 'scripts\start-comfy-host-agent.ps1'
if (-not (Test-Path $vbs)) { throw "Missing $vbs" } if (-not (Test-Path $vbs)) { throw "Missing $vbs" }
if (-not (Test-Path $startPs1)) { throw "Missing $startPs1" }
$shell = New-Object -ComObject WScript.Shell $shell = New-Object -ComObject WScript.Shell
$desktop = [Environment]::GetFolderPath('Desktop') $desktop = [Environment]::GetFolderPath('Desktop')
$programs = Join-Path ([Environment]::GetFolderPath('StartMenu')) 'Programs\AIGen' $programs = Join-Path ([Environment]::GetFolderPath('StartMenu')) 'Programs\AIGen'
$startup = [Environment]::GetFolderPath('Startup')
New-Item -ItemType Directory -Force -Path $programs | Out-Null New-Item -ItemType Directory -Force -Path $programs | Out-Null
function Save-Shortcut([string]$dir) { function Save-Shortcut([string]$dir, [string]$name = 'Start Comfy Host Agent.lnk') {
$path = Join-Path $dir 'Start Comfy Host Agent.lnk' $path = Join-Path $dir $name
$lnk = $shell.CreateShortcut($path) $lnk = $shell.CreateShortcut($path)
$lnk.TargetPath = (Join-Path $env:SystemRoot 'System32\wscript.exe') $lnk.TargetPath = (Join-Path $env:SystemRoot 'System32\wscript.exe')
$lnk.Arguments = "//B `"$vbs`"" $lnk.Arguments = "//B `"$vbs`""
$lnk.WorkingDirectory = $root $lnk.WorkingDirectory = $root
$lnk.WindowStyle = 7 $lnk.WindowStyle = 7
$lnk.Description = 'Start the AIGen Comfy host agent in the background' $lnk.Description = 'Start THIS AIGen Comfy host agent (YuE2) and replace any stale agent on 8199'
$lnk.Save() $lnk.Save()
$path $path
} }
Save-Shortcut $desktop $saved = @(
Save-Shortcut $programs (Save-Shortcut $desktop),
(Save-Shortcut $programs),
(Save-Shortcut $startup 'AIGen Comfy Host Agent.lnk')
)
# Point the scheduled task at this clone when it already exists.
$taskName = 'AIGen Comfy Host Agent'
$existing = schtasks /Query /TN $taskName 2>$null
if ($LASTEXITCODE -eq 0) {
$hidden = Join-Path $root 'scripts\start-comfy-host-agent-hidden.vbs'
$tr = "wscript.exe //B `"$hidden`""
schtasks /Change /TN $taskName /TR $tr | Out-Null
Write-Host "Updated scheduled task '$taskName' -> $hidden"
}
Write-Host "Installed launchers for $root"
$saved | ForEach-Object { Write-Host " $_" }
Write-Host ''
Write-Host 'One click: double-click Desktop "Start Comfy Host Agent", or log off/on so Startup runs it.'
Write-Host 'That replaces any old node on 8199 with this clone (YuE2 /yue2/status).'
+7 -28
View File
@@ -1,35 +1,14 @@
' Start the Comfy host agent with no window. Safe to run when Cursor is closed. ' Start THIS clone's Comfy host agent (YuE2). Replaces any stale agent on 8199.
' Uses the "AIGen Comfy Host Agent" scheduled task when it exists. ' Safe to run at logon / from Startup / Desktop shortcut when Cursor is closed.
Option Explicit Option Explicit
Dim sh, fso, root, node, agent, rc Dim sh, fso, root, ps1
Set sh = CreateObject("WScript.Shell") Set sh = CreateObject("WScript.Shell")
Set fso = CreateObject("Scripting.FileSystemObject") Set fso = CreateObject("Scripting.FileSystemObject")
If AgentRunning() Then WScript.Quit 0
On Error Resume Next
rc = sh.Run("schtasks /run /tn ""AIGen Comfy Host Agent""", 0, True)
On Error GoTo 0
WScript.Sleep 1200
If AgentRunning() Then WScript.Quit 0
root = fso.GetParentFolderName(WScript.ScriptFullName) root = fso.GetParentFolderName(WScript.ScriptFullName)
root = fso.GetParentFolderName(root) root = fso.GetParentFolderName(root)
node = """C:\Program Files\nodejs\node.exe""" ps1 = root & "\scripts\start-comfy-host-agent.ps1"
agent = """" & root & "\scripts\comfy-host-agent.mjs""" If Not fso.FileExists(ps1) Then WScript.Quit 1
If Not fso.FileExists(Replace(node, """", "")) Then WScript.Quit 1
sh.CurrentDirectory = root
sh.Run node & " " & agent, 0, False
Function AgentRunning() sh.CurrentDirectory = root
Dim svc, query, item sh.Run "powershell.exe -NoProfile -WindowStyle Hidden -ExecutionPolicy Bypass -File """ & ps1 & """", 0, False
AgentRunning = False
Set svc = GetObject("winmgmts:\\.\root\cimv2")
Set query = svc.ExecQuery("SELECT CommandLine FROM Win32_Process WHERE Name='node.exe'")
For Each item In query
If InStr(LCase("" & item.CommandLine), "comfy-host-agent") > 0 Then
AgentRunning = True
Exit Function
End If
Next
End Function
+129
View File
@@ -0,0 +1,129 @@
# Download Qwen2.5-VL NSFW Caption V4 GGUF (Q5_K_M + mmproj only) for Studio 2 caption jobs.
# Does NOT pull the full safetensors repo (~16.6GB).
#
# Example:
# powershell -ExecutionPolicy Bypass -File scripts\setup-caption.ps1
# powershell -File scripts\setup-caption.ps1 -SkipDownload
[CmdletBinding()]
param(
[string]$ModelsRoot = '',
[string]$LlamaServer = '',
[switch]$SkipDownload,
[switch]$Force
)
$ErrorActionPreference = 'Stop'
$ProgressPreference = 'Continue'
function Write-Step([string]$Message) {
Write-Host ""
Write-Host "==> $Message" -ForegroundColor Cyan
}
function Write-Ok([string]$Message) {
Write-Host " $Message" -ForegroundColor Green
}
function Write-Warn([string]$Message) {
Write-Host " warn $Message" -ForegroundColor Yellow
}
function Resolve-ModelsRoot([string]$Hint) {
$shared = Join-Path $env:LOCALAPPDATA 'Comfy-Desktop\ComfyUI-Shared\models'
$candidates = @(
$Hint,
$env:COMFY_MODELS_ROOT,
$shared
) | Where-Object { $_ } | ForEach-Object { $_.TrimEnd('\', '/') }
foreach ($path in $candidates) {
if (Test-Path $path) { return (Resolve-Path $path).Path }
}
New-Item -ItemType Directory -Force -Path $shared | Out-Null
return (Resolve-Path $shared).Path
}
function Find-HfCli {
foreach ($name in @('hf', 'huggingface-cli')) {
$cmd = Get-Command $name -ErrorAction SilentlyContinue
if ($cmd) { return @{ Name = $name; Source = $cmd.Source } }
}
throw "Hugging Face CLI not found (hf / huggingface-cli). Install with: pip install -U huggingface_hub"
}
function Ensure-Dir([string]$Path) {
if (-not (Test-Path $Path)) {
New-Item -ItemType Directory -Force -Path $Path | Out-Null
}
}
Write-Step 'Resolve models root'
$modelsRoot = Resolve-ModelsRoot $ModelsRoot
$captionDir = Join-Path $modelsRoot 'caption\qwen25vl-7b-nsfw-v4'
Ensure-Dir $captionDir
Write-Ok "Models root: $modelsRoot"
Write-Ok "Caption dir: $captionDir"
$model = 'Qwen2.5-VL-7B-NSFW-Caption-V4.Q5_K_M.gguf'
$mmproj = 'Qwen2.5-VL-7B-NSFW-Caption-V4.mmproj-f16.gguf'
$modelPath = Join-Path $captionDir $model
$mmprojPath = Join-Path $captionDir $mmproj
if (-not $SkipDownload) {
Write-Step 'Download GGUF + mmproj only (~7GB)'
$needModel = $Force -or -not (Test-Path $modelPath)
$needMmproj = $Force -or -not (Test-Path $mmprojPath)
if (-not $needModel -and -not $needMmproj) {
Write-Ok 'Both weight files already present.'
} else {
$hf = Find-HfCli
if ($hf.Name -eq 'hf') {
& $hf.Source download mradermacher/Qwen2.5-VL-7B-NSFW-Caption-V4-GGUF `
--include $model `
--include $mmproj `
--local-dir $captionDir
} else {
& $hf.Source download mradermacher/Qwen2.5-VL-7B-NSFW-Caption-V4-GGUF `
--include $model `
--include $mmproj `
--local-dir $captionDir
}
if (-not (Test-Path $modelPath)) { throw "Missing $model after download." }
if (-not (Test-Path $mmprojPath)) { throw "Missing $mmproj after download." }
Write-Ok 'Download complete.'
}
} else {
Write-Warn 'SkipDownload set — not fetching weights.'
}
Write-Step 'Locate llama-server'
$server = $LlamaServer
if (-not $server) { $server = $env:CAPTION_LLAMA_SERVER }
if (-not $server) {
$cmd = Get-Command llama-server -ErrorAction SilentlyContinue
if ($cmd) { $server = $cmd.Source }
}
if ($server) {
Write-Ok "llama-server: $server"
} else {
Write-Warn 'llama-server not on PATH. Install with: winget install ggml.llamacpp'
Write-Warn 'Or set CAPTION_LLAMA_SERVER to a CUDA build of llama-server.exe'
}
$ready = @{
model = $modelPath
mmproj = $mmprojPath
modelsDir = $captionDir
llamaServer = $server
backend = 'llama.cpp'
updatedAt = (Get-Date).ToString('o')
}
$readyPath = Join-Path $captionDir 'aigen-ready.json'
$ready | ConvertTo-Json | Set-Content -Path $readyPath -Encoding utf8
Write-Ok "Wrote $readyPath"
Write-Host ""
Write-Host "Caption host looks for:" -ForegroundColor Cyan
Write-Host " $modelPath"
Write-Host " $mmprojPath"
Write-Host "Restart the Comfy host agent after install so /caption/status reports configured=true."
+225
View File
@@ -0,0 +1,225 @@
# Download Qwen Image 2.1 GGUF + companions into the live Comfy Desktop Shared models tree.
# Also ensures city96/ComfyUI-GGUF is present on the ComfyUI (1) install.
#
# Example:
# powershell -ExecutionPolicy Bypass -File scripts\setup-qwen21.ps1
# powershell -File scripts\setup-qwen21.ps1 -SkipDownload
# powershell -File scripts\setup-qwen21.ps1 -ComfyRoot "C:\...\ComfyUI (1)\ComfyUI"
[CmdletBinding()]
param(
[string]$ComfyRoot = '',
[string]$ModelsRoot = '',
[switch]$SkipDownload,
[switch]$SkipNode,
[switch]$Force
)
$ErrorActionPreference = 'Stop'
$ProgressPreference = 'Continue'
function Write-Step([string]$Message) {
Write-Host ""
Write-Host "==> $Message" -ForegroundColor Cyan
}
function Write-Ok([string]$Message) {
Write-Host " $Message" -ForegroundColor Green
}
function Write-Warn([string]$Message) {
Write-Host " warn $Message" -ForegroundColor Yellow
}
function Resolve-ComfyRoot([string]$Hint) {
$candidates = @(
$Hint,
$env:COMFY_ROOT,
$env:COMFYUI_ROOT,
(Join-Path $env:LOCALAPPDATA 'Comfy-Desktop\ComfyUI-Installs\ComfyUI (1)\ComfyUI'),
(Join-Path $env:LOCALAPPDATA 'Comfy-Desktop\ComfyUI-Installs\ComfyUI\ComfyUI'),
'C:\ComfyUI',
(Join-Path $env:USERPROFILE 'ComfyUI')
) | Where-Object { $_ } | ForEach-Object { $_.TrimEnd('\', '/') }
foreach ($path in $candidates) {
if (Test-Path (Join-Path $path 'main.py')) { return (Resolve-Path $path).Path }
if (Test-Path (Join-Path $path 'ComfyUI\main.py')) { return (Resolve-Path (Join-Path $path 'ComfyUI')).Path }
}
throw "Could not find ComfyUI root (main.py). Pass -ComfyRoot."
}
function Resolve-ModelsRoot([string]$Hint, [string]$ComfyRootPath) {
$shared = Join-Path $env:LOCALAPPDATA 'Comfy-Desktop\ComfyUI-Shared\models'
$candidates = @(
$Hint,
$env:COMFY_MODELS_ROOT,
$shared,
(Join-Path $ComfyRootPath 'models')
) | Where-Object { $_ } | ForEach-Object { $_.TrimEnd('\', '/') }
foreach ($path in $candidates) {
if (Test-Path $path) { return (Resolve-Path $path).Path }
}
New-Item -ItemType Directory -Force -Path $shared | Out-Null
return (Resolve-Path $shared).Path
}
function Find-HfCli {
foreach ($name in @('hf', 'huggingface-cli')) {
$cmd = Get-Command $name -ErrorAction SilentlyContinue
if ($cmd) { return $cmd.Source }
}
throw "Hugging Face CLI not found (hf / huggingface-cli). Install with: pip install -U huggingface_hub"
}
function Ensure-Dir([string]$Path) {
if (-not (Test-Path $Path)) {
New-Item -ItemType Directory -Force -Path $Path | Out-Null
}
}
function Assert-ModelFile([string]$Path, [long]$MinBytes) {
if (-not (Test-Path $Path)) { throw "Missing model file: $Path" }
$item = Get-Item $Path
if ($item.Length -lt $MinBytes) {
throw "Suspicious size for $Path ($($item.Length) bytes). Expected >= $MinBytes. Delete and re-run."
}
Write-Ok ("{0} {1:N0} bytes" -f $item.FullName, $item.Length)
}
function Flatten-IfNested([string]$DestDir, [string]$FileName) {
$nested = Get-ChildItem $DestDir -Recurse -File -Filter $FileName -ErrorAction SilentlyContinue |
Where-Object { $_.DirectoryName -ne $DestDir } |
Select-Object -First 1
if ($nested) {
$target = Join-Path $DestDir $FileName
if ((Test-Path $target) -and -not $Force) {
Write-Warn "Nested copy found but destination already exists: $target"
return
}
Move-Item -Force $nested.FullName $target
Write-Ok "Flattened $($nested.FullName) -> $target"
}
}
function Invoke-HfFile {
param(
[string]$Cli,
[string]$Repo,
[string]$File,
[string]$LocalDir
)
Ensure-Dir $LocalDir
$dest = Join-Path $LocalDir (Split-Path $File -Leaf)
if ((Test-Path $dest) -and -not $Force) {
$len = (Get-Item $dest).Length
if ($len -gt 1MB) {
Write-Ok "Already present: $dest ($len bytes)"
return
}
}
Write-Step "hf download $Repo $File"
& $Cli download $Repo $File --local-dir $LocalDir
if ($LASTEXITCODE -ne 0) { throw "hf download failed for $Repo / $File (exit $LASTEXITCODE)" }
Flatten-IfNested -DestDir $LocalDir -FileName (Split-Path $File -Leaf)
}
# --- resolve paths ---
Write-Step "Resolve ComfyUI root"
$root = Resolve-ComfyRoot $ComfyRoot
Write-Ok $root
Write-Step "Resolve models root (Desktop Shared preferred)"
$models = Resolve-ModelsRoot $ModelsRoot $root
Write-Ok $models
$Diff = Join-Path $models 'diffusion_models'
$TE = Join-Path $models 'text_encoders'
$VAE = Join-Path $models 'vae'
Ensure-Dir $Diff
Ensure-Dir $TE
Ensure-Dir $VAE
if (-not $SkipDownload) {
$cli = Find-HfCli
Write-Ok "HF CLI: $cli"
# DiT GGUF only (no Q6/Q5/Q4, no full bf16 DiT)
# Exact invocation required on this host:
# hf download hf://abenzerps/Qwen-Image-2.1-Uncensored-GGUF/qwen-image-2.1-Q8_0.gguf
# That form writes into the Hugging Face hub cache (not --local-dir).
Write-Step "hf download hf://abenzerps/Qwen-Image-2.1-Uncensored-GGUF/qwen-image-2.1-Q8_0.gguf"
$ditName = 'qwen-image-2.1-Q8_0.gguf'
$ditDest = Join-Path $Diff $ditName
if ((Test-Path $ditDest) -and -not $Force -and (Get-Item $ditDest).Length -gt 1MB) {
Write-Ok "Already present: $ditDest ($((Get-Item $ditDest).Length) bytes)"
} else {
& $cli download "hf://abenzerps/Qwen-Image-2.1-Uncensored-GGUF/$ditName"
if ($LASTEXITCODE -ne 0) { throw "hf download failed for DiT GGUF (exit $LASTEXITCODE)" }
$cacheRoot = Join-Path $env:USERPROFILE '.cache\huggingface\hub\models--abenzerps--Qwen-Image-2.1-Uncensored-GGUF'
$found = Get-ChildItem $cacheRoot -Recurse -File -Filter $ditName -ErrorAction SilentlyContinue |
Sort-Object LastWriteTime -Descending |
Select-Object -First 1
if (-not $found) { throw "hf download finished but $ditName was not found under $cacheRoot" }
Ensure-Dir $Diff
Copy-Item -Force $found.FullName $ditDest
Write-Ok "Copied $($found.FullName) -> $ditDest ($((Get-Item $ditDest).Length) bytes)"
}
# abenzerps ships kv_count=0 + Q8_0 1D RMSNorms (packed 136 vs logical 128).
# Tag architecture=qwen_image and promote small/1D tensors to F32 for ComfyUI-GGUF.
Write-Step "Tag DiT GGUF (architecture + F32 norms)"
$tagger = Join-Path $PSScriptRoot 'tag-qwen21-gguf.py'
$pyCandidates = @(
(Join-Path $root '.venv\Scripts\python.exe'),
(Join-Path (Split-Path $root) 'standalone-env\python.exe'),
(Get-Command python -ErrorAction SilentlyContinue | Select-Object -ExpandProperty Source)
) | Where-Object { $_ -and (Test-Path $_) }
$py = $pyCandidates | Select-Object -First 1
if (-not $py) { throw "No Python found to run $tagger" }
$fixed = Join-Path $Diff 'qwen-image-2.1-Q8_0.fixed.gguf'
& $py $tagger $ditDest $fixed
if ($LASTEXITCODE -ne 0) { throw "tag-qwen21-gguf.py failed (exit $LASTEXITCODE)" }
Copy-Item -Force $fixed $ditDest
Write-Ok "Installed tagged GGUF at $ditDest"
# TE + VAE companions from Comfy-Org (INT8 TE for 16 GB VRAM)
Invoke-HfFile -Cli $cli -Repo 'Comfy-Org/Qwen-Image-2.1' -File 'text_encoders/qwen3vl_8b_int8_convrot.safetensors' -LocalDir $TE
Invoke-HfFile -Cli $cli -Repo 'Comfy-Org/Qwen-Image-2.1' -File 'vae/qwen_image_2.1_vae_bf16.safetensors' -LocalDir $VAE
}
Write-Step "Verify weights"
Assert-ModelFile (Join-Path $Diff 'qwen-image-2.1-Q8_0.gguf') 7000000000
Assert-ModelFile (Join-Path $TE 'qwen3vl_8b_int8_convrot.safetensors') 1000000000
Assert-ModelFile (Join-Path $VAE 'qwen_image_2.1_vae_bf16.safetensors') 100000000
if (-not $SkipNode) {
Write-Step "city96 ComfyUI-GGUF"
$nodes = Join-Path $root 'custom_nodes\ComfyUI-GGUF'
if (-not (Test-Path $nodes)) {
git clone https://github.com/city96/ComfyUI-GGUF $nodes
if ($LASTEXITCODE -ne 0) { throw "git clone ComfyUI-GGUF failed" }
Write-Ok "Cloned $nodes"
} else {
Write-Ok "Already present: $nodes"
}
$py = Join-Path $root '.venv\Scripts\python.exe'
if (-not (Test-Path $py)) {
$py = Join-Path (Split-Path $root) 'standalone-env\python.exe'
}
if (Test-Path $py) {
Write-Step "Ensure gguf package in Comfy python ($py)"
& $py -m pip install -q gguf
if ($LASTEXITCODE -ne 0) { Write-Warn "pip install gguf returned $LASTEXITCODE" }
else { Write-Ok "gguf importable" }
} else {
Write-Warn "Comfy python not found; install gguf into the interpreter that runs main.py"
}
}
Write-Host ""
Write-Host "Done. Restart Comfy only when /status shows gpu.busy=false." -ForegroundColor Cyan
Write-Host "ComfyRoot=$root"
Write-Host "ModelsRoot=$models"
+66
View File
@@ -0,0 +1,66 @@
# Standalone YuE2 only. Never install into Comfy's Python environment.
# Downloads YuE2-3B + YuE2-Vae into YUE2_ROOT\models so the UI does not hit HF on first Generate.
[CmdletBinding()]
param(
[string]$Root = 'C:\Users\ianjm\Development\YuE2',
[string]$Uv = 'uv',
[string]$Models = ''
)
$ErrorActionPreference = 'Stop'
$Root = [IO.Path]::GetFullPath($Root)
if (-not $Models) { $Models = Join-Path $Root 'models' }
$Models = [IO.Path]::GetFullPath($Models)
function Run([string]$Exe, [string[]]$Arguments) {
& $Exe @Arguments
if ($LASTEXITCODE -ne 0) { throw "$Exe failed ($LASTEXITCODE)" }
}
Write-Host "YuE2 setup root=$Root models=$Models"
if (!(Test-Path -LiteralPath $Root)) {
Run git @('clone', 'https://github.com/multimodal-art-projection/YuE.git', $Root)
}
Run $Uv @('venv', '--allow-existing', '--python', '3.12', (Join-Path $Root '.venv'))
$python = Join-Path $Root '.venv\Scripts\python.exe'
# Install the package first, then force CUDA torch back on (editable deps can pull CPU wheels).
Run $Uv @('pip', 'install', '--python', $python, '-e', $Root)
Run $Uv @('pip', 'install', '--python', $python, 'soundfile', 'psutil', 'huggingface_hub', 'hf_xet')
Run $Uv @('pip', 'install', '--python', $python, 'torch', 'torchaudio', '--reinstall', '--index-url', 'https://download.pytorch.org/whl/cu128')
New-Item -ItemType Directory -Force -Path $Models | Out-Null
$modelDir = Join-Path $Models 'YuE2-3B'
$vaeDir = Join-Path $Models 'YuE2-Vae'
if (!(Test-Path -LiteralPath (Join-Path $modelDir 'config.json')) -and !(Test-Path -LiteralPath (Join-Path $modelDir 'model.safetensors.index.json'))) {
Write-Host "Downloading m-a-p/YuE2-3B → $modelDir"
Run $python @(
'-c',
'from huggingface_hub import snapshot_download; import sys; snapshot_download(sys.argv[1], local_dir=sys.argv[2])',
'm-a-p/YuE2-3B',
$modelDir
)
} else {
Write-Host "Reusing existing model at $modelDir"
}
if (!(Test-Path -LiteralPath (Join-Path $vaeDir 'config.json'))) {
Write-Host "Downloading m-a-p/YuE2-Vae → $vaeDir"
Run $python @(
'-c',
'from huggingface_hub import snapshot_download; import sys; snapshot_download(sys.argv[1], local_dir=sys.argv[2])',
'm-a-p/YuE2-Vae',
$vaeDir
)
} else {
Write-Host "Reusing existing VAE at $vaeDir"
}
Run $python @('-c', 'import torch; from yue2 import YuE2Pipeline; assert torch.cuda.is_available(); print(torch.__version__, torch.cuda.get_device_name(0))')
$ready = @{
model = $modelDir
vae = $vaeDir
modelId = 'm-a-p/YuE2-3B'
vaeId = 'm-a-p/YuE2-Vae'
modelsDir = $Models
python = $python
root = $Root
}
$ready | ConvertTo-Json | Set-Content -LiteralPath (Join-Path $Root 'aigen-ready.json') -Encoding utf8
Write-Host "YuE2 ready at $Root"
Write-Host "Local model=$modelDir"
Write-Host "Local vae=$vaeDir"
Write-Host "Set YUE2_ROOT=$Root YUE2_PYTHON=$python YUE2_MODEL=$modelDir YUE2_VAE=$vaeDir if not using defaults. No Comfy environment was modified."
+12 -8
View File
@@ -1,11 +1,15 @@
' Launch the Comfy host agent with no console window. ' Launch the Comfy host agent with no console window (Task Scheduler wait mode).
' Window style 0 = hidden. Wait = True so Task Scheduler keeps the task marked Running. ' Replaces any stale agent on 8199, then runs THIS clone's agent until it exits.
Option Explicit Option Explicit
Dim sh, root, node, agent Dim sh, fso, root, ps1
Set sh = CreateObject("WScript.Shell") Set sh = CreateObject("WScript.Shell")
root = CreateObject("Scripting.FileSystemObject").GetParentFolderName(WScript.ScriptFullName) Set fso = CreateObject("Scripting.FileSystemObject")
root = CreateObject("Scripting.FileSystemObject").GetParentFolderName(root)
node = """C:\Program Files\nodejs\node.exe""" root = fso.GetParentFolderName(WScript.ScriptFullName)
agent = """" & root & "\scripts\comfy-host-agent.mjs""" root = fso.GetParentFolderName(root)
ps1 = root & "\scripts\start-comfy-host-agent.ps1"
If Not fso.FileExists(ps1) Then WScript.Quit 1
sh.CurrentDirectory = root sh.CurrentDirectory = root
sh.Run node & " " & agent, 0, True ' Wait=True so Task Scheduler keeps the task marked Running while the agent lives.
sh.Run "powershell.exe -NoProfile -WindowStyle Hidden -ExecutionPolicy Bypass -File """ & ps1 & """ -Wait", 0, True
+34
View File
@@ -0,0 +1,34 @@
# Replace any stale Comfy host agent, then start THIS clone's agent on 8199.
# Called by launch-comfy-host-agent.vbs (Startup / Desktop / Start Menu).
# -Wait: run in-foreground for Task Scheduler (keeps task marked Running).
param([switch]$Wait)
$ErrorActionPreference = 'SilentlyContinue'
$root = (Resolve-Path (Join-Path $PSScriptRoot '..')).Path
$node = 'C:\Program Files\nodejs\node.exe'
$agent = Join-Path $root 'scripts\comfy-host-agent.mjs'
if (-not (Test-Path $node)) { throw "Missing $node" }
if (-not (Test-Path $agent)) { throw "Missing $agent" }
Get-CimInstance Win32_Process -Filter "Name='node.exe'" |
Where-Object { $_.CommandLine -match 'comfy-host-agent' } |
ForEach-Object { Stop-Process -Id $_.ProcessId -Force }
Get-NetTCPConnection -LocalPort 8199 -State Listen -ErrorAction SilentlyContinue |
ForEach-Object { Stop-Process -Id $_.OwningProcess -Force }
Start-Sleep -Milliseconds 400
if ($Wait) {
Set-Location $root
& $node $agent
exit $LASTEXITCODE
}
$psi = New-Object System.Diagnostics.ProcessStartInfo
$psi.FileName = $node
$psi.Arguments = "`"$agent`""
$psi.WorkingDirectory = $root
$psi.UseShellExecute = $false
$psi.CreateNoWindow = $true
$psi.WindowStyle = [System.Diagnostics.ProcessWindowStyle]::Hidden
[void][System.Diagnostics.Process]::Start($psi)
+6 -1
View File
@@ -1,6 +1,11 @@
import { resolve, relative, isAbsolute } from 'node:path' import { resolve, relative, isAbsolute } from 'node:path'
import { existsSync, realpathSync, statSync, unlinkSync } from 'node:fs' import { existsSync, realpathSync, statSync, unlinkSync } from 'node:fs'
import { scopedFile } from '../shared/studio2/contracts.mjs' // Keep the host cleanup endpoint independently deployable from the web application.
function scopedFile(file, prefix) {
const path=[file.subfolder,file.filename].map(s=>String(s||'').replaceAll('\\','/')).filter(Boolean).join('/')
const p=String(prefix||'').replaceAll('\\','/').replace(/\/$/,'')
return !!p && !path.startsWith('/') && !path.split('/').some(s=>s==='..'||s==='.') && path.startsWith(p+'/')
}
/** Exact Studio 2 files only: no folder fallback, sibling deletion, or sweeping. */ /** Exact Studio 2 files only: no folder fallback, sibling deletion, or sweeping. */
export function purgeStudio2Files({prefix, files}, rootsForType) { export function purgeStudio2Files({prefix, files}, rootsForType) {
+132
View File
@@ -0,0 +1,132 @@
#!/usr/bin/env python3
"""Make abenzerps Qwen Image 2.1 DiT GGUF loadable in city96 ComfyUI-GGUF.
abenzerps/Qwen-Image-2.1-Uncensored-GGUF ships Q8_0 with:
- kv_count=0 (no general.architecture)
- 1D RMSNorm weights quantized to Q8_0 (logical 128 -> packed 136),
which breaks Comfy's fused rms_rope path
This rewrite:
1. Adds general.architecture=qwen_image
2. Dequantizes small / 1D tensors to F32 (city96 convert keeps them hiprec)
"""
from __future__ import annotations
import argparse
import sys
from pathlib import Path
# Match city96 ComfyUI-GGUF/tools/convert.py QUANTIZATION_THRESHOLD
QUANTIZATION_THRESHOLD = 1024
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("src", type=Path)
ap.add_argument("dst", type=Path, nargs="?", default=None)
ap.add_argument("--arch", default="qwen_image")
ap.add_argument("--name", default="qwen-image-2.1")
ap.add_argument("--inplace", action="store_true", help="Replace src after a successful tag")
args = ap.parse_args()
import gguf
import numpy as np
src = args.src.resolve()
if not src.is_file():
print(f"missing source: {src}", file=sys.stderr)
return 1
dst = (args.dst.resolve() if args.dst else src.with_name(src.stem + ".tagged.gguf"))
reader = gguf.GGUFReader(str(src))
def get_field(name: str):
field = reader.fields.get(name)
if field is None:
return None
try:
return field.contents()
except Exception:
return None
if dst == src:
dst = src.with_name(src.stem + ".retag.gguf")
f32 = gguf.GGMLQuantizationType.F32
compat = {gguf.GGMLQuantizationType.F32, gguf.GGMLQuantizationType.F16, gguf.GGMLQuantizationType.BF16}
print(f"rewriting {src} -> {dst} (arch={args.arch}, tensors={len(reader.tensors)})")
writer = gguf.GGUFWriter(str(dst), arch=args.arch, use_temp_file=True)
writer.add_name(args.name)
try:
writer.add_type("model")
except Exception:
pass
promoted = 0
for tensor in reader.tensors:
logical = tuple(int(x) for x in tensor.shape)
# GGUF stores dims reversed vs torch; logical numel is what matters
numel = 1
for d in logical:
numel *= d
qtype = tensor.tensor_type
data = tensor.data
if hasattr(data, "copy"):
data = data.copy()
needs_f32 = qtype not in compat and (len(logical) <= 1 or numel <= QUANTIZATION_THRESHOLD)
if needs_f32:
# Dequantize packed blocks -> float32 with logical shape (GGUF order)
dequant = gguf.quants.dequantize(np.asarray(data), qtype).astype(np.float32, copy=False)
expected = numel
if dequant.size != expected:
print(
f"dequant size mismatch {tensor.name}: got {dequant.size} want {expected} "
f"shape={logical} qtype={qtype}",
file=sys.stderr,
)
return 3
data = dequant.reshape(logical)
qtype = f32
promoted += 1
writer.add_tensor(tensor.name, data, raw_dtype=qtype)
writer.write_header_to_file()
writer.write_kv_data_to_file()
writer.write_tensors_to_file(progress=True)
writer.close()
check = gguf.GGUFReader(str(dst))
field = check.fields.get("general.architecture")
got = field.contents() if field is not None else None
print(
f"verified architecture={got!r} tensors={len(check.tensors)} "
f"promoted_f32={promoted} size={dst.stat().st_size}"
)
if got != args.arch or len(check.tensors) != len(reader.tensors):
print("tag failed", file=sys.stderr)
return 2
# Spot-check a known norm weight is F32 @ 128
sample = next((t for t in check.tensors if t.name.endswith("attn.norm_q.weight")), None)
if sample is not None:
print(
f"sample {sample.name}: shape={tuple(int(x) for x in sample.shape)} "
f"type={sample.tensor_type.name} data_shape={tuple(sample.data.shape)}"
)
if args.inplace:
bak = src.with_suffix(src.suffix + ".untagged.bak")
if bak.exists():
bak.unlink()
src.replace(bak)
dst.replace(src)
print(f"inplace: {src} (backup {bak})")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+134
View File
@@ -0,0 +1,134 @@
import { spawn } from 'node:child_process'
import { existsSync, mkdirSync, readFileSync, writeFileSync, renameSync, appendFileSync, readdirSync } from 'node:fs'
import { join, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
export function validateYue2Request(body) {
const duration = body.duration ?? 60
if (!Number.isInteger(duration) || duration < 15 || duration > 150) throw new Error('YuE2 target length must be 15–150 seconds.')
if (!Number.isInteger(body.seed) || body.seed < 0 || body.seed > 2147483647) throw new Error('Invalid seed.')
if (!/^[a-zA-Z0-9-]{12,80}$/.test(body.id || '')) throw new Error('Invalid job ID.')
const tags = String(body.tags || '').trim()
const lyrics = String(body.lyrics || '').trim()
if (!tags || tags.length > 2000 || !lyrics || lyrics.length > 8000) throw new Error('Genre tags and non-empty lyrics are required.')
return { id: body.id, duration, seed: body.seed, tags, lyrics }
}
/** One isolated Python process per song. A process exit is the GPU release boundary. */
export function createYue2Host({ prepare, leaseValid, spawnProcess = spawn, root, python, dataDir } = {}) {
const repo = resolve(root || process.env.YUE2_ROOT || 'C:\\Users\\ianjm\\Development\\YuE2')
const executable = python || process.env.YUE2_PYTHON || join(repo, '.venv', 'Scripts', 'python.exe')
const data = resolve(dataDir || process.env.YUE2_JOBS_DIR || join(repo, 'aigen-jobs'))
const worker = fileURLToPath(new URL('./yue2-worker.py', import.meta.url))
let active = null
// A worker whose parent died exits within two seconds. Block new GPU owners
// across a host restart until that watchdog has had time to run.
const restartHoldUntil = existsSync(data) && readdirSync(data).some(id => {
try { return ['running', 'starting', 'cancelling'].includes(JSON.parse(readFileSync(join(data, id, 'status.json'), 'utf8')).status) }
catch { return false }
}) ? Date.now() + 10000 : 0
const dir = id => {
if (!/^[a-zA-Z0-9-]{12,80}$/.test(id || '')) throw new Error('Invalid job ID.')
return join(data, id)
}
const persist = job => {
const target = join(dir(job.id), 'status.json')
writeFileSync(target + '.tmp', JSON.stringify(job))
renameSync(target + '.tmp', target)
}
const read = id => {
if (active?.job.id === id) return { ...active.job }
const path = join(dir(id), 'status.json')
if (!existsSync(path)) return null
const job = JSON.parse(readFileSync(path, 'utf8'))
if (['running', 'starting', 'cancelling'].includes(job.status)) {
job.status = 'error'; job.error = 'YuE2 host restarted. Intermediate files are preserved.'
}
return job
}
return {
busy: () => Boolean(active) || Date.now() < restartHoldUntil,
configured: () => existsSync(executable) && existsSync(join(repo, 'aigen-ready.json')),
read,
audio(id) {
return read(id)?.status === 'complete' ? join(dir(id), 'audio.wav') : null
},
async start(body, lease) {
const request = validateYue2Request(body)
const previous = read(request.id)
if (previous) return previous // POST retries cannot generate the same song twice.
if (active || Date.now() < restartHoldUntil) throw Object.assign(new Error('YuE2 is already running or releasing a previous worker.'), { statusCode: 409 })
if (!existsSync(executable) || !existsSync(join(repo, 'aigen-ready.json'))) throw new Error('YuE2 is not installed. Run scripts/setup-yue2.ps1 on the GPU host.')
const job = { id: request.id, status: 'starting', message: 'Preparing GPU for YuE2', progress: 0, startedAt: Date.now(), checkedAt: Date.now() }
active = { job, child: null, cancelled: false }
const run = active
mkdirSync(dir(job.id), { recursive: true })
persist(job)
try {
await prepare()
if (run.cancelled || !leaseValid(lease)) throw new Error('YuE2 start cancelled or GPU reservation expired.')
const installation = JSON.parse(readFileSync(join(repo, 'aigen-ready.json'), 'utf8').replace(/^\uFEFF/, ''))
const localModel = process.env.YUE2_MODEL || installation.model || 'm-a-p/YuE2-3B'
const localVae = process.env.YUE2_VAE || installation.vae || 'm-a-p/YuE2-Vae'
writeFileSync(join(dir(job.id), 'request.json'), JSON.stringify({
...request,
model: localModel,
vae: localVae
}))
const child = spawnProcess(executable, ['-u', worker, '--root', repo, '--request', join(dir(job.id), 'request.json')],
{ cwd: repo, windowsHide: true, shell: false, stdio: ['ignore', 'pipe', 'pipe'], env: { ...process.env, PYTHONUTF8: '1', TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD: '1' } })
run.child = child
job.status = 'running'; job.message = 'Loading YuE2'; persist(job)
let tail = ''
let lines = ''
const log = chunk => { appendFileSync(join(dir(job.id), 'worker.log'), chunk); tail = (tail + chunk.toString()).slice(-6000) }
child.stderr.on('data', log)
child.stdout.on('data', chunk => {
log(chunk); lines += chunk.toString()
const parts = lines.split(/\r?\n/); lines = parts.pop().slice(-64000)
for (const line of parts) {
if (!line.startsWith('AIGEN_EVENT ')) continue
try {
const event = JSON.parse(line.slice(12))
for (const key of ['stage', 'message', 'progress', 'step', 'maxStep', 'error', 'duration']) if (event[key] !== undefined) job[key] = event[key]
job.checkedAt = Date.now(); persist(job)
} catch { /* Invalid log lines are not state transitions. */ }
}
})
const watchdog = setInterval(() => {
if (!leaseValid(lease)) { job.error = 'GPU reservation expired; YuE2 stopped.'; child.kill() }
}, 5000)
let finished = false
const finish = (code, error) => {
if (finished) return
finished = true
clearInterval(watchdog)
job.status = run.cancelled ? 'cancelled' : !error && code === 0 && existsSync(join(dir(job.id), 'audio.wav')) ? 'complete' : 'error'
if (job.status === 'error') job.error ||= error?.message || tail || `YuE2 exited with code ${code}`
job.message = job.status === 'complete' ? 'Audio ready' : job.status === 'cancelled' ? 'Cancelled' : job.error
job.checkedAt = Date.now(); persist(job)
if (active === run) active = null
}
child.once('error', error => finish(null, error))
child.once('close', code => finish(code))
return { ...job }
} catch (error) {
job.status = run.cancelled ? 'cancelled' : 'error'; job.error = error.message; persist(job)
if (active === run) active = null
throw error
}
},
async cancel(id) {
if (active?.job.id !== id) return read(id)
const run = active
run.cancelled = true
run.job.status = 'cancelling'
persist(run.job)
if (run.child) {
const child = run.child
await new Promise(resolve => { child.once('close', resolve); child.kill() })
}
return read(id)
}
}
}
+200
View File
@@ -0,0 +1,200 @@
"""Headless YuE2 adapter: plan → generate_semantic → synthesize → decode.
No Comfy imports. No score editor, covers, or auto-fallback to YuEGP / ACE.
Stdout AIGEN_EVENT lines are consumed by the device-local host agent.
"""
import argparse
import gc
import json
import os
from pathlib import Path
import sys
import time
import threading
# Must be set before the first torch import on this 16GB Windows host.
os.environ.setdefault('PYTORCH_CUDA_ALLOC_CONF', 'expandable_segments:True')
def emit(**event):
print('AIGEN_EVENT ' + json.dumps(event), flush=True)
def normalize_lyrics(text):
# YuE2 accepts multiple [Verse]/[Chorus] headings. Only wrap when none exist.
import re
text = str(text or '').strip()
if not text:
raise ValueError('YuE2 requires non-empty lyrics.')
if re.search(r'\[[^\]]+\]', text):
return text if text.endswith('\n') else text + '\n'
return '[song]\n' + text + '\n\n'
def is_cuda_oom(error):
message = str(error).lower()
return 'out of memory' in message or ('cuda' in message and 'alloc' in message) or 'cudnn_status_alloc_failed' in message
def resolve_attention_backend(torch_mod):
"""Never flash. Prefer torch-eager (no CUDA graphs) on the 16GB 5080."""
# Windows wheels expose flash ops without USE_FLASH_ATTENTION; never select flash.
# torch-eager disables GraphAR / CUDA graphs via YuE2Pipeline.backend.
return 'sdpa', 'torch-eager'
def patch_graph_attention(attention_backend):
from yue2.cuda_graph import GraphAR
original = GraphAR.__init__
def init(self, model, prefixes, max_tokens, *, capture=True, attention_backend='auto', fuse_projections=False):
if attention_backend in ('auto', 'flash'):
attention_backend = patch_graph_attention.forced
return original(self, model, prefixes, max_tokens, capture=capture,
attention_backend=attention_backend, fuse_projections=fuse_projections)
patch_graph_attention.forced = attention_backend
GraphAR.__init__ = init
def free_cuda(torch_mod):
gc.collect()
if torch_mod.cuda.is_available():
torch_mod.cuda.empty_cache()
torch_mod.cuda.synchronize()
def memory_snapshot(torch_mod, stage):
if not torch_mod.cuda.is_available():
emit(stage=stage, message=f'YuE2 {stage}', cuda=False)
return
free, total = torch_mod.cuda.mem_get_info()
emit(
stage=stage,
message=f'YuE2 {stage}',
gpuAllocatedMiB=round(torch_mod.cuda.memory_allocated() / 1024 ** 2),
gpuReservedMiB=round(torch_mod.cuda.memory_reserved() / 1024 ** 2),
gpuFreeMiB=round(free / 1024 ** 2),
gpuTotalMiB=round(total / 1024 ** 2),
)
def main():
import psutil
parent = psutil.Process(os.getppid())
def parent_watchdog():
while parent.is_running():
time.sleep(2)
os._exit(2) # The host died: never leave an orphan consuming the GPU.
threading.Thread(target=parent_watchdog, daemon=True).start()
parser = argparse.ArgumentParser()
parser.add_argument('--root', required=True)
parser.add_argument('--request', required=True)
cli = parser.parse_args()
request = json.loads(Path(cli.request).read_text(encoding='utf-8-sig'))
root = Path(cli.root).resolve()
output = Path(cli.request).resolve().parent
duration = int(request.get('duration', 60))
if not 15 <= duration <= 150:
raise ValueError('Target length must be 15–150 seconds.')
# VAE downsampling_ratio 1920 @ 48 kHz → 25 semantic / latent frames per second.
max_tokens = max(200, min(9000, duration * 25))
lyrics = normalize_lyrics(request['lyrics'])
style = ' '.join(str(request['tags']).split())
seed = int(request['seed'])
model = request.get('model') or os.environ.get('YUE2_MODEL') or 'm-a-p/YuE2-3B'
vae = request.get('vae') or os.environ.get('YUE2_VAE') or 'm-a-p/YuE2-Vae'
os.chdir(root)
if str(root) not in sys.path:
sys.path.insert(0, str(root))
import torch
import soundfile as sf
attention_backend, pipeline_backend = resolve_attention_backend(torch)
emit(stage='loading', progress=1, message='Loading YuE2', model=model, vae=vae,
duration=duration, maxTokens=max_tokens, attention_backend=attention_backend,
pipeline_backend=pipeline_backend,
cudaAllocConf=os.environ.get('PYTORCH_CUDA_ALLOC_CONF'))
from yue2 import YuE2Pipeline
from yue2.protocol import Sampling
if not torch.cuda.is_available():
raise RuntimeError('YuE2 requires a CUDA GPU; CPU fallback is disabled.')
patch_graph_attention(attention_backend)
free_cuda(torch)
memory_snapshot(torch, 'memory-before-load')
# Prefer tiled VAE; do not cap the process with set_per_process_memory_fraction.
pipe_load = dict(
device='cuda',
backend=pipeline_backend,
vae_core_frames=512,
offload_ar=True,
progress=False,
)
cot = 'full'
pipe_kwargs = dict(style=style, lyrics=lyrics, cot=cot, seed=seed)
# Cap semantic length from target seconds. Do not FFmpeg-trim after decode.
semantic_sampling = Sampling(max_tokens=max_tokens, min_tokens=min(200, max_tokens))
# YuE2Pipeline.__init__ always calls set_per_process_memory_fraction; skip it on this 16GB host.
_set_fraction = torch.cuda.set_per_process_memory_fraction
torch.cuda.set_per_process_memory_fraction = lambda *args, **kwargs: None
try:
pipe_cm = YuE2Pipeline.from_pretrained(model, vae=vae, **pipe_load)
finally:
torch.cuda.set_per_process_memory_fraction = _set_fraction
with pipe_cm as pipe:
memory_snapshot(torch, 'memory-loaded')
emit(stage='plan', message='Planning melody and chords', progress=5, cot=cot)
try:
plan = pipe.plan(**pipe_kwargs)
except Exception as error:
if cot != 'full' or not is_cuda_oom(error):
raise
free_cuda(torch)
cot = 'melody'
pipe_kwargs['cot'] = cot
emit(stage='plan', message='Full CoT OOM; retrying melody CoT', progress=5, cot=cot,
error=str(error))
memory_snapshot(torch, 'memory-before-melody-plan')
plan = pipe.plan(**pipe_kwargs)
free_cuda(torch)
memory_snapshot(torch, 'memory-after-plan')
emit(stage='semantic', message='Generating semantic tokens', progress=25, cot=cot,
maxTokens=max_tokens, targetSeconds=duration)
semantic = pipe.generate_semantic(plan, sampling=semantic_sampling)
free_cuda(torch)
memory_snapshot(torch, 'memory-after-semantic')
emit(stage='synthesize', message='Synthesizing acoustic latents', progress=55, cot=cot)
latents = pipe.synthesize(semantic)
free_cuda(torch)
memory_snapshot(torch, 'memory-after-synthesize')
emit(stage='decode', message='Decoding audio (tiled)', progress=80, cot=cot)
# full=False uses decode_tiled with vae_core_frames / halo; model offloads to CPU after.
audio = pipe.decode(latents, full=False)
free_cuda(torch)
memory_snapshot(torch, 'memory-after-decode')
free_cuda(torch)
wave = audio
if hasattr(audio, 'detach'):
wave = audio.detach().cpu().numpy()
import numpy as np
wave = np.asarray(wave)
if wave.ndim == 1:
pass
elif wave.shape[0] <= 8 and wave.shape[0] < wave.shape[-1]:
wave = wave.T
sample_rate = 48000
target = output / 'audio.wav'
sf.write(str(target), wave, sample_rate, subtype='PCM_16')
info = sf.info(str(target))
if info.frames <= 0:
raise RuntimeError('YuE2 produced empty audio.')
emit(stage='complete', message='Audio ready', progress=100, duration=info.duration, cot=cot)
if __name__ == '__main__':
try:
main()
except Exception as error:
emit(stage='error', message=str(error), error=str(error))
raise
+20 -58
View File
@@ -1,15 +1,7 @@
import { yueGpConfigured } from '~/server/utils/yueGp' import { yue2Configured } from '~/server/utils/yue2'
import { addStudioJob, kickStudioQueue, listStudioJobs, videoJobsBusy } from '~/server/utils/studioQueue' import { addStudioJob, kickStudioQueue, listStudioJobs, videoJobsBusy } from '~/server/utils/studioQueue'
import { comfyConfigured } from '~/server/utils/comfy'
import { import {
clampLyricsStrength,
clampMusicCfg,
clampMusicDuration, clampMusicDuration,
clampMusicSteps,
MUSIC_CFG_DEFAULT,
MUSIC_CFG_DEFAULT_15,
MUSIC_STEPS_DEFAULT,
MUSIC_STEPS_DEFAULT_15,
parseMusicEngine, parseMusicEngine,
parseMusicSeed, parseMusicSeed,
sanitizeMusicLyrics, sanitizeMusicLyrics,
@@ -22,36 +14,28 @@ export default defineEventHandler(async (event) => {
tags?: unknown tags?: unknown
lyrics?: unknown lyrics?: unknown
duration?: unknown duration?: unknown
steps?: unknown
seed?: unknown seed?: unknown
cfg?: unknown
lyricsStrength?: unknown
instrumental?: unknown
folderId?: unknown folderId?: unknown
name?: unknown name?: unknown
yueProfile?: unknown
engine?: unknown engine?: unknown
sampler_name?: unknown
scheduler?: unknown
}>(event).catch(() => ({} as Record<string, unknown>)) }>(event).catch(() => ({} as Record<string, unknown>))
const engine = parseMusicEngine(body.engine) const engine = parseMusicEngine(body.engine)
if (engine === 'yue' && !yueGpConfigured()) throw createError({ statusCode: 503, statusMessage: 'YuEGP host is not configured. Set COMFY_CONTROL_URL.' }) if (engine !== 'yue2') {
if (engine !== 'yue' && !comfyConfigured()) { throw createError({ statusCode: 400, statusMessage: 'Only YuE2 is supported for music generation.' })
throw createError({ }
statusCode: 503, if (!yue2Configured()) {
statusMessage: 'Beast ComfyUI is not configured. Set COMFY_HOST.' throw createError({ statusCode: 503, statusMessage: 'YuE2 host is not configured. Set COMFY_CONTROL_URL.' })
})
} }
const instrumental = body.instrumental === true || body.instrumental === 'true'
const tags = sanitizeMusicTags(body.tags) const tags = sanitizeMusicTags(body.tags)
const lyrics = sanitizeMusicLyrics(body.lyrics, instrumental) const lyrics = sanitizeMusicLyrics(body.lyrics, false)
if (!tags) { if (!tags) {
throw createError({ statusCode: 400, statusMessage: 'Genre and style tags are required' }) throw createError({ statusCode: 400, statusMessage: 'Genre and style tags are required' })
} }
if (!instrumental && !lyrics) { const lyricsProblem = yueLyricsProblem(lyrics)
throw createError({ statusCode: 400, statusMessage: 'Lyrics are required unless Instrumental is on' }) if (lyricsProblem) {
throw createError({ statusCode: 400, statusMessage: lyricsProblem })
} }
const ownerKey = libraryOwnerKey(event) const ownerKey = libraryOwnerKey(event)
@@ -66,31 +50,12 @@ export default defineEventHandler(async (event) => {
const destFolder = library.folders.find(folder => folder.id === folderId) const destFolder = library.folders.find(folder => folder.id === folderId)
const folderLocked = Boolean(destFolder?.protected && !destFolder.unlocked) const folderLocked = Boolean(destFolder?.protected && !destFolder.unlocked)
if (engine === 'yue' && instrumental) throw createError({ statusCode: 400, statusMessage: 'YuEGP currently requires lyrics. Use ACE for instrumental music.' })
const yueProfile = body.yueProfile ?? 1
if (engine === 'yue' && yueProfile !== 1 && yueProfile !== 3) throw createError({ statusCode: 400, statusMessage: 'Choose YuEGP profile 1 or manual fallback 3.' })
if (engine === 'yue' && !instrumental) {
const problem = yueLyricsProblem(lyrics)
if (problem) throw createError({ statusCode: 400, statusMessage: problem })
}
const duration = clampMusicDuration(body.duration) const duration = clampMusicDuration(body.duration)
if (engine === 'yue' && duration > 150) throw createError({ statusCode: 400, statusMessage: 'YuEGP supports up to 150 seconds per section.' })
const steps = clampMusicSteps(
body.steps,
engine === 'ace-step-1.5' ? MUSIC_STEPS_DEFAULT_15 : MUSIC_STEPS_DEFAULT
)
const cfg = clampMusicCfg(
body.cfg,
engine === 'ace-step-1.5' ? MUSIC_CFG_DEFAULT_15 : MUSIC_CFG_DEFAULT
)
const lyricsStrength = clampLyricsStrength(body.lyricsStrength)
const seed = parseMusicSeed(body.seed) const seed = parseMusicSeed(body.seed)
const clipName = allocateOutputName(ownerKey, folderId, String(body.name || '').trim(), { const clipName = allocateOutputName(ownerKey, folderId, String(body.name || '').trim(), {
engine: engine === 'ace-step-1.5' ? 'ace15' : engine === 'yue' ? 'yue' : 'ace' engine: 'yue2'
}) })
const samplerName = String(body.sampler_name || 'euler')
const scheduler = String(body.scheduler || 'simple')
const familyId = crypto.randomUUID() const familyId = crypto.randomUUID()
const studio = await addStudioJob({ const studio = await addStudioJob({
ownerKey, ownerKey,
@@ -103,15 +68,15 @@ export default defineEventHandler(async (event) => {
aspect: 'audio', aspect: 'audio',
width: 0, width: 0,
height: 0, height: 0,
steps, steps: 0,
turbo: false, turbo: false,
seed, seed,
cfg, cfg: 0,
fps: 0, fps: 0,
samplerName, samplerName: 'euler',
scheduler, scheduler: 'simple',
duration, duration,
sound: !instrumental, sound: true,
workflow: 'v1', workflow: 'v1',
useIdentityRefs: false, useIdentityRefs: false,
hideThumbnail: false, hideThumbnail: false,
@@ -120,10 +85,9 @@ export default defineEventHandler(async (event) => {
extensions: [], extensions: [],
queueAutoRun: false, queueAutoRun: false,
lyrics, lyrics,
instrumental, instrumental: false,
lyricsStrength, lyricsStrength: 0,
musicEngine: engine, musicEngine: 'yue2'
...(engine === 'yue' ? { yueProfile: yueProfile as 1 | 3 } : {})
} }
}) })
await kickStudioQueue() await kickStudioQueue()
@@ -140,11 +104,9 @@ export default defineEventHandler(async (event) => {
queued: !liveJobId, queued: !liveJobId,
seed, seed,
duration, duration,
steps,
cfg,
folderId, folderId,
folderLocked, folderLocked,
engine, engine: 'yue2' as const,
name: clipName name: clipName
} }
}) })
+11 -1
View File
@@ -48,11 +48,21 @@ export default defineEventHandler(async () => {
host: comfy.host || '', host: comfy.host || '',
queueRunning: queue.running, queueRunning: queue.running,
queuePending: queue.pending, queuePending: queue.pending,
asleep: Boolean(control?.asleep) && !comfyReachable, asleep: Boolean(control?.asleep) && !comfyReachable && !control?.yue2?.busy && !control?.caption?.busy,
processUp: Boolean(control?.process ?? control?.processUp), processUp: Boolean(control?.process ?? control?.processUp),
lastActivityAt: control?.lastActivityAt || null, lastActivityAt: control?.lastActivityAt || null,
stats: comfy.ok ? comfy.stats : undefined stats: comfy.ok ? comfy.stats : undefined
}, },
yue2: {
configured: Boolean(control?.yue2?.configured),
busy: Boolean(control?.yue2?.busy),
backend: control?.yue2?.backend || 'yue2'
},
caption: {
configured: Boolean(control?.caption?.configured),
busy: Boolean(control?.caption?.busy),
backend: control?.caption?.backend || 'llama.cpp'
},
imageComfy: { imageComfy: {
configured: imageConfigured, configured: imageConfigured,
ok: imageOk, ok: imageOk,
+2 -1
View File
@@ -1,5 +1,6 @@
import { existsSync } from 'node:fs' import { existsSync } from 'node:fs'
import { sendPathWithRange } from '~/server/utils/httpRange' import { sendPathWithRange } from '~/server/utils/httpRange'
import { inlineFilename } from '~/shared/content-disposition.mjs'
export default defineEventHandler((event) => { export default defineEventHandler((event) => {
const { owner } = assertLibraryOwner(event) const { owner } = assertLibraryOwner(event)
@@ -11,7 +12,7 @@ export default defineEventHandler((event) => {
throw createError({ statusCode: 404, statusMessage: 'Video file is missing' }) throw createError({ statusCode: 404, statusMessage: 'Video file is missing' })
} }
return sendPathWithRange(event, path, 'video/mp4', { return sendPathWithRange(event, path, 'video/mp4', {
'Content-Disposition': `inline; filename="${safeDownloadName(clipTitle(clip))}.mp4"`, 'Content-Disposition': inlineFilename(`${safeDownloadName(clipTitle(clip))}.mp4`),
'Cache-Control': 'private, max-age=0, must-revalidate' 'Cache-Control': 'private, max-age=0, must-revalidate'
}) })
}) })
+74
View File
@@ -0,0 +1,74 @@
import { validateCaptionRequest } from '~/shared/studio2/caption.mjs'
import { saveRecord, readRecord } from '../../utils/studio2/store'
import { addStudioJob, kickStudioQueue, type StudioJobPayload } from '../../utils/studioQueue'
import { getStill } from '../../utils/library'
export default defineEventHandler(async (event) => {
const { owner } = assertLibraryOwner(event)
const raw = await readBody(event)
let request: ReturnType<typeof validateCaptionRequest>
try {
request = validateCaptionRequest(raw)
} catch (e: any) {
throw createError({ statusCode: e.statusCode || 400, statusMessage: e.message })
}
assertFolderAccess(event, request.folderId)
let stillId = request.stillId
if (!stillId && request.sourcePath) {
// sourcePath may be a still id when the client already resolved a library asset.
stillId = String(request.sourcePath)
}
const still = getStill(owner, stillId)
assertFolderAccess(event, still.folderId)
request.stillId = still.id
request.imageAId = still.id
request.sourcePath = still.id
const id = crypto.randomUUID()
const record = {
id,
owner,
request,
state: 'waiting',
queuedAt: Date.now(),
familyId: crypto.randomUUID(),
outputs: [],
resultText: '',
purgeResult: 'Not yet saved'
}
saveRecord(record)
const row = await addStudioJob({
ownerKey: owner,
kind: 'caption',
familyId: record.familyId,
payload: {
studio2Id: id,
prompt: request.compiledPrompt,
name: `Describe · ${request.captionStyle}`,
folderId: request.folderId,
stillId: still.id,
captionStyle: request.captionStyle,
aspect: 'auto',
width: 0,
height: 0,
steps: 1,
turbo: true,
seed: 0,
cfg: 0,
fps: 24,
samplerName: 'euler',
scheduler: 'simple',
duration: 0,
sound: false,
workflow: 'minimax',
useIdentityRefs: false,
hideThumbnail: false,
referenceStillIds: [],
extensions: [],
queueAutoRun: true
} as StudioJobPayload
})
saveRecord({ ...readRecord(owner, id), queueId: row.id })
kickStudioQueue()
return { id, captionStyle: request.captionStyle, stillId: still.id }
})
@@ -0,0 +1,15 @@
import { records, saveRecord } from '../../utils/studio2/store';
import { listStudioJobs } from '../../utils/studioQueue';
export default defineEventHandler(event => {
const { owner } = assertLibraryOwner(event), queue=listStudioJobs(owner);
let cleared=0;
for (const r of records(owner)) {
try { assertFolderAccess(event,r.request.folderId); } catch { continue; }
const status=queue.find(q=>q.id===r.queueId)?.status;
if (['complete','failed','cancelled'].includes(r.state) || ['cancelled','error'].includes(status || '')) {
r.dismissed=true; saveRecord(r); cleared++;
}
}
// Retain internal manifests for cleanup retries; remove these records from the visible jobs list.
return {cleared};
});
+4 -2
View File
@@ -1,9 +1,11 @@
import { jobView } from '~/shared/studio2/queue-view.mjs';
import { getJob } from '../../utils/jobs';
import { records } from '../../utils/studio2/store'; import { records } from '../../utils/studio2/store';
import { listStudioJobs } from '../../utils/studioQueue'; import { listStudioJobs } from '../../utils/studioQueue';
export default defineEventHandler(event => { const { owner } = assertLibraryOwner(event); const rows = listStudioJobs(owner); return records(owner).filter(r => { try { export default defineEventHandler(event => { const { owner } = assertLibraryOwner(event); const rows = listStudioJobs(owner); return records(owner).filter(r => !r.dismissed).filter(r => { try {
assertFolderAccess(event, r.request.folderId); assertFolderAccess(event, r.request.folderId);
return true; return true;
} }
catch { catch {
return false; return false;
} }).map(r => { const q = rows.find(q => q.id === r.queueId); return { ...r, state: q?.status === 'cancelled' ? 'cancelled' : q?.status === 'error' ? 'failed' : r.state, error: r.error || q?.lastError }; }); }); } }).map(r => { const q = rows.find(q => q.id === r.queueId); const live=r.liveId?getJob(r.liveId):null; if (['waiting','queued','held'].includes(r.state) && !q && !live) return null; return jobView(r,q,live); }).filter(Boolean); });
+5 -3
View File
@@ -1,4 +1,6 @@
import {filterLoraStackForUse} from '~/utils/loras';
import { validateRequest } from '~/shared/studio2/contracts.mjs'; import { validateRequest } from '~/shared/studio2/contracts.mjs';
import { resolveRequestSize } from '../../utils/studio2/size';
import { parsePostedLoraStack, assertImageV2LoraStack } from '../../utils/loras'; import { parsePostedLoraStack, assertImageV2LoraStack } from '../../utils/loras';
import { saveRecord, readRecord } from '../../utils/studio2/store'; import { saveRecord, readRecord } from '../../utils/studio2/store';
import { addStudioJob, kickStudioQueue, type StudioJobPayload } from '../../utils/studioQueue'; import { addStudioJob, kickStudioQueue, type StudioJobPayload } from '../../utils/studioQueue';
@@ -12,9 +14,8 @@ export default defineEventHandler(async (event) => {
throw createError({ statusCode: e.statusCode || 400, statusMessage: e.message }); throw createError({ statusCode: e.statusCode || 400, statusMessage: e.message });
} }
const video = ['video', 'extend'].includes(request.mode); const video = ['video', 'extend'].includes(request.mode);
if (video && request.engine !== 'minimax')
throw createError({ statusCode: 400, statusMessage: 'Use the existing xAIGen studio for LTX. Studio 2 video currently supports MiniMax.' });
request.settings.loraStack = parsePostedLoraStack(request.settings.loraStack, video ? 'video' : 'image'); request.settings.loraStack = parsePostedLoraStack(request.settings.loraStack, video ? 'video' : 'image');
request.settings.loraStack = filterLoraStackForUse(request.settings.loraStack, request.engine, useRuntimeConfig().public.studio === 'xaigen');
if (!video) if (!video)
assertImageV2LoraStack(request.settings.loraStack, request.engine); assertImageV2LoraStack(request.settings.loraStack, request.engine);
assertFolderAccess(event, request.folderId); assertFolderAccess(event, request.folderId);
@@ -22,10 +23,11 @@ export default defineEventHandler(async (event) => {
assertFolderAccess(event, getStill(owner, id).folderId); assertFolderAccess(event, getStill(owner, id).folderId);
if (request.startClipId) if (request.startClipId)
assertFolderAccess(event, getClip(owner, request.startClipId).folderId); assertFolderAccess(event, getClip(owner, request.startClipId).folderId);
await resolveRequestSize(owner,request);
const id = crypto.randomUUID(), record = { id, owner, request, state: 'waiting', queuedAt: Date.now(), familyId: crypto.randomUUID(), outputs: [], purgeResult: 'Not yet saved' }; const id = crypto.randomUUID(), record = { id, owner, request, state: 'waiting', queuedAt: Date.now(), familyId: crypto.randomUUID(), outputs: [], purgeResult: 'Not yet saved' };
saveRecord(record); saveRecord(record);
const row = await addStudioJob({ ownerKey: owner, kind: ['video', 'extend'].includes(request.mode) ? 'video' : 'edit', familyId: record.familyId, payload: { ...request.settings, studio2Id: id, prompt: request.compiledPrompt, folderId: request.folderId, extensions: [], referenceStillIds: [], useIdentityRefs: false, queueAutoRun: true } as StudioJobPayload }); const row = await addStudioJob({ ownerKey: owner, kind: ['video', 'extend'].includes(request.mode) ? 'video' : 'edit', familyId: record.familyId, payload: { ...request.settings, studio2Id: id, prompt: request.compiledPrompt, folderId: request.folderId, extensions: [], referenceStillIds: [], useIdentityRefs: false, queueAutoRun: true } as StudioJobPayload });
saveRecord({ ...readRecord(owner, id), queueId: row.id }); saveRecord({ ...readRecord(owner, id), queueId: row.id });
kickStudioQueue(); kickStudioQueue();
return { id }; return { id, seed:request.settings.seed, width:request.settings.width, height:request.settings.height };
}); });
+9
View File
@@ -0,0 +1,9 @@
import { existsSync } from 'node:fs';
import { studio2LoraCatalog } from '../../utils/studio2/lora-catalog';
import { publicLibrary, stillPath, clipVideoPath } from '../../utils/library';
export default defineEventHandler(event => {
const { owner } = assertLibraryOwner(event);
const library = publicLibrary(event);
const catalog=studio2LoraCatalog(owner);
return {...library,loraAliases:catalog.aliases,loraTriggers:catalog.triggers, stills:library.stills.filter(s=>existsSync(stillPath(owner,s.id))), clips:library.clips.filter(c=>existsSync(clipVideoPath(owner,c.id))), tracks:library.tracks || []};
});
+19
View File
@@ -0,0 +1,19 @@
import { readFileSync,writeFileSync,renameSync } from 'node:fs'
import { join } from 'node:path'
import { cachedLoras } from '~/shared/studio2/lora-cache.mjs'
import { listStudioLoras } from '../../utils/loras'
import { studio2Root } from '../../utils/studio2/store'
import { studio2LoraCatalog } from '../../utils/studio2/lora-catalog'
export default defineEventHandler(async event => {
const { owner } = assertLibraryOwner(event)
const path = join(studio2Root(), `loras-${owner}.cache`)
// Same live Comfy scan as Settings (/api/loras?fresh=1). Always fresh so CACHE_MS cannot freeze adds.
const listed = await cachedLoras({
read: () => { try { return JSON.parse(readFileSync(path, 'utf8')) } catch { return null } },
write: value => { writeFileSync(path + '.tmp', JSON.stringify(value)); renameSync(path + '.tmp', path) },
live: () => listStudioLoras({ fresh: true }),
fresh: true
})
return { ...listed, ...studio2LoraCatalog(owner) }
})
+15
View File
@@ -0,0 +1,15 @@
import { imageDimensions } from '../../utils/resolution'
export default defineEventHandler(async event => {
const parts=await readMultipartFormData(event)
const image=parts?.find(p=>p.name==='image' && p.filename && p.data?.length)
const field=(name:string)=>parts?.find(p=>p.name===name)?.data.toString('utf8') || ''
if (!image) throw createError({statusCode:400,statusMessage:'Choose an image to upload.'})
const {owner}=assertLibraryOwner(event),folderId=field('folderId')
assertFolderAccess(event,folderId)
const size=imageDimensions(image.data)
if (!size) throw createError({statusCode:400,statusMessage:'Use a PNG, JPEG, WebP, GIF, or BMP image.'})
const filename=image.filename!.split(/[\\/]/).at(-1)!
const still=await saveStill({ownerKey:owner,folderId,filename,name:filename.replace(/\.[^.]+$/,''),data:image.data,...size,hideInput:field('hideInput')==='true',role:'input'})
if (!still) throw createError({statusCode:400,statusMessage:'This file was recognized as an internal video handoff frame and was not added. Choose a regular source image.'})
return {still,...publicLibrary(event,{light:true})}
})
+18
View File
@@ -0,0 +1,18 @@
import {upscaleRecords} from '../../utils/videoUpscale';
import {listStudioJobs} from '../../utils/studioQueue';
import {getClip} from '../../utils/library';
export default defineEventHandler(event=>{
const {owner}=assertLibraryOwner(event),records=upscaleRecords(owner);
const rows=listStudioJobs(owner).filter(j=>j.payload.upscale);
const ids=new Set([...rows.map(r=>r.id),...records.map(r=>r.id)]);
return [...ids].flatMap(id=>{
const row=rows.find(r=>r.id===id),record=records.find(r=>r.id===id);
try {
const clip=getClip(owner,row?.payload.upscale?.sourceId || record?.sourceId);
assertFolderAccess(event,clip.folderId);
const state=['waiting','held','running'].includes(row?.status || '')?row!.status:record?.status || row?.status;
return [{id,folderId:clip.folderId,name:clip.name || 'Video',status:({waiting:'queued',held:'queued',complete:'done',error:'failed',cancelled:'failed'} as any)[state || ''] || state,
progress:record?.progress || 0,width:record?.width,height:record?.height,elapsedMs:record?.elapsedMs || (record?.createdAt?Date.now()-record.createdAt:0),message:record?.message || row?.waitReason,error:record?.error || row?.lastError,outputClipId:record?.outputClipId}];
}catch{return [];}
});
});
+9 -7
View File
@@ -50,22 +50,24 @@ export default defineEventHandler(async (event) => {
payload.prompt = text(body.tags, 2000) payload.prompt = text(body.tags, 2000)
row.prompt = payload.prompt row.prompt = payload.prompt
} }
if (body.yueProfile != null && row.kind === 'music' && payload.musicEngine === 'yue') {
if (body.yueProfile !== 1 && body.yueProfile !== 3) throw createError({ statusCode: 400, statusMessage: 'Choose YuEGP profile 1 or 3.' })
payload.yueProfile = body.yueProfile
}
if (body.lyrics != null) payload.lyrics = text(body.lyrics) if (body.lyrics != null) payload.lyrics = text(body.lyrics)
const instrumental = optionalBool(body.instrumental) const instrumental = optionalBool(body.instrumental)
if (instrumental != null) payload.instrumental = instrumental if (instrumental != null && row.kind === 'music') {
if (instrumental) throw createError({ statusCode: 400, statusMessage: 'YuE2 requires lyrics. Instrumental mode is not supported.' })
payload.instrumental = false
} else if (instrumental != null) {
payload.instrumental = instrumental
}
if (body.promptPre != null) payload.promptPre = text(body.promptPre) if (body.promptPre != null) payload.promptPre = text(body.promptPre)
if (body.promptMid != null) payload.promptMid = text(body.promptMid) if (body.promptMid != null) payload.promptMid = text(body.promptMid)
if (body.promptPost != null) payload.promptPost = text(body.promptPost) if (body.promptPost != null) payload.promptPost = text(body.promptPost)
if (body.negative != null) payload.negative = text(body.negative, 2000) if (body.negative != null) payload.negative = text(body.negative, 2000)
const duration = optionalNumber(body.duration) const duration = optionalNumber(body.duration)
if (duration != null) { if (duration != null) {
if (row.kind === 'music' && payload.musicEngine === 'yue') { if (row.kind === 'music') {
if (!Number.isInteger(duration) || duration < 30 || duration > 150) throw createError({ statusCode: 400, statusMessage: 'YuEGP duration must be 30–150 seconds.' }) if (!Number.isInteger(duration) || duration < 15 || duration > 150) throw createError({ statusCode: 400, statusMessage: 'YuE2 target length must be 15–150 seconds.' })
payload.duration = duration payload.duration = duration
payload.musicEngine = 'yue2'
} else payload.duration = Math.min(120, Math.max(0.5, duration)) } else payload.duration = Math.min(120, Math.max(0.5, duration))
} }
const steps = optionalNumber(body.steps) const steps = optionalNumber(body.steps)
+75
View File
@@ -0,0 +1,75 @@
{
"4": {
"inputs": {
"unet_name": "qwen-image-2.1-Q8_0.gguf"
},
"class_type": "UnetLoaderGGUF",
"_meta": { "title": "Load Qwen 2.1 GGUF" }
},
"3": {
"inputs": {
"shift": 3.1,
"model": ["4", 0]
},
"class_type": "ModelSamplingAuraFlow",
"_meta": { "title": "Qwen 2.1 shift" }
},
"5": {
"inputs": {
"clip_name": "qwen3vl_8b_int8_convrot.safetensors",
"type": "qwen_image",
"device": "default"
},
"class_type": "CLIPLoader",
"_meta": { "title": "Load Qwen 2.1 CLIP" }
},
"6": {
"inputs": {
"vae_name": "qwen_image_2.1_vae_bf16.safetensors"
},
"class_type": "VAELoader",
"_meta": { "title": "Load Qwen 2.1 VAE" }
},
"9": {
"inputs": {
"clip": ["5", 0],
"prompt": "",
"negative_prompt": "",
"resolution": 1024
},
"class_type": "TextEncodeQwenImage21",
"_meta": { "title": "Text Encode Qwen Image 2.1" }
},
"15": {
"inputs": {
"seed": 1,
"steps": 25,
"cfg": 1,
"sampler_name": "euler",
"scheduler": "simple",
"denoise": 1,
"model": ["3", 0],
"positive": ["9", 0],
"negative": ["9", 1],
"latent_image": ["9", 2]
},
"class_type": "KSampler",
"_meta": { "title": "KSampler" }
},
"20": {
"inputs": {
"samples": ["15", 0],
"vae": ["6", 0]
},
"class_type": "VAEDecode",
"_meta": { "title": "VAE Decode" }
},
"21": {
"inputs": {
"filename_prefix": "studio2-qwen21",
"images": ["20", 0]
},
"class_type": "SaveImage",
"_meta": { "title": "Save Image" }
}
}
+3
View File
@@ -0,0 +1,3 @@
import { resumeCaptionJobs } from '~/server/utils/caption'
export default defineNitroPlugin(() => { resumeCaptionJobs() })
+2 -1
View File
@@ -1,2 +1,3 @@
import { retryCleanup } from '../utils/studio2/cleanup';
import { resumeStudio2Jobs } from '../utils/studio2/runner'; import { resumeStudio2Jobs } from '../utils/studio2/runner';
export default defineNitroPlugin(() => { resumeStudio2Jobs(); }); export default defineNitroPlugin(() => { resumeStudio2Jobs(); const timer = setInterval(() => { void retryCleanup().catch(error => console.warn('[Studio 2 cleanup retry]', error.message)); }, 30000); timer.unref(); });
+4
View File
@@ -0,0 +1,4 @@
import { resumeYue2Jobs } from '~/server/utils/yue2'
// Restore live music IDs before queue repair examines their durable studio rows.
export default defineNitroPlugin(() => { resumeYue2Jobs() })
+2 -4
View File
@@ -1,4 +1,2 @@
import { resumeYueGpJobs } from '~/server/utils/yueGp' // YuEGP resume removed — music is YuE2-only. scripts/yuegp-* remain on disk unused.
export default defineNitroPlugin(() => {})
// Restore live music IDs before queue repair examines their durable studio rows.
export default defineNitroPlugin(() => { resumeYueGpJobs() })
+268
View File
@@ -0,0 +1,268 @@
import { createReadStream, existsSync, mkdirSync, readdirSync, readFileSync, writeFileSync, renameSync, unlinkSync } from 'node:fs'
import { join } from 'node:path'
import { createJob, emitJob, getJob, restoreJob, type Job } from './jobs'
import { getStill, stillPath } from './library'
import { sharedGpuHeaders } from './sharedGpu'
import { CAPTION_STYLES } from '~/shared/studio2/caption.mjs'
function settings() {
const config = useRuntimeConfig()
return {
url: String(config.comfyControlUrl || process.env.COMFY_CONTROL_URL || '').replace(/\/$/, ''),
token: String(config.comfyControlToken || process.env.COMFY_CONTROL_TOKEN || '')
}
}
export function captionConfigured() { return Boolean(settings().url) }
function pendingRoot() {
return join(String(useRuntimeConfig().libraryDir || process.env.LIBRARY_DIR || '/data/library'), 'caption-pending')
}
export function captionPendingAlive(jobId: string) {
if (!jobId) return false
return existsSync(join(pendingRoot(), `${jobId}.json`))
}
async function request(path: string, method = 'GET', body?: unknown) {
const { url, token } = settings()
if (!url) throw new Error('Caption host is not configured. Set COMFY_CONTROL_URL.')
const response = await fetch(`${url}/caption/${path}`, {
method,
headers: {
...(token ? { Authorization: `Bearer ${token}` } : {}),
...(method === 'GET' ? {} : sharedGpuHeaders()),
...(method === 'POST' ? { 'Content-Type': 'application/json' } : {})
},
body: method === 'POST' ? JSON.stringify(body) : body as BodyInit | undefined,
...(method === 'PUT' ? { duplex: 'half' as const } : {}),
signal: AbortSignal.timeout(method === 'PUT' ? 120_000 : 60_000)
} as RequestInit)
if (!response.ok) {
const detail = await response.json().catch(() => ({})) as { message?: string; error?: string }
throw Object.assign(new Error(detail.message || detail.error || `Caption host returned ${response.status}`), { statusCode: response.status })
}
return response
}
function persist(record: Record<string, unknown>) {
mkdirSync(pendingRoot(), { recursive: true })
const path = join(pendingRoot(), `${record.id}.json`)
writeFileSync(path + '.tmp', JSON.stringify(record))
renameSync(path + '.tmp', path)
}
function clearPending(id: string) {
const path = join(pendingRoot(), `${id}.json`)
if (existsSync(path)) unlinkSync(path)
}
function writeSidecar(owner: string, stillId: string, text: string) {
try {
const image = stillPath(owner, stillId)
if (!existsSync(image)) return
writeFileSync(`${image}.txt`, text, 'utf8')
} catch { /* optional sidecar */ }
}
export async function cancelCaptionJob(job: Job) {
await request(`jobs/${job.id}/cancel`, 'POST', {})
job.status = 'cancelled'
emitJob(job, { type: 'error', error: 'Cancelled', message: 'Cancelled' })
}
async function settle(job: Job) {
const { onLiveVideoSettled } = await import('./studioQueue')
await onLiveVideoSettled(job)
clearPending(job.id)
}
async function watch(job: Job, record: {
id: string
owner: string
stillId: string
folderId: string
captionStyle: string
studio2Id?: string
liveId: string
}) {
let failures = 0
while (job.status !== 'cancelled') {
try {
const state = await (await request(`jobs/${job.id}`)).json() as {
status: string
message?: string
error?: string
progress?: number
text?: string
}
if (job.status === 'cancelled') { await settle(job); return }
if (state.status === 'error' || state.status === 'cancelled') {
job.status = state.status === 'cancelled' ? 'cancelled' : 'error'
job.error = state.error || state.message || 'Caption failed'
emitJob(job, { type: 'error', error: job.error, message: job.error })
if (record.studio2Id) {
const { readRecord, saveRecord } = await import('./studio2/store')
try {
const r = readRecord(record.owner, record.studio2Id)
r.state = job.status === 'cancelled' ? 'cancelled' : 'failed'
r.error = job.error
r.finishedAt = Date.now()
saveRecord(r)
} catch { /* record may already be gone */ }
}
await settle(job)
return
}
if (state.status === 'complete') {
const text = String(state.text || '').trim()
job.message = 'Caption ready'
job.progress = 100
job.status = 'complete'
;(job as Job & { resultText?: string }).resultText = text
if (record.studio2Id) {
const { readRecord, saveRecord } = await import('./studio2/store')
const r = readRecord(record.owner, record.studio2Id)
r.state = 'complete'
r.resultText = text
r.finishedAt = Date.now()
r.savedAt = Date.now()
saveRecord(r)
}
writeSidecar(record.owner, record.stillId, text)
emitJob(job, { type: 'complete', message: 'Caption ready', progress: 100 })
await settle(job)
return
}
failures = 0
job.status = 'running'
emitJob(job, {
type: 'progress',
message: state.message || 'Captioning',
progress: Math.min(95, Math.max(1, Number(state.progress || 10)))
})
} catch (error) {
failures++
emitJob(job, { type: 'status', message: `Caption host check failed; retrying: ${error instanceof Error ? error.message : String(error)}` })
if (failures >= 10) {
job.status = 'error'
job.error = 'Caption host unreachable.'
emitJob(job, { type: 'error', error: job.error, message: job.error })
await settle(job)
return
}
}
await new Promise(resolve => setTimeout(resolve, 1500))
}
await settle(job)
}
export async function startCaptionJob(params: {
ownerKey: string
folderId: string
stillId: string
captionStyle: string
studio2Id?: string
name?: string
}) {
if (!CAPTION_STYLES.includes(params.captionStyle)) throw new Error('Unknown caption style.')
getStill(params.ownerKey, params.stillId)
const job = createJob('caption')
job.caption = true
job.library = {
ownerKey: params.ownerKey,
folderId: params.folderId,
hideThumbnail: false,
name: params.name || `Describe · ${params.captionStyle}`,
prompt: params.captionStyle,
aspect: 'image',
width: 0,
height: 0,
steps: 1,
turbo: true,
seed: 0,
stillId: params.stillId,
engine: 'caption'
}
const record = {
id: job.id,
liveId: job.id,
owner: params.ownerKey,
stillId: params.stillId,
folderId: params.folderId,
captionStyle: params.captionStyle,
studio2Id: params.studio2Id,
createdAt: Date.now(),
status: 'running'
}
persist(record)
void (async () => {
try {
if (job.status === 'cancelled') { await settle(job); return }
await request(`jobs/${job.id}/input`, 'PUT', createReadStream(stillPath(params.ownerKey, params.stillId)))
if (job.status === 'cancelled') { await settle(job); return }
try {
await request('jobs', 'POST', { id: job.id, style: params.captionStyle })
} catch {
/* Confirm by ID; never caption twice on an HTTP timeout. */
}
await watch(job, record)
} catch (error) {
const statusCode = (error as { statusCode?: number })?.statusCode
if (statusCode && statusCode >= 400 && statusCode < 500) {
job.status = 'error'
job.error = error instanceof Error ? error.message : String(error)
emitJob(job, { type: 'error', error: job.error, message: job.error })
if (params.studio2Id) {
try {
const { readRecord, saveRecord } = await import('./studio2/store')
const r = readRecord(params.ownerKey, params.studio2Id)
r.state = 'failed'
r.error = job.error
r.finishedAt = Date.now()
saveRecord(r)
} catch { /* ignore */ }
}
await settle(job)
return
}
emitJob(job, { type: 'status', message: `Checking caption submission: ${error instanceof Error ? error.message : String(error)}` })
await watch(job, record)
}
})()
return job
}
export function resumeCaptionJobs() {
if (!existsSync(pendingRoot())) return
for (const file of readdirSync(pendingRoot()).filter(file => /^[a-zA-Z0-9-]+\.json$/.test(file))) {
try {
const record = JSON.parse(readFileSync(join(pendingRoot(), file), 'utf8'))
if (getJob(record.id)) continue
const job = restoreJob({
id: record.liveId || record.id,
clientId: record.clientId || crypto.randomUUID(),
promptId: '',
startedAt: record.createdAt || Date.now(),
library: {
ownerKey: record.owner,
folderId: record.folderId,
hideThumbnail: false,
prompt: record.captionStyle || 'descriptive',
aspect: 'image',
width: 0,
height: 0,
steps: 1,
turbo: true,
seed: 0,
stillId: record.stillId,
engine: 'caption'
}
})
job.caption = true
job.kind = 'caption'
job.message = 'Reconnecting to caption host'
void watch(job, record)
} catch { /* Preserve invalid records for diagnosis. */ }
}
}
+2 -7
View File
@@ -166,17 +166,12 @@ function sanitizeImageV2Settings(raw: unknown): ImageV2PresetSettings {
function sanitizeMusicSettings(raw: unknown): MusicPresetSettings { function sanitizeMusicSettings(raw: unknown): MusicPresetSettings {
const rec = raw && typeof raw === 'object' ? raw as Record<string, unknown> : {} const rec = raw && typeof raw === 'object' ? raw as Record<string, unknown> : {}
const instrumental = rec.instrumental === true
return { return {
tags: sanitizeMusicTags(rec.tags), tags: sanitizeMusicTags(rec.tags),
lyrics: sanitizeMusicLyrics(rec.lyrics, instrumental), lyrics: sanitizeMusicLyrics(rec.lyrics, false),
instrumental,
duration: clampMusicDuration(rec.duration), duration: clampMusicDuration(rec.duration),
steps: clampMusicSteps(rec.steps),
seed: typeof rec.seed === 'number' && Number.isFinite(rec.seed) ? Math.max(0, Math.floor(rec.seed)) : undefined, seed: typeof rec.seed === 'number' && Number.isFinite(rec.seed) ? Math.max(0, Math.floor(rec.seed)) : undefined,
cfg: rec.cfg == null ? undefined : clampMusicCfg(rec.cfg), engine: 'yue2'
engine: parseMusicEngine(rec.engine),
...(parseMusicEngine(rec.engine) === 'yue' ? { yueProfile: rec.yueProfile === 3 ? 3 as const : 1 as const } : {})
} }
} }
+4 -1
View File
@@ -35,9 +35,12 @@ export interface Job {
studio2?: boolean studio2?: boolean
upscale?: boolean upscale?: boolean
yueGp?: boolean yueGp?: boolean
yue2?: boolean
caption?: boolean
resultText?: string
musicActivity?: { checkedAt: number; running: boolean } musicActivity?: { checkedAt: number; running: boolean }
id: string id: string
kind?: 'video' | 'edit' | 'music' kind?: 'video' | 'edit' | 'music' | 'caption'
promptId?: string promptId?: string
clientId: string clientId: string
status: JobStatus status: JobStatus
+4
View File
@@ -1788,6 +1788,10 @@ export function findYueGpTrack(owner: string, jobId: string) {
return readCatalog(owner).tracks.find(track => track.comfyFilename === `yuegp-${jobId}.wav`) return readCatalog(owner).tracks.find(track => track.comfyFilename === `yuegp-${jobId}.wav`)
} }
export function findYue2Track(owner: string, jobId: string) {
return readCatalog(owner).tracks.find(track => track.comfyFilename === `yue2-${jobId}.wav`)
}
export async function saveTrack(params: { export async function saveTrack(params: {
ownerKey: string ownerKey: string
folderId: string folderId: string
+57 -21
View File
@@ -44,6 +44,7 @@ type LoraCache = {
let cache: LoraCache | null = null let cache: LoraCache | null = null
let inflight: Promise<LoraCache> | null = null let inflight: Promise<LoraCache> | null = null
let inflightFresh = false let inflightFresh = false
let loadSeq = 0
function uniqueNames(values: unknown[]) { function uniqueNames(values: unknown[]) {
const seen = new Set<string>() const seen = new Set<string>()
@@ -80,14 +81,44 @@ function namesFromObjectInfoNode(info: unknown) {
return uniqueNames(names) return uniqueNames(names)
} }
async function fetchJson(path: string, via: 'video' | 'image') { async function fetchJson(path: string, via: 'video' | 'image', timeoutMs = 8000) {
const res = via === 'image' const res = via === 'image'
? await imageComfyFetch(path, { signal: AbortSignal.timeout(8000) }) ? await imageComfyFetch(path, { signal: AbortSignal.timeout(timeoutMs) })
: await comfyFetch(path, { signal: AbortSignal.timeout(8000) }) : await comfyFetch(path, { signal: AbortSignal.timeout(timeoutMs) })
if (!res.ok) return null if (!res.ok) return null
return res.json().catch(() => null) return res.json().catch(() => null)
} }
function collectObjectInfoLoras(all: Record<string, unknown>) {
const image: string[] = []
const video: string[] = []
let any = false
for (const node of OBJECT_INFO_NODES) {
const info = all[node]
if (!info || typeof info !== 'object') continue
const names = namesFromObjectInfoNode(info)
if (!names.length) continue
any = true
if (node === LORA_LOADER) image.push(...names)
else video.push(...names)
if (node !== LORA_LOADER && node !== LORA_MODEL_ONLY) {
image.push(...names)
video.push(...names)
}
}
if (any) return { image: uniqueNames(image), video: uniqueNames(video) }
const imageAll: string[] = []
const videoAll: string[] = []
for (const [classType, info] of Object.entries(all)) {
if (!/lora/i.test(classType)) continue
const names = namesFromObjectInfoNode(info)
if (classType === LORA_LOADER) imageAll.push(...names)
else videoAll.push(...names)
}
return { image: uniqueNames(imageAll), video: uniqueNames(videoAll) }
}
async function fetchModelsLoras(via: 'video' | 'image') { async function fetchModelsLoras(via: 'video' | 'image') {
const payload = await fetchJson('/models/loras', via) const payload = await fetchJson('/models/loras', via)
if (Array.isArray(payload)) return uniqueNames(payload) if (Array.isArray(payload)) return uniqueNames(payload)
@@ -97,7 +128,14 @@ async function fetchModelsLoras(via: 'video' | 'image') {
return [] as string[] return [] as string[]
} }
async function fetchObjectInfoLoras(via: 'video' | 'image') { async function fetchObjectInfoLoras(via: 'video' | 'image', fresh = false) {
// Full /object_info is Comfy's model-list refresh (UI Refresh) and lists LoraLoader / model-only / rgthree / WanVideoLoraSelect.
if (fresh) {
const all = await fetchJson('/object_info', via, 45_000)
if (all && typeof all === 'object') return collectObjectInfoLoras(all as Record<string, unknown>)
return { image: [] as string[], video: [] as string[] }
}
const image: string[] = [] const image: string[] = []
const video: string[] = [] const video: string[] = []
let any = false let any = false
@@ -120,22 +158,14 @@ async function fetchObjectInfoLoras(via: 'video' | 'image') {
return { image: uniqueNames(image), video: uniqueNames(video) } return { image: uniqueNames(image), video: uniqueNames(video) }
} }
const all = await fetchJson('/object_info', via) const all = await fetchJson('/object_info', via, 45_000)
if (!all || typeof all !== 'object') return { image: [] as string[], video: [] as string[] } if (!all || typeof all !== 'object') return { image: [] as string[], video: [] as string[] }
const imageAll: string[] = [] return collectObjectInfoLoras(all as Record<string, unknown>)
const videoAll: string[] = []
for (const [classType, info] of Object.entries(all as Record<string, unknown>)) {
if (!/lora/i.test(classType)) continue
const names = namesFromObjectInfoNode(info)
if (classType === LORA_LOADER) imageAll.push(...names)
else videoAll.push(...names)
}
return { image: uniqueNames(imageAll), video: uniqueNames(videoAll) }
} }
async function discoverFromHost(via: 'video' | 'image') { async function discoverFromHost(via: 'video' | 'image', fresh = false) {
const fromInfo = await fetchObjectInfoLoras(via, fresh).catch(() => ({ image: [] as string[], video: [] as string[] }))
const models = await fetchModelsLoras(via).catch(() => [] as string[]) const models = await fetchModelsLoras(via).catch(() => [] as string[])
const fromInfo = await fetchObjectInfoLoras(via).catch(() => ({ image: [] as string[], video: [] as string[] }))
const image = uniqueNames([...fromInfo.image, ...models]) const image = uniqueNames([...fromInfo.image, ...models])
const video = uniqueNames([...fromInfo.video, ...models]) const video = uniqueNames([...fromInfo.video, ...models])
if (!image.length && video.length) return { image: video, video } if (!image.length && video.length) return { image: video, video }
@@ -147,17 +177,19 @@ async function loadLoraCache(options: { fresh?: boolean } = {}): Promise<LoraCac
const fresh = options.fresh === true const fresh = options.fresh === true
const now = Date.now() const now = Date.now()
if (!fresh && cache && now - cache.at < CACHE_MS) return cache if (!fresh && cache && now - cache.at < CACHE_MS) return cache
// Fresh must not reuse a non-fresh inflight (stale list / frozen CACHE_MS window).
if (inflight && (!fresh || inflightFresh)) return inflight if (inflight && (!fresh || inflightFresh)) return inflight
const seq = ++loadSeq
inflightFresh = fresh inflightFresh = fresh
inflight = (async () => { inflight = (async () => {
const videoHost = comfyConfigured() ? getComfyHost() : '' const videoHost = comfyConfigured() ? getComfyHost() : ''
const imageHost = getBeastImageHost() const imageHost = getBeastImageHost()
const same = Boolean(videoHost && imageHost && sameImageHost(videoHost, imageHost)) const same = Boolean(videoHost && imageHost && sameImageHost(videoHost, imageHost))
const video = videoHost const video = videoHost
? await discoverFromHost('video').catch(() => ({ image: [] as string[], video: [] as string[] })) ? await discoverFromHost('video', fresh).catch(() => ({ image: [] as string[], video: [] as string[] }))
: { image: [] as string[], video: [] as string[] } : { image: [] as string[], video: [] as string[] }
const image = imageHost && !same const image = imageHost && !same
? await discoverFromHost('image').catch(() => ({ image: [] as string[], video: [] as string[] })) ? await discoverFromHost('image', fresh).catch(() => ({ image: [] as string[], video: [] as string[] }))
: video : video
const next: LoraCache = { const next: LoraCache = {
at: Date.now(), at: Date.now(),
@@ -166,15 +198,19 @@ async function loadLoraCache(options: { fresh?: boolean } = {}): Promise<LoraCac
} }
// Comfy asleep / unreachable returns empty. Keep the last good list so the picker // Comfy asleep / unreachable returns empty. Keep the last good list so the picker
// does not vanish — but do NOT refresh `at`, or a blip freezes new folder drops out. // does not vanish — but do NOT refresh `at`, or a blip freezes new folder drops out.
// Never merge a shorter fresh list back into the old one: deletes must disappear. // Never merge a shorter live list into the old one: deletes must disappear; adds must appear.
if (!next.image.length && !next.video.length && cache && (cache.image.length || cache.video.length)) { if (!next.image.length && !next.video.length && cache && (cache.image.length || cache.video.length)) {
return cache return cache
} }
// Superseded non-fresh must not overwrite a newer fresh result.
if (seq !== loadSeq) return cache || next
cache = next cache = next
return next return next
})().finally(() => { })().finally(() => {
inflight = null if (seq === loadSeq) {
inflightFresh = false inflight = null
inflightFresh = false
}
}) })
return inflight return inflight
} }
+11 -409
View File
@@ -1,12 +1,4 @@
import { startYueGpJob } from './yueGp' import { startYue2Job } from './yue2'
import { createJob, emitJob, type Job } from '~/server/utils/jobs'
import { extractAudio, fetchHistory, fetchHistoryAll, findHistoryAudio, purgeComfyArtifacts, queuePrompt } from '~/server/utils/comfy'
import { comfyWsUrl, comfyFetch } from '~/server/utils/comfy'
import { ensureComfyReady } from '~/server/utils/comfyLifecycle'
import { downloadComfyAudio, saveTrack } from '~/server/utils/library'
import { buildMusicWorkflow, musicFilenamePrefix, assertMusicEngineNodes } from '~/server/utils/musicWorkflow'
import { NODE_LABELS } from '~/server/utils/workflow'
import { musicExtension } from '~/utils/music'
import type { MusicEngine } from '~/utils/music' import type { MusicEngine } from '~/utils/music'
export type MusicJobParams = { export type MusicJobParams = {
@@ -22,412 +14,22 @@ export type MusicJobParams = {
lyricsStrength: number lyricsStrength: number
instrumental: boolean instrumental: boolean
folderLocked?: boolean folderLocked?: boolean
yueProfile?: 1 | 3
engine?: MusicEngine engine?: MusicEngine
samplerName?: string samplerName?: string
scheduler?: string scheduler?: string
} }
function sleep(ms: number) { /** Music generation is YuE2-only. ACE Comfy and YuEGP paths are removed. */
return new Promise(resolve => setTimeout(resolve, ms)) export async function startMusicJob(params: MusicJobParams) {
} if (params.instrumental) {
throw new Error('YuE2 requires lyrics. Instrumental mode is not supported.')
function classifyError(message: string) {
const lower = message.toLowerCase()
if (lower.includes('out of memory') || lower.includes('exceed allowed memory') || (lower.includes('cuda') && lower.includes('alloc')) || lower.includes('vram')) {
return 'ComfyUI ran out of VRAM.'
} }
if (lower.includes('econnrefused') || lower.includes('unreachable') || lower.includes('fetch failed')) { if (!String(params.lyrics || '').trim()) {
return 'ComfyUI host connection dropped. Confirm the desktop instance is running.' throw new Error('YuE2 needs lyrics.')
} }
if (lower.includes('timeout')) { return startYue2Job({
return 'Network timeout talking to ComfyUI. The job may still be running on the desktop.' ...params,
} engine: 'yue2',
return message instrumental: false
}
async function settleMusicStudio(job: Job) {
try {
const { onLiveVideoSettled } = await import('~/server/utils/studioQueue')
await onLiveVideoSettled(job)
} catch {
/* queue kick is best-effort */
}
}
function nodeLabel(node: string) {
if (!node) return ''
return NODE_LABELS[node] || ''
}
async function waitForSocket(job: Job, ms = 4000) {
const started = Date.now()
while (Date.now() - started < ms) {
if (job.socketReady) return
await sleep(100)
}
}
function watchMusicJob(job: Job): Promise<void> {
job.socketReady = false
job.saving = false
const ws = new WebSocket(comfyWsUrl(job.clientId))
let settled = false
let finishing = false
let pollTimer: ReturnType<typeof setInterval> | null = null
let timeout: ReturnType<typeof setTimeout> | null = null
return new Promise((resolve) => {
let localProgress = 0
const startedAt = Date.now()
let lastActivity = Date.now()
const IDLE_MS = 30 * 60 * 1000
const ABSOLUTE_MS = 3 * 60 * 60 * 1000
const markActivity = () => { lastActivity = Date.now() }
const emitLocal = (event: Parameters<typeof emitJob>[1], samplePct?: number) => {
markActivity()
if (typeof event.progress === 'number') localProgress = event.progress
if (event.message && typeof samplePct === 'number') {
event.message = `${event.message} (${samplePct}%)`
}
emitJob(job, event)
}
const cleanup = () => {
if (timeout) clearTimeout(timeout)
if (pollTimer) clearInterval(pollTimer)
timeout = null
pollTimer = null
try { ws.close() } catch { /* ignore */ }
}
const armIdle = () => {
if (timeout) clearTimeout(timeout)
if (settled) return
const absLeft = ABSOLUTE_MS - (Date.now() - startedAt)
if (absLeft <= 0) {
void fail('Timed out waiting for ComfyUI (3 hours).')
return
}
const idleLeft = IDLE_MS - (Date.now() - lastActivity)
timeout = setTimeout(() => {
if (settled || finishing) return
if (Date.now() - lastActivity >= IDLE_MS) {
void fail('Timed out waiting for ComfyUI progress (30 minutes with no updates).')
return
}
armIdle()
}, Math.max(1000, Math.min(idleLeft, absLeft)))
}
const fail = async (error: string) => {
if (settled || finishing) return
settled = true
cleanup()
job.status = job.status === 'cancelled' ? 'cancelled' : 'error'
job.error = classifyError(error)
emitJob(job, { type: 'error', error: job.error, message: job.error })
await settleMusicStudio(job)
resolve()
}
const adoptPromptId = (id: unknown) => {
const promptId = String(id || '').trim()
if (promptId && !job.promptId) job.promptId = promptId
}
const loadJobAudio = async () => {
if (job.promptId) {
const scoped = await fetchHistory(job.promptId)
const fromPrompt = extractAudio(scoped, job.promptId)
if (fromPrompt) return fromPrompt
}
const found = findHistoryAudio(await fetchHistoryAll(), {
promptId: job.promptId,
tags: job.library?.tags,
since: job.startedAt - 60_000
})
if (found) {
job.promptId = found.promptId
return found.audio
}
return null
}
const succeed = async () => {
if (settled || finishing) return false
finishing = true
job.saving = true
try {
const audio = await loadJobAudio()
if (!audio) {
finishing = false
job.saving = false
return false
}
if (settled) return false
settled = true
cleanup()
try {
job.audio = audio
emitLocal({ type: 'status', message: 'Saving to library...', progress: 98 })
if (job.library) {
const buffer = await downloadComfyAudio(audio)
const ext = musicExtension(audio.filename)
const track = await saveTrack({
ownerKey: job.library.ownerKey,
folderId: job.library.folderId,
name: job.library.name,
tags: job.library.tags || job.library.prompt,
lyrics: job.library.lyrics || '',
duration: job.library.duration || 0,
steps: job.library.steps,
seed: job.library.seed,
cfg: job.library.cfg || 0,
instrumental: job.library.instrumental === true,
audio: buffer,
ext,
engine: job.library.engine,
samplerName: job.library.samplerName,
scheduler: job.library.scheduler,
lyricsStrength: job.library.lyricsStrength,
comfyFilename: audio.filename
})
job.trackId = track.id
job.library.audioExt = ext
await purgeComfyArtifacts({
audio,
promptId: job.promptId
})
}
job.status = 'complete'
emitJob(job, {
type: 'complete',
message: job.library?.folderLocked
? 'Saved to the locked folder. Unlock it to listen.'
: 'Track ready',
progress: 100,
filename: job.library?.folderLocked ? undefined : audio.filename,
subfolder: job.library?.folderLocked ? undefined : audio.subfolder,
mediaType: job.library?.folderLocked ? undefined : audio.type,
trackId: job.trackId,
folderLocked: job.library?.folderLocked
})
} catch (saveError) {
const message = saveError instanceof Error ? saveError.message : String(saveError)
job.status = 'error'
job.error = `Audio generated but library save failed: ${message}`
emitJob(job, { type: 'error', error: job.error, message: job.error })
}
await settleMusicStudio(job)
job.saving = false
resolve()
return true
} catch (error) {
finishing = false
job.saving = false
throw error
}
}
const pollHistory = async () => {
if (settled || finishing) return
if (job.promptId) {
try {
const response = await comfyFetch('/queue', { signal: AbortSignal.timeout(2500) })
if (response.ok) {
const queue = await response.json() as { queue_running?: unknown[][] }
job.musicActivity = {
checkedAt: Date.now(),
running: Boolean(queue.queue_running?.some(row => row[1] === job.promptId))
}
}
} catch { /* Keep the last confirmation timestamp so the UI can show stale checks. */ }
}
if (settled || finishing) return
try {
const history = await fetchHistoryAll()
const found = findHistoryAudio(history, {
promptId: job.promptId,
tags: job.library?.tags,
since: job.startedAt - 60_000
})
if (found) {
job.promptId = found.promptId
await succeed()
return
}
const entry = (job.promptId ? history?.[job.promptId] : undefined) as {
status?: {
status_str?: string
completed?: boolean
messages?: Array<[string, Record<string, unknown>]>
}
} | undefined
const status = entry?.status?.status_str
if (status === 'error') {
const err = entry?.status?.messages?.find(([type]) => type === 'execution_error')?.[1]
const detail = String(err?.exception_message || err?.message || '').trim()
await fail(detail || 'ComfyUI reported an execution error')
} else if (status === 'interrupted') await fail('Job interrupted.')
} catch {
// history can miss while the graph is still running
}
}
armIdle()
pollTimer = setInterval(() => { void pollHistory() }, 4000)
ws.addEventListener('open', () => {
job.socketReady = true
emitLocal({ type: 'status', message: 'Connected to ComfyUI', progress: Math.max(localProgress, 4) })
})
ws.addEventListener('error', () => {
job.socketReady = false
})
ws.addEventListener('message', (event) => {
let payload: { type?: string; data?: Record<string, any> }
try {
payload = JSON.parse(String(event.data))
} catch {
return
}
const type = String(payload.type || '')
const data = payload.data || {}
adoptPromptId(data.prompt_id)
if (type === 'status') markActivity()
if (type === 'progress') {
const value = Number(data.value || 0)
const max = Number(data.max || 0)
const pct = max > 0 ? Math.min(95, Math.round((value / max) * 90) + 8) : Math.max(localProgress, 12)
const node = String(data.node || '')
emitLocal({
type: 'progress',
message: nodeLabel(node) || 'Sampling',
progress: pct,
step: value,
maxStep: max,
node
}, max > 0 ? Math.round((value / max) * 100) : undefined)
}
if (type === 'executing') {
const node = data.node == null ? '' : String(data.node)
if (!node) void succeed()
else if (node) {
emitLocal({
type: 'status',
message: nodeLabel(node) || 'Running node',
progress: Math.max(localProgress, 10),
node
})
}
}
if (type === 'executed') {
void succeed()
}
if (type === 'execution_error') {
const message = String(data.exception_message || data.message || 'ComfyUI node execution failed')
void fail(message)
}
if (type === 'execution_interrupted') {
job.status = 'cancelled'
void fail('Job interrupted.')
}
})
}) })
} }
export async function startMusicJob(params: MusicJobParams) {
if (params.engine === 'yue') return startYueGpJob(params)
const job = createJob('music')
job.library = {
ownerKey: params.ownerKey,
folderId: params.folderId,
hideThumbnail: false,
folderLocked: params.folderLocked === true,
name: params.name,
prompt: params.tags,
tags: params.tags,
lyrics: params.lyrics,
instrumental: params.instrumental,
aspect: 'audio',
width: 0,
height: 0,
steps: params.steps,
turbo: false,
seed: params.seed,
cfg: params.cfg,
duration: params.duration,
sound: true,
engine: params.engine || 'ace-step',
samplerName: params.samplerName || 'euler',
scheduler: params.scheduler || 'simple',
lyricsStrength: params.lyricsStrength
}
job.maxStep = params.steps
job.status = 'queued'
emitJob(job, { type: 'status', message: 'Checking ComfyUI...', progress: 1 })
void (async () => {
try {
const done = watchMusicJob(job)
await ensureComfyReady((status) => {
emitJob(job, {
type: status.state === 'busy' ? 'busy' : 'status',
message: status.message,
progress: status.state === 'online' ? 3 : 1,
busy: status.state === 'busy',
queueRunning: status.queueRunning,
queuePending: status.queuePending
})
})
job.status = 'running'
const engineLabel = params.engine === 'ace-step-1.5' ? 'ACE-Step 1.5' : 'ACE-Step'
emitJob(job, { type: 'status', message: `Queueing ${engineLabel}...`, progress: 6 })
await waitForSocket(job, 4000)
await assertMusicEngineNodes(params.engine)
const graph = buildMusicWorkflow({
tags: params.tags,
lyrics: params.instrumental ? '' : params.lyrics,
duration: params.duration,
steps: params.steps,
seed: params.seed,
cfg: params.cfg,
lyricsStrength: params.instrumental ? 0 : params.lyricsStrength,
samplerName: params.samplerName,
scheduler: params.scheduler,
filenamePrefix: musicFilenamePrefix(),
engine: params.engine,
vocalMultiplier: params.instrumental ? 0.2 : 1
})
try {
const queued = await queuePrompt(graph, job.clientId, { timeoutMs: 25_000 })
if (queued?.prompt_id) job.promptId = queued.prompt_id
const runningMessage = 'Job queued on ComfyUI'
emitJob(job, { type: 'status', message: runningMessage, progress: 8 })
} catch (queueError: unknown) {
const statusCode = Number((queueError as { statusCode?: number; status?: number })?.statusCode
|| (queueError as { status?: number })?.status || 0)
if (statusCode >= 400) throw queueError
emitJob(job, {
type: 'status',
message: 'Waiting for ComfyUI to pick up the job...',
progress: 8
})
}
await done
} catch (error) {
const message = error instanceof Error ? error.message : String(error)
if (job.status !== 'cancelled' && job.status !== 'complete') {
job.status = 'error'
job.error = classifyError(message)
emitJob(job, { type: 'error', error: job.error, message: job.error })
await settleMusicStudio(job)
}
}
})()
return job
}
+2
View File
@@ -101,6 +101,7 @@ function buildAce15Workflow(params: MusicWorkflowParams): WorkflowGraph {
export function buildMusicWorkflow(params: MusicWorkflowParams): WorkflowGraph { export function buildMusicWorkflow(params: MusicWorkflowParams): WorkflowGraph {
const engine = params.engine || 'ace-step' const engine = params.engine || 'ace-step'
if (engine === 'yue') throw new Error('YuE requires the standalone YuEGP backend.') if (engine === 'yue') throw new Error('YuE requires the standalone YuEGP backend.')
if (engine === 'yue2') throw new Error('YuE2 requires the standalone YuE2 backend.')
if (engine === 'ace-step-1.5') return buildAce15Workflow(params) if (engine === 'ace-step-1.5') return buildAce15Workflow(params)
return buildAceV1Workflow(params) return buildAceV1Workflow(params)
} }
@@ -108,6 +109,7 @@ export function buildMusicWorkflow(params: MusicWorkflowParams): WorkflowGraph {
export async function assertMusicEngineNodes(engine: MusicEngine | undefined) { export async function assertMusicEngineNodes(engine: MusicEngine | undefined) {
const { comfyHasClassType } = await import('~/server/utils/comfy') const { comfyHasClassType } = await import('~/server/utils/comfy')
if (engine === 'yue') throw new Error('YuE cannot run on Comfy.') if (engine === 'yue') throw new Error('YuE cannot run on Comfy.')
if (engine === 'yue2') throw new Error('YuE2 cannot run on Comfy.')
if (engine === 'ace-step-1.5') { if (engine === 'ace-step-1.5') {
const present = await comfyHasClassType('TextEncodeAceStepAudio1.5') const present = await comfyHasClassType('TextEncodeAceStepAudio1.5')
if (present === false) { if (present === false) {
+50
View File
@@ -0,0 +1,50 @@
import { scopedFile } from '~/shared/studio2/contracts.mjs';
import { sharedGpuHeaders, withSharedGpuStart } from '../sharedGpu';
import { records, saveRecord } from './store';
/** Called only after the output and handoff frame are durably saved. */
export async function purge(r: any, includeCurrent = true) {
const config = useRuntimeConfig(), prefix = String(config.comfyFilenamePrefix);
const pending = r.cleanupPending || [];
const files = [...pending, ...(includeCurrent ? r.files || [] : [])].filter((file, i, all) => all.findIndex(f => f.filename === file.filename && f.subfolder === file.subfolder && f.type === file.type) === i);
r.cleanupPending = files;
r.purgeResult = 'Cleanup pending';
saveRecord(r);
if (!files.length) return;
try {
if (files.some(file => !scopedFile(file, prefix))) throw new Error('Cleanup manifest is outside this studio');
for (let offset = 0; offset < files.length; offset += 100) {
const batch = files.slice(offset, offset + 100);
const response = await fetch(String(config.comfyControlUrl).replace(/\/$/, '') + '/studio2/purge', {
method: 'POST', headers: { 'Content-Type': 'application/json', ...sharedGpuHeaders() },
body: JSON.stringify({ prefix, files: batch }), signal: AbortSignal.timeout(12000)
});
const result = await response.json() as any;
if (!response.ok || !result.ok || result.cleared !== batch.length) throw new Error(`Local cleanup was not confirmed (${response.status})`);
r.cleanupPending = files.slice(offset + batch.length);
saveRecord(r);
}
r.purgeResult = 'Cleared input + output';
r.cleanupError = '';
} catch (error: any) {
r.cleanupError = error.message || 'Local cleanup unavailable';
console.warn('[Studio 2 cleanup]', r.id, r.cleanupError);
}
saveRecord(r);
}
let retrying = false;
export async function retryCleanup() {
if (retrying) return;
retrying = true;
try {
const pending = records().filter(r => r.cleanupPending?.length && ['complete', 'failed', 'cancelled'].includes(r.state));
if (!pending.length) return;
await withSharedGpuStart(async () => {
for (const r of pending) {
// Never add a later failed shot's unsaved files to the saved cleanup manifest.
await purge(r, false);
}
}, async () => {});
} finally { retrying = false; }
}
@@ -0,0 +1,22 @@
"""Export only LoRA labels/triggers; invoked by the host's catalog change watcher."""
import json
import os
import sys
from pathlib import Path
def export(source, destination):
catalog = json.loads(Path(source).read_text())
result = {key: {name: value for name, value in catalog.get(key, {}).items()
if isinstance(value, str)} for key in ('loraAliases', 'loraTriggers')}
target = Path(destination)
content = json.dumps(result, ensure_ascii=False)
if target.exists() and target.read_text() == content:
return
target.parent.mkdir(parents=True, exist_ok=True)
temporary = target.with_suffix('.tmp')
temporary.write_text(content)
os.chmod(temporary, 0o644)
temporary.replace(target)
if __name__ == '__main__':
export(*sys.argv[1:])
+15
View File
@@ -0,0 +1,15 @@
import {readFileSync} from 'node:fs';
import {getLoraAliases,getLoraTriggers} from '../library';
import {loraIdentityKey} from '~/utils/loras';
import {join} from 'node:path';
import {studio2Root} from './store';
/** Optional metadata-only production export; no production media/catalog mount. */
export function studio2LoraCatalog(owner: string) {
const local={aliases:getLoraAliases(owner),triggers:getLoraTriggers(owner)};
try {
const source=JSON.parse(readFileSync(process.env.STUDIO2_LORA_CATALOG_PATH || join(studio2Root(),'shared-lora-catalog.cache'),'utf8'));
const normalize=(map: any)=>Object.fromEntries(Object.entries(map || {}).filter(([,value])=>typeof value==='string').map(([key,value])=>[loraIdentityKey(key),value]));
return {aliases:{...local.aliases,...normalize(source.loraAliases)},triggers:{...local.triggers,...normalize(source.loraTriggers)}};
} catch { return local; }
}
+10
View File
@@ -2,6 +2,7 @@ import { mkdtempSync, writeFileSync, readFileSync, rmSync } from 'node:fs'
import { join } from 'node:path' import { join } from 'node:path'
import { tmpdir } from 'node:os' import { tmpdir } from 'node:os'
import { runFfmpeg } from '../ffmpeg' import { runFfmpeg } from '../ffmpeg'
import { imageDimensions } from '../resolution'
export async function fitStill(data: Buffer, width: number, height: number) { export async function fitStill(data: Buffer, width: number, height: number) {
const dir=mkdtempSync(join(tmpdir(),'studio2-fit-')) const dir=mkdtempSync(join(tmpdir(),'studio2-fit-'))
try { try {
@@ -11,3 +12,12 @@ export async function fitStill(data: Buffer, width: number, height: number) {
return readFileSync(output) return readFileSync(output)
} finally { rmSync(dir,{recursive:true,force:true}) } } finally { rmSync(dir,{recursive:true,force:true}) }
} }
/** Only skip the scale+pad when the still already matches the job canvas pixel-for-pixel.
* A same-aspect-but-larger still (e.g. a 4032x2268 phone photo against a 1344x768 canvas)
* still needs fitting: MiniMaxH3AddGuide expects an image sized to the job's own latent,
* not merely the same aspect ratio. Unreadable dimensions always fit (safe default). */
export function needsFit(data: Buffer, width: number, height: number) {
const size = imageDimensions(data)
if (!size?.width || !size?.height) return true
return !(size.width === width && size.height === height)
}
+121 -42
View File
@@ -1,11 +1,17 @@
import { kleinIdentityPlan, applyKleinIdentity } from '~/shared/studio2/klein-identity.mjs';
import { stylePrompt } from '~/shared/studio2/styles.mjs';
import { purge } from './cleanup';
import { watchProgress } from './progress'; import { watchProgress } from './progress';
import { fitStill } from './media'; import { fitStill, needsFit } from './media';
import { resolveSize } from '~/shared/studio2/input.mjs'; import { resolveRequestSize } from './size';
import { readFileSync, mkdirSync, existsSync, unlinkSync } from 'node:fs'; import { queueSeeds } from '~/shared/studio2/seed.mjs';
import { readFileSync, writeFileSync, mkdirSync, existsSync, unlinkSync } from 'node:fs';
import { join } from 'node:path'; import { join } from 'node:path';
import template from '../../assets/studio2_minimax_native.json'; import template from '../../assets/studio2_minimax_native.json';
import { nativeVideoGraph, attachHeroReference } from '~/shared/studio2/graphs.mjs'; import qwen21Template from '../../assets/studio2_qwen21_t2i.json';
import { nativeVideoGraph, attachHeroReference, applyResolvedImageSize } from '~/shared/studio2/graphs.mjs';
import { compilePrompt, scopedFile } from '~/shared/studio2/contracts.mjs'; import { compilePrompt, scopedFile } from '~/shared/studio2/contracts.mjs';
import { resolveQwen21Size } from '~/shared/studio2/input.mjs';
import { createJob, restoreJob, getJob, emitJob, type Job } from '../jobs'; import { createJob, restoreJob, getJob, emitJob, type Job } from '../jobs';
import { markStudioLive, onLiveVideoSettled, type StudioJob } from '../studioQueue'; import { markStudioLive, onLiveVideoSettled, type StudioJob } from '../studioQueue';
import { readRecord, saveRecord, studio2Root, records } from './store'; import { readRecord, saveRecord, studio2Root, records } from './store';
@@ -16,7 +22,7 @@ import { buildImageV2Workflow, resolveKreaGenerateAssets } from '../imageWorkflo
import { resolveGraphLoraNames, applyUserLoraToGraph, ensureComfyLoraNames } from '../loras'; import { resolveGraphLoraNames, applyUserLoraToGraph, ensureComfyLoraNames } from '../loras';
import { getClip, getStill, stillPath, clipVideoPath, saveStill, saveClip, downloadComfyImage, downloadComfyVideo, attachStudio2Metadata } from '../library'; import { getClip, getStill, stillPath, clipVideoPath, saveStill, saveClip, downloadComfyImage, downloadComfyVideo, attachStudio2Metadata } from '../library';
import { imageDimensions } from '../resolution'; import { imageDimensions } from '../resolution';
import { stitchExtension } from '../ffmpeg'; import { stitchExtension, probeDuration } from '../ffmpeg';
import { videoSourcePaths } from '../videoSources'; import { videoSourcePaths } from '../videoSources';
import { resolveExtensionHandoffFrame, persistClipAnchorFrame } from '../extensionFrame'; import { resolveExtensionHandoffFrame, persistClipAnchorFrame } from '../extensionFrame';
import { acquireSharedGpu, sharedGpuHeaders } from '../sharedGpu'; import { acquireSharedGpu, sharedGpuHeaders } from '../sharedGpu';
@@ -25,10 +31,19 @@ type DiskFile = {
subfolder: string; subfolder: string;
type: string; type: string;
}; };
function update(r: any, state: string) { r.state = state; r.updatedAt = Date.now(); saveRecord(r); } function update(r: any, state: string) { r.state = state; r.updatedAt = Date.now(); if (['complete','failed','cancelled'].includes(state)) r.finishedAt ??= r.updatedAt; saveRecord(r); }
// Labels whose pixels must never be resampled: the hero identity still, and the
// extend/video start frame (which is either the exact anchor PNG or an already-canvas-sized
// extracted last frame). Fitting these every hop is what destroys identity across chained extends.
const NEVER_FIT = new Set(['hero', 'start']);
async function upload(r: any, name: string, data: Buffer): Promise<string> { async function upload(r: any, name: string, data: Buffer): Promise<string> {
const prefix = String(useRuntimeConfig().comfyFilenamePrefix).replace(/\/$/, '') + `/studio2/${r.id}/${r.index}`; const prefix = String(useRuntimeConfig().comfyFilenamePrefix).replace(/\/$/, '') + `/studio2/${r.id}/${r.index}`;
if (name !== 'hero') data = await fitStill(data, r.request.settings.width, r.request.settings.height); let fitted = false;
if (!NEVER_FIT.has(name) && needsFit(data, r.request.settings.width, r.request.settings.height)) {
data = await fitStill(data, r.request.settings.width, r.request.settings.height);
fitted = true;
}
if (name === 'start') { r.handoffFitted = fitted; saveRecord(r); }
const body = new FormData(); const body = new FormData();
body.append('image', new Blob([new Uint8Array(data)]), name + '.png'); body.append('image', new Blob([new Uint8Array(data)]), name + '.png');
body.append('subfolder', prefix); body.append('subfolder', prefix);
@@ -45,60 +60,110 @@ async function upload(r: any, name: string, data: Buffer): Promise<string> {
saveRecord(r); saveRecord(r);
return `${disk.subfolder}/${disk.filename}`; return `${disk.subfolder}/${disk.filename}`;
} }
async function purge(r: any) {
const config = useRuntimeConfig(), prefix = String(config.comfyFilenamePrefix);
const files = (r.files as DiskFile[]).filter(file => scopedFile(file, prefix));
r.purgeResult = 'Left on host';
if (files.length !== r.files.length || !files.length)
return;
try {
const response = await fetch(String(config.comfyControlUrl).replace(/\/$/, '') + '/studio2/purge', {
method: 'POST', headers: { 'Content-Type': 'application/json', ...sharedGpuHeaders() },
body: JSON.stringify({ prefix, files }), signal: AbortSignal.timeout(12000)
});
const result = await response.json() as any;
if (response.ok && result.ok && result.cleared === files.length)
r.purgeResult = 'Cleared input + output';
}
catch { /* Keep the durable library copy and report unconfirmed host cleanup. */ }
}
async function prepareGraph(r: any) { async function prepareGraph(r: any) {
const q = r.request, s = q.settings, prefix = String(useRuntimeConfig().comfyFilenamePrefix).replace(/\/$/, '') + `/studio2/${r.id}/${r.index}`; const q = r.request, s = q.settings, prefix = String(useRuntimeConfig().comfyFilenamePrefix).replace(/\/$/, '') + `/studio2/${r.id}/${r.index}`;
const load = async (id: string, label: string) => id ? upload(r, label, readFileSync(stillPath(r.owner, getStill(r.owner, id).id))) : ''; const load = async (id: string, label: string) => id ? upload(r, label, readFileSync(stillPath(r.owner, getStill(r.owner, id).id))) : '';
const source = q.startClipId ? getClip(r.owner,q.startClipId) : (q.imageAId || q.identityStillId) ? getStill(r.owner,q.imageAId || q.identityStillId) : null; const video=['video','extend'].includes(q.mode);
Object.assign(s,resolveSize(s.aspect,source,s)); const identityPlan=kleinIdentityPlan(q);
const sourceId=identityPlan ? identityPlan.sourceId : q.imageAId;
await resolveRequestSize(r.owner,identityPlan ? {...q,imageAId:sourceId} : q);
saveRecord(r); saveRecord(r);
const hero = q.lockFace === false && q.lockOutfit === false ? '' : await load(q.identityStillId, 'hero'), a = await load(q.imageAId, 'source'), b = await load(q.imageBId, 'compose'), mask = await load(q.maskId, 'mask'); const a=await load(sourceId,'source'),b=await load(q.imageBId,'compose'),mask=await load(q.maskId,'mask');
const attachHero=(video || q.engine==='flux') && (q.lockFace!==false || q.lockOutfit!==false) && !(!video && sourceId && sourceId===q.identityStillId);
const hero=attachHero ? await load(q.identityStillId,'hero') : '';
r.sourceStillId=sourceId || null;
r.heroReferenceAttached=!!hero || (!video && q.engine==='flux' && !!sourceId && sourceId===q.identityStillId);
let graph: any; let graph: any;
if (['video', 'extend'].includes(q.mode)) { if (['video', 'extend'].includes(q.mode)) {
if (q.engine !== 'minimax')
throw new Error('Studio 2 native identity video currently requires MiniMax. Use the existing xAIGen studio for LTX.');
let start = a; let start = a;
if (q.startClipId) { if (q.startClipId) {
const dir = join(studio2Root(), r.id); const dir = join(studio2Root(), r.id);
mkdirSync(dir, { recursive: true }); mkdirSync(dir, { recursive: true });
const dest = join(dir, 'handoff.png'); const dest = join(dir, 'handoff.png');
await resolveExtensionHandoffFrame({ ownerKey: r.owner, sourceClipId: q.startClipId, sourceVideoPath: clipVideoPath(r.owner, q.startClipId), destPath: dest }); const handoff = await resolveExtensionHandoffFrame({ ownerKey: r.owner, sourceClipId: q.startClipId, sourceVideoPath: clipVideoPath(r.owner, q.startClipId), destPath: dest });
r.handoffSource = handoff.source;
start = await upload(r, 'start', readFileSync(dest)); start = await upload(r, 'start', readFileSync(dest));
unlinkSync(dest); unlinkSync(dest);
} }
const end = await load(q.endStillId, 'end'), guides = []; if (q.engine === 'ltx') {
for (const [i, g] of q.guides.entries()) const { buildWorkflow } = await import('../workflow');
guides.push({ image: await load(g.stillId, `guide${i}`), frame: g.frame }); const { frameLength } = await import('../videoChain');
graph = nativeVideoGraph(template, q, { hero, start, end, guides }, prefix); const { ltxWorkflowEnabled, LTX_DISABLED_MESSAGE } = await import('~/utils/videoModels');
await ensureComfyLoraNames('video'); if (!ltxWorkflowEnabled()) throw new Error(LTX_DISABLED_MESSAGE);
resolveGraphLoraNames(graph, 'video'); const textToVideo = q.mode === 'video' && !start;
applyUserLoraToGraph(graph, s.loraStack); if (!start && !textToVideo) throw new Error('Choose a start still for LTX video.');
r.graphId = 'studio2_minimax_native.json'; const fps = s.fps || 24;
graph = buildWorkflow({
workflow: textToVideo ? 'ltx-t2v' : 'ltx',
imageName: start || '',
prompt: q.compiledPrompt,
width: s.width,
height: s.height,
length: frameLength(s.duration || 5, fps),
fps,
steps: s.steps,
cfg: s.cfg,
seed: s.seed,
samplerName: 'euler',
scheduler: 'simple',
turbo: false,
duration: s.duration || 5,
filenamePrefix: prefix + '/video',
loraStack: s.loraStack,
useIdentityRefs: false
});
await ensureComfyLoraNames('video');
r.graphId = textToVideo ? 'workflow_ltx_video.json#t2v' : 'workflow_ltx_video.json';
} else {
if (q.engine !== 'minimax')
throw new Error('Studio 2 video supports MiniMax H3 and LTX (xAIGen).');
const end = await load(q.endStillId, 'end'), guides = [];
for (const [i, g] of q.guides.entries())
guides.push({ image: await load(g.stillId, `guide${i}`), frame: g.frame });
graph = nativeVideoGraph(template, q, { hero, start, end, guides }, prefix);
await ensureComfyLoraNames('video');
resolveGraphLoraNames(graph, 'video');
applyUserLoraToGraph(graph, s.loraStack);
r.graphId = 'studio2_minimax_native.json';
}
}
else if (q.engine === 'qwen21') {
if (q.mode !== 'generate')
throw new Error('Qwen 2.1 is T2I in this build.');
const size = resolveQwen21Size(s.aspect);
s.width = size.width;
s.height = size.height;
graph = structuredClone(qwen21Template);
graph['9'].inputs.prompt = q.compiledPrompt;
graph['9'].inputs.negative_prompt = stylePrompt(q.imageStyles, true);
// TextEncodeQwenImage21 builds the 64-ch empty latent from resolution (square T2I).
graph['9'].inputs.resolution = Math.max(size.width, size.height);
graph['15'].inputs.seed = s.seed;
graph['15'].inputs.steps = s.steps || 25;
graph['15'].inputs.cfg = s.cfg ?? 1;
graph['15'].inputs.sampler_name = 'euler';
graph['15'].inputs.scheduler = 'simple';
graph['21'].inputs.filename_prefix = prefix + '/image';
r.sampleLatent = 'qwen21 textencode latent';
r.sampleDenoise = null;
r.heroReferenceAttached = false;
r.graphId = 'studio2_qwen21_t2i.json';
} }
else { else {
const mode = q.mode === 'iterate' ? (a ? 'edit' : 'generate') : q.mode; const mode = identityPlan?.mode ?? (q.mode === 'iterate' ? (a ? 'edit' : 'generate') : q.mode);
const found = q.engine === 'krea' ? await resolveKreaGenerateAssets() : null; const found = q.engine === 'krea' ? await resolveKreaGenerateAssets() : null;
const assets = found ? { kreaUnetName: found.unet, kreaClipName: found.clip, kreaVaeName: found.vae, kreaConceptLora: found.conceptLora } : {}; const assets = found ? { kreaUnetName: found.unet, kreaClipName: found.clip, kreaVaeName: found.vae, kreaConceptLora: found.conceptLora } : {};
graph = buildImageV2Workflow({ ...s, ...assets, engine: q.engine, mode, task: 'scene', prompt: q.compiledPrompt, imageAName: a, imageBName: mode === 'compose' ? b : undefined, maskName: mode === 'refine' ? mask : undefined, filenamePrefix: prefix + '/image' }).graph; graph = buildImageV2Workflow({ ...s, ...assets, engine: q.engine, mode, task: 'scene', prompt: q.compiledPrompt, negative: stylePrompt(q.imageStyles, true), imageAName: a, imageBName: mode === 'compose' ? b : undefined, maskName: mode === 'refine' ? mask : undefined, filenamePrefix: prefix + '/image' }).graph;
applyResolvedImageSize(graph,s);
if (identityPlan) Object.assign(r,applyKleinIdentity(graph,identityPlan));
if (q.engine === 'flux') if (q.engine === 'flux')
attachHeroReference(graph, hero); attachHeroReference(graph, hero);
else if (hero && ['edit', 'compose', 'iterate'].includes(q.mode)) if (!identityPlan) {
r.sampleLatent=q.mode==='refine'?'masked source latent':'empty latent';
r.sampleDenoise=null;
}
if (q.engine !== 'flux' && hero && ['edit', 'compose', 'iterate'].includes(q.mode))
throw new Error('Krea native hero-reference binding is not supported by this graph. Select Klein explicitly; no fallback is performed.'); throw new Error('Krea native hero-reference binding is not supported by this graph. Select Klein explicitly; no fallback is performed.');
await ensureComfyLoraNames('image'); await ensureComfyLoraNames('image');
resolveGraphLoraNames(graph, 'image'); resolveGraphLoraNames(graph, 'image');
@@ -112,6 +177,7 @@ async function run(r: any, job: Job) {
try { try {
const prompts = r.prompts || [r.request.promptSections, ...r.request.batch]; const prompts = r.prompts || [r.request.promptSections, ...r.request.batch];
r.prompts = prompts; r.prompts = prompts;
r.request.shotSeeds ||= queueSeeds(r.request.settings,prompts.length);
saveRecord(r); saveRecord(r);
for (r.index = r.index || 0; r.index < prompts.length; r.index++) { for (r.index = r.index || 0; r.index < prompts.length; r.index++) {
if (job.status === 'cancelled') if (job.status === 'cancelled')
@@ -122,6 +188,7 @@ async function run(r: any, job: Job) {
throw new Error('Cancelled'); throw new Error('Cancelled');
await new Promise(resolve => setTimeout(resolve, 2000)); await new Promise(resolve => setTimeout(resolve, 2000));
} }
r.request.settings.seed = r.request.shotSeeds[r.index];
r.files = []; r.files = [];
r.progress = null; r.progress = null;
r.request.promptSections = prompts[r.index]; r.request.promptSections = prompts[r.index];
@@ -193,10 +260,21 @@ async function run(r: any, job: Job) {
let asset: any = r.savedAssetId ? (video ? getClip(r.owner, r.savedAssetId) : getStill(r.owner, r.savedAssetId)) : null; let asset: any = r.savedAssetId ? (video ? getClip(r.owner, r.savedAssetId) : getStill(r.owner, r.savedAssetId)) : null;
if (!asset && video) { if (!asset && video) {
const data = await downloadComfyVideo(file); const data = await downloadComfyVideo(file);
if (!data.length || data.length < 64)
throw new Error('Comfy wrote an empty video file (<64 bytes); the job did not render. Nothing was saved.');
const sourceSegments = q.startClipId ? videoSourcePaths(clipVideoPath(r.owner, q.startClipId)) : []; const sourceSegments = q.startClipId ? videoSourcePaths(clipVideoPath(r.owner, q.startClipId)) : [];
const tmp = join(studio2Root(), r.id); const tmp = join(studio2Root(), r.id);
mkdirSync(tmp, { recursive: true }); mkdirSync(tmp, { recursive: true });
const assembled = q.startClipId ? await stitchExtension({ part1Path: clipVideoPath(r.owner, q.startClipId), part2: data, tmpDir: tmp, sourcePaths: sourceSegments }) : data; const assembled = q.startClipId ? await stitchExtension({ part1Path: clipVideoPath(r.owner, q.startClipId), part2: data, tmpDir: tmp, sourcePaths: sourceSegments }) : data;
if (!assembled.length || assembled.length < 64)
throw new Error('The assembled video file is empty (<64 bytes); the job did not render. Nothing was saved.');
const checkPath = join(tmp, 'duration-check.mp4');
writeFileSync(checkPath, assembled);
let duration = 0;
try { duration = await probeDuration(checkPath); }
finally { try { unlinkSync(checkPath); } catch { /* ignore */ } }
if (!Number.isFinite(duration) || duration <= 0)
throw new Error('The rendered video has no readable duration; the job did not render. Nothing was saved.');
asset = await saveClip({ ownerKey: r.owner, folderId: q.folderId, prompt: q.compiledPrompt, ...s, aspect: s.aspect || 'auto', hideThumbnail: false, video: assembled, originalSegment: data, sourceSegments, fps: s.fps || 24, sound: true, familyId: r.familyId, parentClipId: q.startClipId || undefined, chainIndex: r.index, comfyFilename: file.filename }); asset = await saveClip({ ownerKey: r.owner, folderId: q.folderId, prompt: q.compiledPrompt, ...s, aspect: s.aspect || 'auto', hideThumbnail: false, video: assembled, originalSegment: data, sourceSegments, fps: s.fps || 24, sound: true, familyId: r.familyId, parentClipId: q.startClipId || undefined, chainIndex: r.index, comfyFilename: file.filename });
} }
else if (!asset) { else if (!asset) {
@@ -209,7 +287,7 @@ async function run(r: any, job: Job) {
saveRecord(r); saveRecord(r);
r.savedAt = Date.now(); r.savedAt = Date.now();
r.wallTime = (r.savedAt - r.startedAt) / 1000; r.wallTime = (r.savedAt - r.startedAt) / 1000;
const metadata = { ...structuredClone(q), id: r.id, kind: video ? 'video' : 'image', graphId: r.graphId, promptId: r.promptId, queuedAt: r.queuedAt, startedAt: r.startedAt, savedAt: r.savedAt, gpuSeconds: r.gpuSeconds, wallTime: r.wallTime, outputWidth: asset.width || s.width, outputHeight: asset.height || s.height, purgeResult: 'Left on host' }; const metadata = { ...structuredClone(q), id: r.id, sourceStillId:r.sourceStillId,heroReferenceAttached:r.heroReferenceAttached,sampleLatent:r.sampleLatent,sampleDenoise:r.sampleDenoise,kind: video ? 'video' : 'image', graphId: r.graphId, promptId: r.promptId, queuedAt: r.queuedAt, startedAt: r.startedAt, savedAt: r.savedAt, gpuSeconds: r.gpuSeconds, wallTime: r.wallTime, outputWidth: asset.width || s.width, outputHeight: asset.height || s.height, purgeResult: 'Left on host', handoffSource: r.handoffSource || null, fitted: video && q.startClipId ? !!r.handoffFitted : null };
await attachStudio2Metadata(r.owner, asset.id, metadata); await attachStudio2Metadata(r.owner, asset.id, metadata);
if (!r.outputs.some((o:any)=>o.id===asset.id)) r.outputs.push({id:asset.id,kind:video?'clip':'still',studio2:metadata}); if (!r.outputs.some((o:any)=>o.id===asset.id)) r.outputs.push({id:asset.id,kind:video?'clip':'still',studio2:metadata});
saveRecord(r); saveRecord(r);
@@ -261,6 +339,7 @@ export async function startStudio2Job(item: StudioJob) {
job.library = { ownerKey: item.ownerKey, folderId: r.request.folderId, extensions: [], queueAutoRun: false } as any; job.library = { ownerKey: item.ownerKey, folderId: r.request.folderId, extensions: [], queueAutoRun: false } as any;
job.status = 'running'; job.status = 'running';
r.startedAt = Date.now(); r.startedAt = Date.now();
delete r.finishedAt;
r.liveId = job.id; r.liveId = job.id;
r.clientId = job.clientId; r.clientId = job.clientId;
saveRecord(r); saveRecord(r);
+25
View File
@@ -0,0 +1,25 @@
import { readFileSync } from 'node:fs'
import { execFile } from 'node:child_process'
import { promisify } from 'node:util'
import { getClip, getStill, clipVideoPath, stillPath } from '../library'
import { imageDimensions } from '../resolution'
import { resolveSize } from '~/shared/studio2/input.mjs'
const exec = promisify(execFile)
export async function resolveRequestSize(owner: string, request: any) {
const video=['video','extend'].includes(request.mode)
let dimensions:any
if (video && request.startClipId) {
getClip(owner,request.startClipId)
const result=await exec('ffprobe',['-v','error','-select_streams','v:0','-show_entries','stream=width,height','-of','json',clipVideoPath(owner,request.startClipId)],{windowsHide:true,timeout:15000})
dimensions=JSON.parse(result.stdout).streams?.[0]
} else {
const id=request.imageAId || request.identityStillId
if (id) {
getStill(owner,id)
dimensions=imageDimensions(readFileSync(stillPath(owner,id)))
if (!dimensions) throw new Error('Cannot measure the source still.')
}
}
if (video && request.startClipId && !dimensions?.width) throw new Error('Cannot measure the previous clip.')
Object.assign(request.settings,resolveSize(request.settings.aspect,dimensions,undefined,request.engine))
}
+85 -13
View File
@@ -13,7 +13,7 @@ import { imageV2StackSpecials } from '~/utils/imageV2'
import { allowIdentityRefs, type PermanenceRef } from '~/utils/globalLocks' import { allowIdentityRefs, type PermanenceRef } from '~/utils/globalLocks'
export type StudioJobStatus = 'waiting' | 'running' | 'held' | 'complete' | 'error' | 'cancelled' export type StudioJobStatus = 'waiting' | 'running' | 'held' | 'complete' | 'error' | 'cancelled'
export type StudioJobKind = 'video' | 'edit' | 'music' export type StudioJobKind = 'video' | 'edit' | 'music' | 'caption'
export interface StudioJobPayload { export interface StudioJobPayload {
studio2Id?: string studio2Id?: string
@@ -40,6 +40,7 @@ export interface StudioJobPayload {
useIdentityRefs: boolean useIdentityRefs: boolean
stillId?: string stillId?: string
stillFilename?: string stillFilename?: string
captionStyle?: string
hideThumbnail: boolean hideThumbnail: boolean
hideInput?: boolean hideInput?: boolean
folderLocked?: boolean folderLocked?: boolean
@@ -224,6 +225,7 @@ export function listStudioJobs(owner: string) {
export function studioJobKind(job: Pick<StudioJob, 'kind'> | { kind?: string }) { export function studioJobKind(job: Pick<StudioJob, 'kind'> | { kind?: string }) {
if (job.kind === 'edit') return 'edit' if (job.kind === 'edit') return 'edit'
if (job.kind === 'music') return 'music' if (job.kind === 'music') return 'music'
if (job.kind === 'caption') return 'caption'
return 'video' return 'video'
} }
@@ -240,6 +242,8 @@ export function summarizeStudioJob(job: StudioJob) {
familyId: job.familyId, familyId: job.familyId,
shotQueueId: job.shotQueueId, shotQueueId: job.shotQueueId,
liveJobId: job.liveJobId, liveJobId: job.liveJobId,
folderId: job.payload.folderId,
musicEngine: job.payload.musicEngine,
stillId: job.payload.stillId, stillId: job.payload.stillId,
workflow: job.payload.workflow, workflow: job.payload.workflow,
imagePipeline: job.payload.imagePipeline || 'v1', imagePipeline: job.payload.imagePipeline || 'v1',
@@ -286,7 +290,7 @@ function failZombieLiveJob(job: Job, error: string) {
function sweepStaleLiveJobs() { function sweepStaleLiveJobs() {
const now = Date.now() const now = Date.now()
for (const job of listJobs()) { for (const job of listJobs()) {
if (job.studio2 || job.yueGp || job.upscale) continue if (job.studio2 || job.yueGp || job.yue2 || job.upscale || job.caption) continue
if (job.saving) continue if (job.saving) continue
if (job.status === 'queued' && !job.promptId && now - job.startedAt >= QUEUED_GRACE_MS) { if (job.status === 'queued' && !job.promptId && now - job.startedAt >= QUEUED_GRACE_MS) {
failZombieLiveJob(job, 'Job never started') failZombieLiveJob(job, 'Job never started')
@@ -306,12 +310,12 @@ async function reapZombieLiveJobs() {
const { fetchHistory } = await import('~/server/utils/comfy') const { fetchHistory } = await import('~/server/utils/comfy')
for (const job of listJobs()) { for (const job of listJobs()) {
// Never interrupt download/stitch/library save — Comfy is idle then by design. // Never interrupt download/stitch/library save — Comfy is idle then by design.
if (job.studio2 || job.yueGp || job.upscale) continue if (job.studio2 || job.yueGp || job.yue2 || job.upscale || job.caption) continue
if (job.saving) continue if (job.saving) continue
if (job.library?.chainContinuing) continue if (job.library?.chainContinuing) continue
if (job.status !== 'running' && job.status !== 'uploading' && job.status !== 'queued') continue if (job.status !== 'running' && job.status !== 'uploading' && job.status !== 'queued') continue
if (jobIsLocallySubmitting(job)) continue if (jobIsLocallySubmitting(job)) continue
const zombieMs = (job.kind === 'edit' || job.kind === 'music') ? 45_000 : ZOMBIE_EMPTY_COMFY_MS const zombieMs = (job.kind === 'edit' || job.kind === 'music' || job.kind === 'caption') ? 45_000 : ZOMBIE_EMPTY_COMFY_MS
if (jobAgeMs(job) < zombieMs) continue if (jobAgeMs(job) < zombieMs) continue
if (!job.promptId) { if (!job.promptId) {
if (jobAgeMs(job) >= QUEUED_GRACE_MS) failZombieLiveJob(job, 'Job never started on ComfyUI') if (jobAgeMs(job) >= QUEUED_GRACE_MS) failZombieLiveJob(job, 'Job never started on ComfyUI')
@@ -351,7 +355,7 @@ async function reapZombieLiveJobs() {
} }
function liveJobOwnsGpu(job: Job) { function liveJobOwnsGpu(job: Job) {
if ((job.studio2 || job.yueGp || job.upscale) && ['running', 'queued', 'uploading'].includes(job.status)) return true if ((job.studio2 || job.yueGp || job.yue2 || job.upscale || job.caption) && ['running', 'queued', 'uploading'].includes(job.status)) return true
if (job.library?.stopAfterCurrent) return false if (job.library?.stopAfterCurrent) return false
if (job.library?.chainContinuing) return true if (job.library?.chainContinuing) return true
if (job.saving) return true if (job.saving) return true
@@ -371,7 +375,7 @@ function liveJobOwnsGpu(job: Job) {
*/ */
function clearDeadGpuClaimsForForceStart() { function clearDeadGpuClaimsForForceStart() {
for (const live of listJobs()) { for (const live of listJobs()) {
if (live.studio2 || live.yueGp || live.upscale) continue if (live.studio2 || live.yueGp || live.yue2 || live.upscale || live.caption) continue
if (live.saving) continue if (live.saving) continue
if (jobIsLocallySubmitting(live)) continue if (jobIsLocallySubmitting(live)) continue
if (live.status !== 'running' && live.status !== 'queued' && live.status !== 'uploading') { if (live.status !== 'running' && live.status !== 'queued' && live.status !== 'uploading') {
@@ -465,8 +469,11 @@ export async function addStudioJob(params: {
kind?: StudioJobKind kind?: StudioJobKind
}) { }) {
const now = Date.now() const now = Date.now()
const kind = params.kind === 'edit' ? 'edit' : params.kind === 'music' ? 'music' : 'video' const kind = params.kind === 'edit' ? 'edit'
const shotCount = kind === 'music' : params.kind === 'music' ? 'music'
: params.kind === 'caption' ? 'caption'
: 'video'
const shotCount = kind === 'music' || kind === 'caption'
? 1 ? 1
: kind === 'edit' : kind === 'edit'
? 1 + (params.payload.passes?.length || 0) ? 1 + (params.payload.passes?.length || 0)
@@ -598,6 +605,14 @@ export async function clearStuckStudioWork(owner: string) {
const { cancelYueGpJob } = await import('./yueGp') const { cancelYueGpJob } = await import('./yueGp')
await cancelYueGpJob(job) await cancelYueGpJob(job)
} }
if (job.yue2) {
const { cancelYue2Job } = await import('./yue2')
await cancelYue2Job(job)
}
if (job.caption) {
const { cancelCaptionJob } = await import('./caption')
await cancelCaptionJob(job)
}
job.status = 'cancelled' job.status = 'cancelled'
job.error = 'Cleared by force reset' job.error = 'Cleared by force reset'
if (job.library) { if (job.library) {
@@ -665,6 +680,16 @@ async function stopLiveGeneration(liveJobId?: string, shotQueueId?: string) {
await cancelYueGpJob(live) await cancelYueGpJob(live)
return return
} }
if (live?.yue2) {
const { cancelYue2Job } = await import('./yue2')
await cancelYue2Job(live)
return
}
if (live?.caption) {
const { cancelCaptionJob } = await import('./caption')
await cancelCaptionJob(live)
return
}
if (live) { if (live) {
live.status = 'cancelled' live.status = 'cancelled'
if (live.library) { if (live.library) {
@@ -872,6 +897,14 @@ function pendingAlive(job: StudioJob) {
if (job.liveJobId) { if (job.liveJobId) {
const pending = readPendingJob(job.liveJobId) const pending = readPendingJob(job.liveJobId)
if (pending?.promptId && freshEnough(pending.startedAt)) return true if (pending?.promptId && freshEnough(pending.startedAt)) return true
if (job.kind === 'music') {
const root = join(String(useRuntimeConfig().libraryDir || process.env.LIBRARY_DIR || '/data/library'), 'yue2-pending', `${job.liveJobId}.json`)
if (existsSync(root)) return true
}
if (job.kind === 'caption') {
const root = join(String(useRuntimeConfig().libraryDir || process.env.LIBRARY_DIR || '/data/library'), 'caption-pending', `${job.liveJobId}.json`)
if (existsSync(root)) return true
}
} }
if (!job.shotQueueId) return false if (!job.shotQueueId) return false
return listPendingJobs().some(pending => ( return listPendingJobs().some(pending => (
@@ -914,7 +947,7 @@ function repairStaleJobs(jobs: StudioJob[]) {
job.updatedAt = Date.now() job.updatedAt = Date.now()
continue continue
} }
if (!job.payload.upscale && job.payload.musicEngine !== 'yue' && job.status === 'error' && isTransientComfyError(job.lastError)) { if (!job.payload.upscale && job.kind !== 'music' && job.status === 'error' && isTransientComfyError(job.lastError)) {
job.status = 'waiting' job.status = 'waiting'
job.liveJobId = undefined job.liveJobId = undefined
job.lastError = undefined job.lastError = undefined
@@ -923,7 +956,7 @@ function repairStaleJobs(jobs: StudioJob[]) {
job.updatedAt = Date.now() job.updatedAt = Date.now()
continue continue
} }
if (!job.payload.upscale && job.payload.musicEngine !== 'yue' && job.status === 'held' && isTransientComfyError(job.lastError)) { if (!job.payload.upscale && job.kind !== 'music' && job.status === 'held' && isTransientComfyError(job.lastError)) {
job.status = 'waiting' job.status = 'waiting'
job.liveJobId = undefined job.liveJobId = undefined
job.lastError = undefined job.lastError = undefined
@@ -1380,6 +1413,42 @@ async function startStudioExtendJob(item: StudioJob) {
} }
} }
async function startStudioCaptionJob(item: StudioJob) {
let live: import('~/server/utils/jobs').Job | undefined
try {
const { startCaptionJob } = await import('./caption')
const style = String(item.payload.captionStyle || 'descriptive')
const stillId = String(item.payload.stillId || '')
if (!stillId) throw new Error('Caption requires a still.')
live = await startCaptionJob({
ownerKey: item.ownerKey,
folderId: item.payload.folderId,
stillId,
captionStyle: style,
studio2Id: item.payload.studio2Id,
name: item.payload.name
})
await markStudioLive(item.ownerKey, item.id, live.id)
if (item.payload.studio2Id) {
const { readRecord, saveRecord } = await import('./studio2/store')
const record = readRecord(item.ownerKey, item.payload.studio2Id)
record.state = 'rendering'
record.liveId = live.id
record.startedAt = Date.now()
record.updatedAt = Date.now()
saveRecord(record)
}
} catch (error) {
const message = error instanceof Error ? error.message : String(error)
if (live && (live.status === 'queued' || live.status === 'uploading' || live.status === 'running')) {
live.status = 'error'
live.error = message
}
await parkStudioOnStartFailure(item.ownerKey, item.id, message)
kickStudioQueue()
}
}
async function startStudioMusicJob(item: StudioJob) { async function startStudioMusicJob(item: StudioJob) {
let live: import('~/server/utils/jobs').Job | undefined let live: import('~/server/utils/jobs').Job | undefined
try { try {
@@ -1398,8 +1467,7 @@ async function startStudioMusicJob(item: StudioJob) {
lyricsStrength: payload.instrumental ? 0 : (payload.lyricsStrength ?? 0.9), lyricsStrength: payload.instrumental ? 0 : (payload.lyricsStrength ?? 0.9),
instrumental: payload.instrumental === true, instrumental: payload.instrumental === true,
folderLocked: payload.folderLocked, folderLocked: payload.folderLocked,
engine: payload.musicEngine || 'ace-step', engine: 'yue2',
yueProfile: payload.yueProfile ?? 1,
samplerName: payload.samplerName || 'euler', samplerName: payload.samplerName || 'euler',
scheduler: payload.scheduler || 'simple' scheduler: payload.scheduler || 'simple'
}) })
@@ -1423,6 +1491,10 @@ export async function startStudioJob(item: StudioJob) {
} }
async function startStudioJobReserved(item: StudioJob) { async function startStudioJobReserved(item: StudioJob) {
if (studioJobKind(item) === 'caption') {
await startStudioCaptionJob(item)
return
}
if (item.payload.studio2Id) { if (item.payload.studio2Id) {
try { try {
const { startStudio2Job } = await import('./studio2/runner') const { startStudio2Job } = await import('./studio2/runner')
@@ -1687,7 +1759,7 @@ export async function onLiveVideoSettled(job: Job) {
return return
} }
const remaining = remainingStudioShots(job) const remaining = remainingStudioShots(job)
const wakeFail = !job.studio2 && !job.yueGp && !job.upscale && job.status === 'error' && remaining > 0 && isTransientComfyError(job.error) const wakeFail = !job.studio2 && !job.yueGp && !job.yue2 && !job.upscale && !job.caption && job.status === 'error' && remaining > 0 && isTransientComfyError(job.error)
const failed = (job.status === 'error' || job.status === 'cancelled') && !wakeFail const failed = (job.status === 'error' || job.status === 'cancelled') && !wakeFail
await mutateStore(owner, (store) => { await mutateStore(owner, (store) => {
+152
View File
@@ -0,0 +1,152 @@
import { existsSync, mkdirSync, readdirSync, readFileSync, writeFileSync, renameSync, unlinkSync } from 'node:fs'
import { join } from 'node:path'
import { createJob, emitJob, getJob, restoreMusicJob, type Job } from './jobs'
import { saveTrack, findYue2Track } from './library'
import { sharedGpuHeaders } from './sharedGpu'
import type { MusicJobParams } from './musicChain'
function settings() {
const config = useRuntimeConfig()
return { url: String(config.comfyControlUrl || process.env.COMFY_CONTROL_URL || '').replace(/\/$/, ''),
token: String(config.comfyControlToken || process.env.COMFY_CONTROL_TOKEN || '') }
}
export function yue2Configured() { return Boolean(settings().url) }
function pendingRoot() { return join(String(useRuntimeConfig().libraryDir || process.env.LIBRARY_DIR || '/data/library'), 'yue2-pending') }
export function yue2PendingAlive(jobId: string) {
if (!jobId) return false
return existsSync(join(pendingRoot(), `${jobId}.json`))
}
async function request(path: string, body?: unknown) {
const { url, token } = settings()
if (!url) throw new Error('YuE2 host is not configured. Set COMFY_CONTROL_URL.')
const response = await fetch(`${url}/yue2/${path}`, {
method: body === undefined ? 'GET' : 'POST',
headers: { ...(token ? { Authorization: `Bearer ${token}` } : {}),
...(body === undefined ? {} : { ...sharedGpuHeaders(), 'Content-Type': 'application/json' }) },
body: body === undefined ? undefined : JSON.stringify(body), signal: AbortSignal.timeout(60_000)
})
if (!response.ok) {
const detail = await response.json().catch(() => ({})) as { message?: string; error?: string }
throw Object.assign(new Error(detail.message || detail.error || `YuE2 host returned ${response.status}`), { statusCode: response.status })
}
return response
}
function persist(job: Job) {
mkdirSync(pendingRoot(), { recursive: true })
const path = join(pendingRoot(), `${job.id}.json`)
writeFileSync(path + '.tmp', JSON.stringify({ id: job.id, clientId: job.clientId, startedAt: job.startedAt, library: job.library, trackId: job.trackId }))
renameSync(path + '.tmp', path)
}
async function settle(job: Job) {
const { onLiveVideoSettled } = await import('./studioQueue')
await onLiveVideoSettled(job)
const path = join(pendingRoot(), `${job.id}.json`)
if (existsSync(path)) unlinkSync(path)
}
export async function cancelYue2Job(job: Job) {
await request(`jobs/${job.id}/cancel`, {})
job.status = 'cancelled'
emitJob(job, { type: 'error', error: 'Cancelled', message: 'Cancelled' })
}
async function watch(job: Job) {
let failures = 0
while (job.status !== 'cancelled') {
try {
const state = await (await request(`jobs/${job.id}`)).json() as {
status: string; message: string; error?: string; stage?: string; progress?: number; step?: number; maxStep?: number; duration?: number
}
if (job.status === 'cancelled') { await settle(job); return }
job.musicActivity = { checkedAt: Date.now(), running: ['starting', 'running', 'cancelling'].includes(state.status) }
if (state.status === 'error' || state.status === 'cancelled') {
job.status = state.status === 'cancelled' ? 'cancelled' : 'error'
job.error = state.error || state.message
if (/out of memory/i.test(job.error || '')) job.error += ' YuE2 does not switch engines automatically.'
emitJob(job, { type: 'error', error: job.error, message: job.error })
await settle(job)
return
}
if (state.status === 'complete') {
job.saving = true
emitJob(job, { type: 'status', message: 'Saving audio to library…', progress: 98 })
const lib = job.library!
lib.audioExt = 'wav'
job.trackId ||= findYue2Track(lib.ownerKey, job.id)?.id
if (!job.trackId) {
const audio = Buffer.from(await (await request(`jobs/${job.id}/audio`)).arrayBuffer())
const track = await saveTrack({ ownerKey: lib.ownerKey, folderId: lib.folderId, name: lib.name,
tags: lib.tags || lib.prompt, lyrics: lib.lyrics || '', duration: state.duration || lib.duration || 60,
seed: lib.seed, steps: 0, cfg: 0, instrumental: lib.instrumental === true,
engine: 'yue2', audio, ext: 'wav', comfyFilename: `yue2-${job.id}.wav` })
job.trackId = track.id
persist(job)
}
job.status = 'complete'; job.saving = false
emitJob(job, { type: 'complete', message: lib.folderLocked ? 'Saved to the locked folder.' : 'Track ready',
progress: 100, trackId: job.trackId, folderLocked: lib.folderLocked, audioExt: 'wav' })
await settle(job)
return
}
failures = 0
job.status = 'running'
const percent = Number(state.progress || 0)
emitJob(job, { type: 'progress', message: `${state.message}${state.maxStep ? ` · ${state.step}/${state.maxStep}` : ''}`,
progress: Math.min(95, Math.max(1, percent)), step: state.step || 0, maxStep: state.maxStep || 0 })
} catch (error) {
failures++
job.saving = false
emitJob(job, { type: 'status', message: `YuE2 connection/save check failed; retrying: ${error instanceof Error ? error.message : String(error)}` })
if (failures >= 10) {
job.status = 'error'; job.error = 'YuE2 host unreachable or audio save failed. Job files are retained for recovery.'
emitJob(job, { type: 'error', error: job.error, message: job.error })
const { onLiveVideoSettled } = await import('./studioQueue')
await onLiveVideoSettled(job)
return
}
}
await new Promise(resolve => setTimeout(resolve, 2000))
}
await settle(job)
}
export function startYue2Job(params: MusicJobParams) {
const job = createJob('music')
job.yue2 = true
job.library = { ...params, prompt: params.tags, engine: 'yue2', aspect: 'audio', width: 0, height: 0,
hideThumbnail: false, turbo: false, sound: true }
persist(job)
setTimeout(() => { void (async () => {
try {
if (job.status === 'cancelled') { await settle(job); return }
await request('jobs', { id: job.id, tags: params.tags, lyrics: params.lyrics,
duration: params.duration, seed: params.seed })
await watch(job)
} catch (error) {
const statusCode = (error as { statusCode?: number })?.statusCode
if (statusCode && statusCode >= 400 && statusCode < 500) {
job.status = 'error'; job.error = error instanceof Error ? error.message : String(error)
emitJob(job, { type: 'error', error: job.error, message: job.error })
await settle(job)
return
}
emitJob(job, { type: 'status', message: `Checking YuE2 submission: ${error instanceof Error ? error.message : String(error)}` })
await watch(job)
}
})() }, 0)
return job
}
export function resumeYue2Jobs() {
if (!existsSync(pendingRoot())) return
for (const file of readdirSync(pendingRoot()).filter(file => /^[a-zA-Z0-9-]+\.json$/.test(file))) {
try {
const record = JSON.parse(readFileSync(join(pendingRoot(), file), 'utf8'))
if (getJob(record.id)) continue
const job = restoreMusicJob(record)
job.yueGp = false
job.yue2 = true
job.message = 'Reconnecting to YuE2'
void watch(job)
} catch { /* Preserve invalid records for diagnosis. */ }
}
}
+7
View File
@@ -0,0 +1,7 @@
/** ASCII header fallback plus UTF-8 filename; titles may contain emoji or smart punctuation. */
export function inlineFilename(name) {
const clean=String(name).replace(/[\r\n\x00-\x1f\x7f]/g,'').toWellFormed()
const ascii=clean.replace(/[^\x20-\x7e]/g,'_').replace(/["\\]/g,'_') || 'video.mp4'
const encoded=encodeURIComponent(clean || 'video.mp4').replace(/['()*]/g,c=>'%'+c.charCodeAt(0).toString(16).toUpperCase())
return `inline; filename="${ascii}"; filename*=UTF-8''${encoded}`
}
+66
View File
@@ -0,0 +1,66 @@
export const CAPTION_STYLES = ['descriptive', 'klein_prompt', 'delta', 'tags']
export const DELTA_LOCKED_LOOK =
'short light-brown bob, pink tank, light blue shorts, open green Slytherin robe with red-gold trim, barefoot'
export function captionStylePrompt(style) {
switch (style) {
case 'klein_prompt':
return 'Write one self-contained image generation prompt for Flux Klein. Include subject, pose, clothing, setting, lighting, and camera. No preamble or quotes.'
case 'delta':
return `The character's locked baseline look is: ${DELTA_LOCKED_LOOK}. Describe ONLY what changed from that baseline in this image. Do not restate the locked look.`
case 'tags':
return 'List comma-separated descriptive tags for this image. Tags only, no sentences.'
case 'descriptive':
default:
return 'Describe this image in clear, detailed prose. Cover subject, appearance, clothing, pose, setting, and lighting.'
}
}
export function validateCaptionRequest(raw) {
const fail = message => { throw Object.assign(new Error(message), { statusCode: 400 }) }
const body = raw || {}
const style = String(body.captionStyle || body.style || 'descriptive')
if (!CAPTION_STYLES.includes(style)) fail('Unknown caption style.')
const folderId = String(body.folderId || '')
if (!folderId) fail('Choose a folder.')
const stillId = String(body.stillId || body.imageAId || '')
const sourcePath = String(body.sourcePath || '')
if (!stillId && !sourcePath) fail('Choose a still to describe.')
return {
mode: 'caption',
folderId,
stillId,
sourcePath,
captionStyle: style,
promptSections: { action: `Describe · ${style}` },
compiledPrompt: captionStylePrompt(style),
engine: 'caption',
imageStyles: { positive: [], negative: [] },
hideOutput: false,
lockFace: false,
lockOutfit: false,
hideInput: false,
identityStillId: '',
imageAId: stillId,
imageBId: '',
maskId: '',
startClipId: '',
endStillId: '',
guides: [],
batch: [],
settings: {
aspect: 'auto',
fps: 24,
width: 0,
height: 0,
steps: 1,
cfg: 0,
seedMode: 'random',
seed: 0,
duration: 0,
turbo: true,
loraStack: []
}
}
}
+30 -13
View File
@@ -1,3 +1,5 @@
import {normalizeStyles,stylePrompt} from './styles.mjs'
import { queueSeeds } from './seed.mjs'
import { sectionsFrom, compileSections } from './prompt.mjs' import { sectionsFrom, compileSections } from './prompt.mjs'
export { sectionsFrom } from './prompt.mjs' export { sectionsFrom } from './prompt.mjs'
export const cameraFamilies = ['static','dolly in','dolly out','pan left','pan right','tilt up','tilt down','zoom in','zoom out','orbit left','orbit right'] export const cameraFamilies = ['static','dolly in','dolly out','pan left','pan right','tilt up','tilt down','zoom in','zoom out','orbit left','orbit right']
@@ -9,7 +11,15 @@ export function appearanceLock({lockFace = true, lockOutfit = true} = {}) {
return 'The start frame is the only appearance reference for person and clothing, as well as pose, place, and camera. Do not use the hero as an appearance reference.' return 'The start frame is the only appearance reference for person and clothing, as well as pose, place, and camera. Do not use the hero as an appearance reference.'
} }
export function compilePrompt(value, video = false, locks = {}) { export function compilePrompt(value, video = false, locks = {}) {
return [compileSections(value), video ? appearanceLock(locks) : ''].filter(Boolean).join('\n\n') if (video) return [compileSections(value),appearanceLock(locks)].filter(Boolean).join('\n\n')
const hasImage=!!(locks.imageAId || locks.identityStillId)
const attached=hasImage && locks.engine !== 'krea' && locks.engine !== 'qwen21'
const reference=locks.mode!=='generate' && locks.identityStillId && locks.imageAId && locks.identityStillId!==locks.imageAId ? 'second (hero) reference' : 'image reference'
return [sectionsFrom(value).action, stylePrompt(locks.imageStyles),
attached && locks.lockFace !== false ? `Preserve the person’s face, glasses, hair, and body identity from the ${reference}.` : '',
attached && locks.lockOutfit !== false ? `Preserve the wardrobe from the ${reference}.` : '',
attached && locks.lockOutfit === false ? 'Clothing may follow the source and requested scene; do not restore the hero outfit.' : ''
].filter(Boolean).join('\n\n')
} }
export function restoreInput(item) { export function restoreInput(item) {
const saved = structuredClone(item.studio2 || item) const saved = structuredClone(item.studio2 || item)
@@ -21,35 +31,42 @@ export function validateRequest(raw, xaigen = false) {
const mode = raw.mode || 'generate', engine = raw.engine || 'flux' const mode = raw.mode || 'generate', engine = raw.engine || 'flux'
if (!['generate','edit','compose','refine','iterate','video','extend'].includes(mode)) fail('Unknown task.') if (!['generate','edit','compose','refine','iterate','video','extend'].includes(mode)) fail('Unknown task.')
const video = ['video','extend'].includes(mode) const video = ['video','extend'].includes(mode)
if (!(video ? ['minimax','ltx'] : ['flux','krea']).includes(engine)) fail('Engine does not support this task.') if (!(video ? ['minimax','ltx'] : ['flux','krea','qwen21']).includes(engine)) fail('Engine does not support this task.')
if (engine === 'ltx' && !xaigen) fail('LTX is available only in xAIGen.') if (engine === 'ltx' && !xaigen) fail('LTX is available only in xAIGen.')
if (engine === 'krea' && ['edit','compose','iterate'].includes(mode)) fail('Sticky hero edits require Klein. Krea remains available for Generate and masked Refine; no engine fallback is performed.') if (engine === 'krea' && ['edit','compose','iterate'].includes(mode)) fail('Sticky hero edits require Klein. Krea remains available for Generate and masked Refine; no engine fallback is performed.')
if (engine === 'qwen21' && mode !== 'generate') fail('Qwen 2.1 is T2I in this build.')
if (Array.isArray(raw.batch) && raw.batch.length>30) fail('A batch supports up to 31 shots including the first prompt.') if (Array.isArray(raw.batch) && raw.batch.length>30) fail('A batch supports up to 31 shots including the first prompt.')
if (Array.isArray(raw.guides) && raw.guides.length>8) fail('Use up to eight guide stills.') if (Array.isArray(raw.guides) && raw.guides.length>8) fail('Use up to eight guide stills.')
if (!raw.folderId) fail('Choose a folder.') if (!raw.folderId) fail('Choose a folder.')
if ((video || ['edit','compose','iterate'].includes(mode)) && (raw.lockFace !== false || raw.lockOutfit !== false) && !raw.identityStillId) fail('Choose a sticky hero still.') if ((video || ['edit','compose','iterate'].includes(mode)) && (raw.lockFace !== false || raw.lockOutfit !== false) && !raw.identityStillId && (video || !raw.imageAId)) fail('Choose a sticky hero still.')
if (mode === 'compose' && !raw.imageBId) fail('Compose requires a second still.') if (mode === 'compose' && !raw.imageBId) fail('Compose requires a second still.')
if (mode === 'refine' && !raw.maskId) fail('Refine requires a mask.') if (mode === 'refine' && !raw.maskId) fail('Refine requires a mask.')
if (['edit','compose','refine'].includes(mode) && !raw.imageAId) fail('Choose the source still.') if (['edit','compose','refine'].includes(mode) && !raw.imageAId && !(mode==='edit' && engine==='flux' && raw.identityStillId)) fail('Choose the source still.')
if (mode === 'extend' && !raw.startClipId) fail('Choose the video to extend.') if (mode === 'extend' && !raw.startClipId) fail('Select the clip in the family strip or Previous clip.')
const promptSections = sectionsFrom(raw.promptSections || {action:raw.prompt || ''}) const allSections = sectionsFrom(raw.promptSections || {action:raw.prompt || ''})
const promptSections = video ? allSections : {action:allSections.action}
if (!promptSections.action.trim()) fail('Enter an action prompt.') if (!promptSections.action.trim()) fail('Enter an action prompt.')
const number=(key, fallback, min, max) => { const n=Number(raw.settings?.[key] ?? fallback); if (!Number.isFinite(n)||n<min||n>max) fail(`Invalid ${key}.`); return n } const number=(key, fallback, min, max) => { const n=Number(raw.settings?.[key] ?? fallback); if (!Number.isFinite(n)||n<min||n>max) fail(`Invalid ${key}.`); return n }
for (const key of ['steps','seed']) if (!Number.isInteger(number(key,key==='steps'?24:1,0,2147483647))) fail(`Invalid ${key}.`) const qwen = engine === 'qwen21'
for (const key of ['width','height']) if (number(key,key==='width'?1344:768,256,4096)%32) fail('Dimensions must be multiples of 32.') const defaultSteps = video ? 8 : qwen ? 25 : 24
const defaultCfg = video ? 1.5 : qwen ? 1 : 4
const defaultWidth = qwen ? 1024 : 1344
const defaultHeight = qwen ? 1024 : 768
for (const key of ['steps']) if (!Number.isInteger(number(key,key==='steps'?defaultSteps:1,0,2147483647))) fail(`Invalid ${key}.`)
for (const key of ['width','height']) if (number(key,key==='width'?defaultWidth:defaultHeight,256,4096)%32) fail('Dimensions must be multiples of 32.')
const duration=number('duration',5,1,15) const duration=number('duration',5,1,15)
if (video) { if (video) {
if (promptSections.cameraBeats.slice(0,-1).some(b=>b.durationSec == null)) fail('Only the last camera beat can run to the end.') if (promptSections.cameraBeats.slice(0,-1).some(b=>b.durationSec == null)) fail('Only the last camera beat can run to the end.')
if (promptSections.cameraBeats.reduce((sum,b)=>sum+(b.durationSec || 0),0)>duration) fail('Camera beats exceed the clip duration.') if (promptSections.cameraBeats.reduce((sum,b)=>sum+(b.durationSec || 0),0)>duration) fail('Camera beats exceed the clip duration.')
if (raw.lockFace === false && raw.lockOutfit === false && !raw.imageAId && !raw.startClipId) fail('Choose a start still or previous clip when both identity locks are off.')
} }
const frames=Math.ceil((duration*number('fps',24,1,60)-5)/17)*17+5 const frames=Math.ceil((duration*number('fps',24,1,60)-5)/17)*17+5
return { mode, engine, lockFace:raw.lockFace !== false, lockOutfit:raw.lockOutfit !== false, hideInput:!!raw.hideInput, folderId:String(raw.folderId), identityStillId:raw.identityStillId || '', imageAId:raw.imageAId || '', imageBId:raw.imageBId || '', maskId:raw.maskId || '', startClipId:raw.startClipId || '', endStillId:raw.endStillId || '', const shotSeeds=queueSeeds(raw.settings,1+(Array.isArray(raw.batch)?raw.batch.length:0))
return { mode, engine, shotSeeds, imageStyles:video?{positive:[],negative:[]}:normalizeStyles(raw.imageStyles), hideOutput:!!raw.hideOutput, lockFace:qwen ? false : raw.lockFace !== false, lockOutfit:qwen ? false : raw.lockOutfit !== false, hideInput:!!raw.hideInput, folderId:String(raw.folderId), identityStillId:qwen ? '' : (raw.identityStillId || ''), imageAId:raw.imageAId || '', imageBId:raw.imageBId || '', maskId:raw.maskId || '', startClipId:raw.startClipId || '', endStillId:raw.endStillId || '',
startFrameSource: raw.startClipId ? {kind:'previous-last-frame',clipId:raw.startClipId} : {kind:'still',stillId:raw.imageAId || ''}, startFrameSource: raw.startClipId ? {kind:'previous-last-frame',clipId:raw.startClipId} : {kind:'still',stillId:raw.imageAId || ''},
promptSections, compiledPrompt:compilePrompt(promptSections,video,raw), promptSections, compiledPrompt:compilePrompt(promptSections,video,{...raw,engine,lockFace:qwen?false:raw.lockFace,lockOutfit:qwen?false:raw.lockOutfit}),
settings: { aspect:['auto','16:9','9:16','1:1'].includes(raw.settings?.aspect)?raw.settings.aspect:'auto', fps:number('fps',24,1,60), width:number('width',1344,256,4096),height:number('height',768,256,4096),steps:number('steps',video?8:24,1,100),cfg:number('cfg',video?1.5:4,0,20),seed:number('seed',Math.floor(Math.random()*2147483647),0,2147483647),duration:number('duration',5,1,15),turbo:raw.settings?.turbo !== false, loraStack:Array.isArray(raw.settings?.loraStack)?raw.settings.loraStack:[] }, settings: { aspect:['auto','16:9','9:16','1:1'].includes(raw.settings?.aspect)?raw.settings.aspect:'auto', fps:number('fps',24,1,60), width:number('width',defaultWidth,256,4096),height:number('height',defaultHeight,256,4096),steps:number('steps',defaultSteps,1,100),cfg:number('cfg',defaultCfg,0,20),seedMode:raw.settings?.seedMode==='locked'?'locked':'random',seed:shotSeeds[0],duration:number('duration',5,1,15),turbo:raw.settings?.turbo !== false, loraStack:qwen?[]:(Array.isArray(raw.settings?.loraStack)?raw.settings.loraStack:[]) },
guides: (Array.isArray(raw.guides)?raw.guides:[]).slice(0,8).map(g=>{ if (!g.stillId || !Number.isInteger(g.frame) || g.frame<1 || g.frame>=frames) fail('Each guide requires a still and positive frame index.'); return {stillId:String(g.stillId),frame:g.frame} }), guides: (Array.isArray(raw.guides)?raw.guides:[]).slice(0,8).map(g=>{ if (!g.stillId || !Number.isInteger(g.frame) || g.frame<1 || g.frame>=frames) fail('Each guide requires a still and positive frame index.'); return {stillId:String(g.stillId),frame:g.frame} }),
batch: (Array.isArray(raw.batch)?raw.batch:[]).slice(0,30).map(s=>{const next=sectionsFrom({...promptSections,...(typeof s==='string'?{action:s}:s)});if(!next.action.trim())fail('Every shot needs an action prompt.');return next}) } batch: (Array.isArray(raw.batch)?raw.batch:[]).slice(0,30).map(s=>{const next=sectionsFrom({...promptSections,...(typeof s==='string'?{action:s}:s)});if(!next.action.trim())fail('Every shot needs an action prompt.');return video?next:{action:next.action}}) }
} }
export function scopedFile(file, prefix) { export function scopedFile(file, prefix) {
const clean = s => String(s || '').replaceAll('\\','/') const clean = s => String(s || '').replaceAll('\\','/')
+35
View File
@@ -2,9 +2,20 @@ export function nativeVideoGraph(template, request, files, prefix) {
const appearance = request.lockFace !== false || request.lockOutfit !== false const appearance = request.lockFace !== false || request.lockOutfit !== false
if (appearance && !files.hero) throw new Error('Identity still is required; never substitute the start frame.') if (appearance && !files.hero) throw new Error('Identity still is required; never substitute the start frame.')
const g=structuredClone(template), s=request.settings const g=structuredClone(template), s=request.settings
const textToVideo = request.mode === 'video' && !appearance && !files.start
if (appearance) { g.hero.inputs.image=files.hero; g['155'].inputs['ref_images.ref_image_0']=['hero',0] } if (appearance) { g.hero.inputs.image=files.hero; g['155'].inputs['ref_images.ref_image_0']=['hero',0] }
else { delete g.hero; delete g['155'].inputs['ref_images.ref_image_0'] } else { delete g.hero; delete g['155'].inputs['ref_images.ref_image_0'] }
Object.assign(g['155'].inputs,{prompt:request.compiledPrompt,width:s.width,height:s.height,length:Math.ceil((s.duration*(s.fps || 24)-5)/17)*17+5,ref_image_size:'max'}) Object.assign(g['155'].inputs,{prompt:request.compiledPrompt,width:s.width,height:s.height,length:Math.ceil((s.duration*(s.fps || 24)-5)/17)*17+5,ref_image_size:'max'})
if (textToVideo) {
// Studio 1 T2V: MiniMaxH3ImageToVideo with first_frame disconnected (no empty placeholder still).
g['155'].class_type = 'MiniMaxH3ImageToVideo'
g['155']._meta = { title: 'MiniMax H3 Text to Video' }
delete g['155'].inputs['ref_images.ref_image_0']
delete g['155'].inputs.ref_image_size
delete g['155'].inputs.audio_vae
delete g['155'].inputs.first_frame
delete g['155'].inputs.last_frame
}
let conditioning=['155',0] let conditioning=['155',0]
for (const [name,image,frame] of [['start',files.start,0],['end',files.end,-1],...(files.guides||[]).map((f,i)=>[`guide${i}`,f.image,f.frame])]) { for (const [name,image,frame] of [['start',files.start,0],['end',files.end,-1],...(files.guides||[]).map((f,i)=>[`guide${i}`,f.image,f.frame])]) {
if (!image) continue if (!image) continue
@@ -34,3 +45,27 @@ export function attachHeroReference(g, image) {
} }
return g return g
} }
export function sampleKleinSource(graph, denoise = .65) {
if (!graph['11'] || !graph['19'] || !graph['17']) throw new Error('Klein source sampling nodes are missing.')
graph['19'].inputs.latent_image=['11',0]
delete graph['14']
const steps=graph['17'].inputs.steps
graph['17'].inputs.steps=Math.ceil(steps/denoise)
graph.source_sigmas={class_type:'SplitSigmasDenoise',inputs:{sigmas:['17',0],denoise}}
graph['19'].inputs.sigmas=['source_sigmas',1]
return graph
}
/** The shared image builders have their own defaults and megapixel scalers.
* Studio 2 must send its already resolved canvas size through those nodes. */
export function applyResolvedImageSize(graph, {width,height}) {
for (const node of Object.values(graph)) {
if (node.class_type === 'ImageScaleToTotalPixels') {
node.class_type='ImageScale'
node.inputs={image:node.inputs.image,upscale_method:'lanczos',width,height,crop:'disabled'}
} else {
if (typeof node.inputs?.width === 'number') node.inputs.width=width
if (typeof node.inputs?.height === 'number') node.inputs.height=height
}
}
return graph
}
+58 -14
View File
@@ -1,31 +1,75 @@
import {restoreLoras,loraUse} from './lora-memory.mjs'
import { restoreInput } from './contracts.mjs' import { restoreInput } from './contracts.mjs'
export const aspectOptions = [{id:'auto',label:'Automatic',hint:'Match start or hero still'},{id:'16:9',label:'16:9',hint:'1344 × 768 · padded'},{id:'9:16',label:'9:16',hint:'768 × 1344 · padded'},{id:'1:1',label:'1:1',hint:'960 × 960 · padded'}] export const aspectOptions = [{id:'auto',label:'Automatic',hint:'Match start or hero still'},{id:'16:9',label:'16:9',hint:'1344 × 768 · padded'},{id:'9:16',label:'9:16',hint:'768 × 1344 · padded'},{id:'1:1',label:'1:1',hint:'960 × 960 · padded'}]
export function resolveSize(aspect, source, fallback = {width:1344,height:768}) { export function resolveQwen21Size(aspect) {
const presets = {'16:9':[1024,576],'9:16':[576,1024],'1:1':[1024,1024]}
if (presets[aspect]) return {width:presets[aspect][0],height:presets[aspect][1]}
return {width:1024,height:1024}
}
export function resolveSize(aspect, source, fallback = {width:1344,height:768}, engine = '') {
if (engine === 'qwen21') return resolveQwen21Size(aspect)
const presets = {'16:9':[1344,768],'9:16':[768,1344],'1:1':[960,960]} const presets = {'16:9':[1344,768],'9:16':[768,1344],'1:1':[960,960]}
if (presets[aspect]) return {width:presets[aspect][0],height:presets[aspect][1]} if (presets[aspect]) return {width:presets[aspect][0],height:presets[aspect][1]}
if (!source?.width || !source?.height) return {width:fallback.width || 1344,height:fallback.height || 768} if (!source?.width || !source?.height) return {width:fallback.width || 1344,height:fallback.height || 768}
const factor = Math.min(1344/Math.max(source.width,source.height),768/Math.min(source.width,source.height)) const factor = Math.min(1344/Math.max(source.width,source.height),768/Math.min(source.width,source.height))
return {width:Math.max(256,Math.round(source.width*factor/32)*32),height:Math.max(256,Math.round(source.height*factor/32)*32)} return {width:Math.max(256,Math.round(source.width*factor/32)*32),height:Math.max(256,Math.round(source.height*factor/32)*32)}
} }
export function bindClip(form, clip) { export function bindClip(form, clip, xaigen = false, memory) {
if (!clip?.id) return form if (!clip?.id) return form
const saved = clip.studio2 || {}, settings = saved.settings || clip.settings || clip const saved = clip.studio2 || clip.job || {}
const next = {...form, startClipId:clip.id, settings:{...form.settings}} const settings = saved.settings || clip.settings || {}
for (const key of ['duration','aspect','width','height','fps','loraStack','turbo','steps','cfg']) { const next = {...form, startClipId:clip.id, mode:'extend', settings:{...form.settings}}
if (settings[key] != null) next.settings[key] = structuredClone(settings[key]) next.settings.aspect=['auto','16:9','9:16','1:1'].includes(settings.aspect)?settings.aspect:(form.settings?.aspect || 'auto')
for (const key of ['duration','width','height','fps','loraStack','turbo','steps','cfg']) {
if (settings[key] == null) continue
try { next.settings[key] = typeof structuredClone === 'function' ? structuredClone(settings[key]) : JSON.parse(JSON.stringify(settings[key])) }
catch { next.settings[key] = settings[key] }
} }
next.engine = saved.engine || clip.engine || 'minimax' next.engine = xaigen && (saved.engine || clip.engine)==='ltx' ? 'ltx' : 'minimax'
if (saved.identityStillId) next.identityStillId = saved.identityStillId if (saved.identityStillId) next.identityStillId = saved.identityStillId
// Missing legacy metadata must not reset a user's current choice.
for (const key of ['lockFace','lockOutfit']) if (typeof saved[key] === 'boolean') next[key] = saved[key] for (const key of ['lockFace','lockOutfit']) if (typeof saved[key] === 'boolean') next[key] = saved[key]
return next return restoreLoras(next,settings.loraStack,xaigen,memory)
} }
export function restoreAsset(form, item) { export function restoreAsset(form, item, xaigen = false, memory) {
if (item.kind==='clip' || item.studio2?.kind==='video') {
const next=bindClip(form,item,xaigen,memory)
const seed=item.studio2?.settings?.seed ?? item.settings?.seed ?? item.seed
if (seed != null) next.settings.seed=seed
next.settings.seedMode='random'
return {...next,mode:'extend'}
}
const restored = restoreInput(item) const restored = restoreInput(item)
const next = {...form,...restored,settings:{...form.settings,...restored.settings}} const next = {...form,...restored,settings:{...form.settings,...restored.settings}}
return item.kind === 'clip' || item.studio2?.kind === 'video' ? {...bindClip(next,item),mode:'extend'} : next next.settings.seedMode='random'
next.engine=restored.engine==='qwen21'?'qwen21':restored.engine==='krea'?'krea':'flux'
next.mode=['generate','edit','compose','refine','iterate'].includes(restored.mode)?restored.mode:'edit'
next.imageAId=item.id
next.startClipId=''
next.settings.aspect=item.studio2?.settings?.aspect || item.settings?.aspect || 'auto'
return restoreLoras(next,item.studio2?.settings?.loraStack ?? item.settings?.loraStack,xaigen,memory)
} }
export function extendSelected(form, clip) { export function extendSelected(form, clip, xaigen = false, memory) {
// Switching task on the already bound clip must preserve the user's overrides. const next = { ...form, settings:{...form.settings}, mode: 'extend', engine: xaigen && form.engine === 'ltx' ? 'ltx' : 'minimax' }
return {...(clip?.id && form.startClipId !== clip.id ? bindClip(form,clip) : form),mode:'extend'} const target = clip?.id ? clip : null
if (!target?.id) return restoreLoras(next,loraUse(form)===loraUse(next)?form.settings?.loraStack:[],xaigen,memory)
// Already bound to this clip: switching task must not stomp the user's overrides
// (locks, CFG, etc.) by re-copying the clip's saved settings over them.
if (form.startClipId === target.id) return restoreLoras(next,loraUse(form)===loraUse(next)?form.settings?.loraStack:[],xaigen,memory)
try { return bindClip(next, target, xaigen, memory) }
catch { return { ...next, startClipId: target.id } }
}
export function changeSource(form) {
return form.mode==='extend' ? form : {...form,startClipId:'',settings:{...form.settings,aspect:'auto'}}
}
export function useAsHero(form, item) { return {...form,identityStillId:item.id} }
export function useAsSource(form, item) { return {...form,imageAId:item.id} }
export function restoreAll(form,item,xaigen=false,memory) {
const restored=restoreInput(item),next={...form,...restored,settings:{...form.settings,...restored.settings,seedMode:'random'}}
if(item.kind==='clip') return {...bindClip(next,item,xaigen,memory),mode:'extend'}
next.mode=restored.mode==='iterate'?'edit':restored.mode || 'edit'
next.engine=restored.engine==='qwen21'?'qwen21':restored.engine==='krea'?'krea':'flux'
next.imageAId=item.id
next.startClipId=''
return restoreLoras(next,item.studio2?.settings?.loraStack ?? item.settings?.loraStack,xaigen,memory)
} }
+37
View File
@@ -0,0 +1,37 @@
export function kleinIdentityPlan(request) {
if(request.engine!=='flux' || ['video','extend'].includes(request.mode)) return null
const sourceId=request.mode==='generate' ? request.identityStillId || request.imageAId || '' : request.imageAId || request.identityStillId || ''
const mode=['generate','iterate'].includes(request.mode) ? (sourceId?'edit':'generate') : request.mode
return {sourceId,mode}
}
/** Follow the live conditioning chain, including optional hero/compose references. */
function hasSourceReference(graph, link, referenceId) {
const seen=new Set()
while(Array.isArray(link) && !seen.has(String(link[0]))) {
const id=String(link[0]);seen.add(id)
const node=graph[id]
if(id===referenceId) return node?.class_type==='ReferenceLatent' && node.inputs.latent?.[0]==='11' && graph['11']?.class_type==='VAEEncode'
link=node?.inputs?.conditioning
}
return false
}
export function assertKleinSample(graph, sourceId) {
const sampler=graph['19']?.inputs
const latent=graph[sampler?.latent_image?.[0]]
if(latent?.class_type==='EmptyFlux2LatentImage') {
if(!sourceId) return 'empty'
const guider=graph[sampler.guider?.[0]]?.inputs
if(hasSourceReference(graph,guider?.positive,'12') && hasSourceReference(graph,guider?.negative,'13')) return 'empty+reference'
throw new Error('Klein source reference conditioning is missing from the sampler.')
}
// Refine retains its dedicated masked-source workflow.
if(latent?.class_type==='SetLatentNoiseMask' && latent.inputs.samples?.[0]==='11' && graph['11']?.class_type==='VAEEncode') return 'masked source'
throw new Error('Klein edit must sample empty latent with source references, not the source image latent.')
}
export function applyKleinIdentity(graph, plan) {
const sampleLatent=assertKleinSample(graph,plan.sourceId)
return {sampleLatent,sampleDenoise:sampleLatent==='masked source' ? graph['17']?.inputs.denoise ?? null : null}
}
+13
View File
@@ -0,0 +1,13 @@
export async function cachedLoras({read,write,live,fresh: _fresh=false}) {
const cached=read() || {image:[],video:[]};
try {
// Always live-scan. Disk is asleep/empty fallback only — never a stale snapshot that skips Comfy.
const listed=await live();
const image=Array.isArray(listed?.image)?listed.image:[];
const video=Array.isArray(listed?.video)?listed.video:[];
if (!image.length && !video.length) return cached;
const next={image,video};
write(next);
return next;
} catch { return cached; }
}
+45
View File
@@ -0,0 +1,45 @@
import {filterLoraStackForUse} from '../../utils/loras.ts'
export function loraUse(form) {
const video=['video','extend'].includes(form.mode)
if (video) return form.engine==='ltx'?'ltx':'minimax'
if (form.engine==='qwen21') return 'qwen21'
return form.engine==='krea'?'krea':'flux'
}
// One controller per bench. Storage is optional (SSR/private browsing still works).
export function createLoraMemory(xaigen, storage) {
const stacks={}
let active=null
const key=`studio2-lora-stacks:${xaigen?'xaigen':'aigen'}`
try { Object.assign(stacks,JSON.parse(storage?.getItem(key)||'{}')) } catch {}
const load=use=>filterLoraStackForUse(stacks[use],use,xaigen)
const save=(use,stack)=>{
stacks[use]=filterLoraStackForUse(stack,use,xaigen)
try { storage?.setItem(key,JSON.stringify(stacks)) } catch {}
}
return {
sync(form) {
const use=loraUse(form)
if(active!==use) {
// Every stack edit is remembered by the bench watcher before a later switch.
active=use
form.settings.loraStack=load(use)
}
save(use,form.settings.loraStack)
},
restore(form,stack) {
const use=loraUse(form)
form.settings.loraStack=stack?.length ? filterLoraStackForUse(stack,use,xaigen) : load(use)
active=use
save(use,form.settings.loraStack)
return form
}
}
}
export function restoreLoras(form, stack, xaigen=false, memory) {
if(memory) return memory.restore(form,stack)
form.settings.loraStack=filterLoraStackForUse(stack,loraUse(form),xaigen)
return form
}
+16
View File
@@ -0,0 +1,16 @@
const terminal = new Set(['complete','failed','cancelled'])
const running = new Set(['waking','submitting','rendering','saving'])
export function queueJobs(jobs) {
return jobs.filter(j=>running.has(j.state)||['waiting','queued','held'].includes(j.state)).sort((a,b)=>Number(running.has(b.state))-Number(running.has(a.state)) || (a.queuedAt||0)-(b.queuedAt||0))
}
export function jobView(record, queue, live) {
const statuses={complete:'complete',error:'failed',cancelled:'cancelled'}
const state=statuses[queue?.status] || (!live?.saving && statuses[live?.status]) || (queue?.status==='held'?'held':record.state)
const finishedAt=terminal.has(state)?record.finishedAt || (statuses[queue?.status]?queue.updatedAt:0) || (state==='complete'?record.savedAt:0) || record.updatedAt || record.startedAt || record.queuedAt:undefined
return {...record,state,finishedAt,error:record.error || queue?.lastError || live?.error}
}
export function elapsedSeconds(job, now) {
const start=job.startedAt || job.queuedAt
const end=terminal.has(job.state)?job.finishedAt || job.savedAt || job.updatedAt || start:now
return Number.isFinite(start)&&Number.isFinite(end)?Math.max(0,Math.floor((end-start)/1000)):0
}
+17
View File
@@ -0,0 +1,17 @@
let previous
export function randomSeed() {
const bytes = new Uint32Array(1)
globalThis.crypto.getRandomValues(bytes)
let seed = 2 + bytes[0] % 2147483646
if (seed === previous) seed = seed === 2147483647 ? 2 : seed + 1
previous = seed
return seed
}
export function queueSeeds(settings = {}, count = 1) {
if (settings.seedMode === 'locked') {
const seed=Number(settings.seed)
if (!Number.isInteger(seed) || seed<0 || seed>2147483647) throw Object.assign(new Error('Invalid seed.'),{statusCode:400})
return Array(count).fill(seed)
}
return Array.from({length:count},()=>randomSeed())
}
+20
View File
@@ -0,0 +1,20 @@
export function gpuStatus(jobs, health, upscales=[], musicJobs=[], captionJobs=[]) {
if (!health) return {label:'Status unavailable',tone:'unknown',detail:'Checking GPU connection…'}
const upscale=upscales.find(j=>j.status==='running')
if(upscale)return {label:'Upscaling',tone:'busy',detail:upscale.name || 'Video upscale'}
const music=musicJobs.find(j=>['waiting','running','held'].includes(j.status) || j.live?.status==='running' || j.live?.musicActivity?.running)
if(music){
const stage=music.live?.message || music.live?.stage || music.waitReason || 'YuE2'
return {label:music.status==='waiting'?'Queued':'Generating',tone:'busy',detail:String(stage)}
}
const caption=captionJobs.find(j=>['waiting','waking','submitting','rendering','saving'].includes(j.state) || j.state==='queued' || j.state==='held')
if(caption)return {label:caption.state==='waiting'?'Queued':'Describing',tone:'busy',detail:caption.request?.captionStyle || 'Caption'}
if(health.caption?.busy)return {label:'Describing',tone:'busy',detail:'Caption model is using the GPU'}
if(health.yue2?.busy)return {label:'Generating',tone:'busy',detail:'YuE2 is using the GPU'}
const job=jobs.find(j=>['waking','submitting','rendering','saving'].includes(j.state))
if(job)return {label:{waking:'Waking up',submitting:'Submitting',rendering:'Generating',saving:'Saving'}[job.state],tone:'busy',detail:job.request?.promptSections?.action || 'Generation in progress'}
if(health.comfy?.busy || health.imageComfy?.busy)return {label:'Busy',tone:'busy',detail:'GPU is working on another job'}
if(health.comfy?.ok || health.imageComfy?.ok)return {label:'Ready',tone:'ready',detail:'No job running'}
if(health.comfy?.asleep)return {label:'Sleeping',tone:'sleeping',detail:'Wakes when a job starts'}
return {label:'Offline',tone:'unknown',detail:'GPU connection unavailable'}
}
+5
View File
@@ -0,0 +1,5 @@
export const styleGroups={Render:['Photorealistic','Pixar','3D Disney','Comic','Watercolor','Oil painting'],Quality:['Realistic skin','Skin pores','Fine detail','Natural texture'],Lighting:['Golden hour','Soft studio','Window light','Overcast','Cinematic lighting']}
export function normalizeStyles(value={}) { const known=Object.values(styleGroups).flat();const positive=[...new Set((value.positive||[]).filter(s=>known.includes(s)))];return {positive,negative:[...new Set((value.negative||[]).filter(s=>known.includes(s)&&!positive.includes(s)))]} }
export function toggleStyle(value,chip,negative=false) { const next=normalizeStyles(value),key=negative?'negative':'positive',other=negative?'positive':'negative';next[other]=next[other].filter(s=>s!==chip);next[key]=next[key].includes(chip)?next[key].filter(s=>s!==chip):[...next[key],chip];return next }
export function stylePrompt(value,negative=false){return normalizeStyles(value)[negative?'negative':'positive'].join(', ')}
export function parseBatch(value){return String(value||'').split(/\r?\n\s*\r?\n/).map(s=>s.trim()).filter(Boolean)}
+8
View File
@@ -0,0 +1,8 @@
export function applyUploadedStill(form, response, target) {
const still=response?.still
if (!still?.id) throw new Error('The upload did not return a saved still. Please try again.')
const next={...form,settings:{...form.settings}}
if (target==='hero') next.identityStillId=still.id
else {next.imageAId=still.id;if(next.mode!=='extend'){next.startClipId='';next.settings.aspect='auto'}}
return next
}
+22
View File
@@ -0,0 +1,22 @@
export function folderAssets(library, jobs, folderId) {
const metadata=new Map(jobs.flatMap(j=>j.outputs || []).map(a=>[a.id,a.studio2]))
const kinds=[['still','stills'],['clip','clips'],['track','tracks']]
return kinds.flatMap(([kind,key])=>(library[key] || []).filter(a=>a.id && a.folderId===folderId && a.hasVideo!==false && a.available!==false).map(a=>({...a,kind,studio2:a.studio2 || metadata.get(a.id)}))).sort((a,b)=>(a.createdAt || a.studio2?.savedAt || 0)-(b.createdAt || b.studio2?.savedAt || 0))
}
export function newestOutput(jobs, seen, folderId) {
return jobs.filter(j=>j.request.folderId===folderId).flatMap(j=>j.outputs || []).filter(a=>!seen.has(a.id)).sort((a,b)=>(a.studio2?.savedAt || 0)-(b.studio2?.savedAt || 0)).at(-1) || null
}
export function reconcileSelection(selected, assets) { return assets.find(a=>a.id===selected?.id) || assets.at(-1) || null }
export function activeJobs(jobs) { return jobs.filter(j=>['waking','submitting','rendering','saving'].includes(j.state)) }
export function previewHidden(asset, revealed, xaigen) {
if (!xaigen || !asset?.id) return false
return !revealed?.has?.(asset.id)
}
export function selectResult(state, asset) { return {...state,selected:asset} }
export function assetLabel(asset) {
if (!asset) return 'None'
if (asset.name || asset.originalName || asset.filename) return asset.name || asset.originalName || asset.filename
if (asset.kind==='track') return 'Track '+asset.id.slice(0,8)
if (asset.kind==='clip') return 'Clip '+asset.id.slice(0,8)
return 'Still '+asset.id.slice(0,8)
}
+139
View File
@@ -0,0 +1,139 @@
import test from 'node:test'
import assert from 'node:assert/strict'
import { EventEmitter } from 'node:events'
import { PassThrough } from 'node:stream'
import { mkdtempSync, writeFileSync, readFileSync, mkdirSync, existsSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { createCaptionHost, validateCaptionHostRequest, captionModelFiles } from '../scripts/caption-host.mjs'
import { validateCaptionRequest, captionStylePrompt, CAPTION_STYLES, DELTA_LOCKED_LOOK } from '../shared/studio2/caption.mjs'
test('caption styles and delta lock text', () => {
assert.deepEqual(CAPTION_STYLES, ['descriptive', 'klein_prompt', 'delta', 'tags'])
assert.match(captionStylePrompt('delta'), new RegExp(DELTA_LOCKED_LOOK))
assert.match(captionStylePrompt('klein_prompt'), /Flux Klein/)
assert.match(captionStylePrompt('tags'), /comma-separated/)
})
test('validateCaptionRequest requires folder + still', () => {
assert.throws(() => validateCaptionRequest({}), /folder/)
assert.throws(() => validateCaptionRequest({ folderId: 'f1' }), /still/)
assert.throws(() => validateCaptionRequest({ folderId: 'f1', stillId: 's1', captionStyle: 'nope' }), /style/)
const ok = validateCaptionRequest({ folderId: 'f1', stillId: 's1', captionStyle: 'descriptive' })
assert.equal(ok.mode, 'caption')
assert.equal(ok.captionStyle, 'descriptive')
assert.equal(ok.lockFace, false)
assert.equal(ok.lockOutfit, false)
})
test('host validates job body', () => {
assert.throws(() => validateCaptionHostRequest({ id: 'bad', style: 'descriptive' }))
assert.deepEqual(
validateCaptionHostRequest({ id: 'caption-job-123456', style: 'tags' }),
{ id: 'caption-job-123456', style: 'tags', imagePath: '' }
)
})
function fixture() {
const root = mkdtempSync(join(tmpdir(), 'aigen-caption-'))
const models = join(root, 'models')
mkdirSync(models)
writeFileSync(join(models, captionModelFiles.model), 'model')
writeFileSync(join(models, captionModelFiles.mmproj), 'mmproj')
const llama = join(root, 'llama-server.exe')
writeFileSync(llama, '')
const dataDir = join(root, 'jobs')
const child = new EventEmitter()
child.pid = 4242
child.stdout = new PassThrough()
child.stderr = new PassThrough()
child.kill = () => { setImmediate(() => child.emit('close', 0)); return true }
child.exitCode = null
const calls = []
let fetchCount = 0
const fetchImpl = async (url, init) => {
fetchCount++
if (String(url).endsWith('/health')) return { ok: true }
if (String(url).includes('/v1/chat/completions')) {
assert.match(JSON.stringify(JSON.parse(init.body)), /Describe this image|Flux Klein|locked baseline|comma-separated/)
return {
ok: true,
json: async () => ({ choices: [{ message: { content: 'a warm caption' } }] })
}
}
throw new Error(`unexpected fetch ${url}`)
}
const host = createCaptionHost({
modelsDir: models,
llamaServer: llama,
dataDir,
prepare: async () => {},
leaseValid: () => true,
spawnProcess: (...args) => { calls.push(args); setImmediate(() => {}); return child },
fetchImpl
})
return { host, child, calls, root, dataDir, fetchCount: () => fetchCount }
}
test('host loads llama-server, captions once, and unloads', async () => {
const f = fixture()
const jobDir = join(f.dataDir, 'caption-job-123456')
mkdirSync(jobDir, { recursive: true })
writeFileSync(join(jobDir, 'input.png'), Buffer.alloc(64, 1))
const started = f.host.start({ id: 'caption-job-123456', style: 'descriptive' }, 'lease')
// Let the host reach waitForServer / chat completion.
await new Promise(r => setTimeout(r, 50))
f.child.emit('spawn')
const result = await started
assert.equal(result.status, 'complete')
assert.equal(result.text, 'a warm caption')
assert.equal(f.calls.length, 1)
const [, args] = f.calls[0]
assert.ok(args.includes('--mmproj'))
assert.ok(args.includes('-m'))
assert.equal(f.host.read('caption-job-123456').text, 'a warm caption')
// busy may briefly remain true during the post-unload hold window
assert.ok(typeof f.host.busy() === 'boolean')
})
test('host rejects a second job while busy', async () => {
const f = fixture()
const dataDir = join(f.root, 'jobs2')
let resolveHealth
const healthGate = new Promise(resolve => { resolveHealth = resolve })
const hung = createCaptionHost({
modelsDir: join(f.root, 'models'),
llamaServer: join(f.root, 'llama-server.exe'),
dataDir,
prepare: async () => {},
leaseValid: () => true,
spawnProcess: () => {
const child = new EventEmitter()
child.pid = 99
child.stdout = new PassThrough()
child.stderr = new PassThrough()
child.kill = () => { setImmediate(() => child.emit('close', 1)); return true }
child.exitCode = null
return child
},
fetchImpl: async (url) => {
if (String(url).endsWith('/health')) {
await healthGate
return { ok: true }
}
return { ok: true, json: async () => ({ choices: [{ message: { content: 'x' } }] }) }
}
})
mkdirSync(join(dataDir, 'caption-job-aaaaaa'), { recursive: true })
writeFileSync(join(dataDir, 'caption-job-aaaaaa', 'input.png'), Buffer.alloc(64, 1))
const first = hung.start({ id: 'caption-job-aaaaaa', style: 'tags' }, 'lease')
await new Promise(r => setTimeout(r, 20))
await assert.rejects(hung.start({ id: 'caption-job-bbbbbb', style: 'tags' }, 'lease'), /already running/)
await hung.cancel('caption-job-aaaaaa')
resolveHealth()
await assert.rejects(first)
})
test('POST /caption sync shape is documented on host module', () => {
assert.equal(typeof createCaptionHost({ prepare: async () => {}, leaseValid: () => true }).captionOnce, 'function')
})
+83
View File
@@ -0,0 +1,83 @@
import test from 'node:test'
import assert from 'node:assert/strict'
import {readFileSync} from 'node:fs'
import ts from 'typescript'
import {folderAssets,reconcileSelection,activeJobs,previewHidden} from '../shared/studio2/viewer.mjs'
import {useAsHero,useAsSource,restoreAll} from '../shared/studio2/input.mjs'
import {toggleStyle,parseBatch,stylePrompt} from '../shared/studio2/styles.mjs'
import {compilePrompt,validateRequest} from '../shared/studio2/contracts.mjs'
import {cachedLoras} from '../shared/studio2/lora-cache.mjs'
test('deleted library files never return from job outputs; selection falls back or clears',()=>{
const deleted={id:'gone',folderId:'f'},keep={id:'keep',folderId:'f',createdAt:2};const jobs=[{request:{folderId:'f'},outputs:[deleted]}]
let assets=folderAssets({stills:[keep],clips:[]},jobs,'f');assert.deepEqual(assets.map(a=>a.id),['keep']);assert.equal(reconcileSelection(deleted,assets).id,'keep');assert.equal(reconcileSelection(deleted,[]),null)
})
test('bench includes only active execution phases and clear finished retains internal cleanup records',()=>{
assert.deepEqual(activeJobs(['waiting','waking','submitting','rendering','saving','complete','cancelled','failed'].map(state=>({state}))).map(j=>j.state),['waking','submitting','rendering','saving'])
const rows=[{id:'done',state:'complete',request:{folderId:'f'},cleanupPending:[{}]},{id:'active',state:'rendering',request:{folderId:'f'}}],saved=[]
const src=readFileSync(new URL('../server/api/studio-2/clear-finished.post.ts',import.meta.url),'utf8').replace(/^import .*;\r?\n/gm,'').replace('export default ','const handler=')
const js=ts.transpileModule(src,{compilerOptions:{target:ts.ScriptTarget.ES2022}}).outputText
const fn=new Function('defineEventHandler','assertLibraryOwner','listStudioJobs','records','assertFolderAccess','saveRecord',js+';return handler')(f=>f,()=>({owner:'o'}),()=>[],()=>rows,()=>{},r=>saved.push(r))
assert.equal(fn({}).cleared,1);assert.equal(rows[0].dismissed,true);assert.equal(rows[0].cleanupPending.length,1);assert.equal(rows[1].dismissed,undefined)
})
test('hidden xAIGen canvas and family render no image or video elements',async()=>{
const revealed=new Set()
assert.equal(previewHidden({id:'a',role:'input'},revealed,true),true)
assert.equal(previewHidden({id:'a',role:'input'},new Set(['a']),true),false)
assert.equal(previewHidden({id:'a'},revealed,false),false)
const src=readFileSync(new URL('../components/studio2/Bench.vue',import.meta.url),'utf8')
assert.match(src,/aria-label="Current family outputs"/)
assert.match(src,/preview hidden/)
assert.doesNotMatch(src,/Hide input stills/)
assert.doesNotMatch(src,/Hide output preview/)
})
test('cached LoRAs survive sleeping GPU and failed explicit refresh',async()=>{
const cache={image:['Klein/portrait.safetensors'],video:['motion.safetensors']};let queried=0
const opts={read:()=>cache,write:()=>assert.fail('Do not erase cache'),live:async()=>{queried++;throw Error('Comfy asleep')}}
assert.deepEqual(await cachedLoras(opts),cache);assert.equal(queried,1)
assert.deepEqual(await cachedLoras({...opts,fresh:true}),cache);assert.equal(queried,2)
})
test('bench load and Refresh always live-scan and replace disk so new Comfy names appear',async()=>{
let store={image:['old.safetensors'],video:['old-vid.safetensors']}
const next=await cachedLoras({
read:()=>store,
write:value=>{store=value},
live:async()=>({image:['old.safetensors','klein-pokemon_anime_style_klein9b.safetensors'],video:['old-vid.safetensors']}),
fresh:false
})
assert.ok(next.image.includes('klein-pokemon_anime_style_klein9b.safetensors'))
assert.deepEqual(store.image,next.image)
const removed=await cachedLoras({
read:()=>store,
write:value=>{store=value},
live:async()=>({image:['klein-pokemon_anime_style_klein9b.safetensors'],video:[]}),
fresh:true
})
assert.deepEqual(removed,{image:['klein-pokemon_anime_style_klein9b.safetensors'],video:[]})
})
test('bench mounts and Refresh LoRAs request a live scan',()=>{
const src=readFileSync(new URL('../components/studio2/Bench.vue',import.meta.url),'utf8')
assert.match(src,/void loadLoras\(true\)/)
assert.match(src,/loadLoras\(true\)/)
assert.match(src,/\/api\/studio-2\/loras'\+\(fresh\?'\?fresh=1':''\)/)
})
test('source/hero actions are independent and full input restores structured settings',()=>{
const form={identityStillId:'hero',imageAId:'source',promptSections:{action:'old'},settings:{}}
assert.equal(useAsHero(form,{id:'new'}).imageAId,'source');assert.equal(useAsSource(form,{id:'new'}).identityStillId,'hero')
const saved={mode:'edit',identityStillId:'h',imageAId:'s',imageStyles:{positive:['Comic'],negative:['Skin pores']},promptSections:{action:'New',cameraBeats:[{moves:['hold']}]},settings:{steps:24,loraStack:[{name:'example.safetensors',strengthModel:.4,strengthClip:.6}]}}
const restored=restoreAll(form,{id:'result',studio2:saved});assert.equal(restored.imageAId,'result');assert.equal(restored.identityStillId,'h');assert.deepEqual(restored.settings.loraStack,saved.settings.loraStack);assert.deepEqual(restored.imageStyles,saved.imageStyles);assert.equal(restored.promptSections.action,'New');const clip=restoreAll(form,{id:'clip',kind:'clip',studio2:{...saved,mode:'video',engine:'minimax'}});assert.equal(clip.startClipId,'clip');assert.equal(clip.mode,'extend');assert.equal(clip.promptSections.action,'New')
})
test('image chips compile positive and negative independently; video chips do not leak',()=>{
let styles=toggleStyle({},'Comic');styles=toggleStyle(styles,'Skin pores',true);styles=toggleStyle(styles,'Comic',true);assert.equal(stylePrompt(styles),'');assert.match(stylePrompt(styles,true),/Comic/)
styles=toggleStyle(styles,'Photorealistic');const q=validateRequest({mode:'edit',engine:'flux',folderId:'f',imageAId:'s',imageStyles:styles,promptSections:{action:'Restaurant',cameraBeats:[{moves:['hold']}],locks:'VIDEO ONLY'}})
assert.match(q.compiledPrompt,/Photorealistic/);assert.doesNotMatch(q.compiledPrompt,/Comic|pores|tripod|hold|VIDEO ONLY/i);assert.deepEqual(q.imageStyles,styles)
assert.doesNotMatch(compilePrompt(q.promptSections,false,q),/Picture 1/)
assert.deepEqual(parseBatch('A wrapped\nsentence\n\nSecond shot\r\n\r\nThird'),['A wrapped\nsentence','Second shot','Third'])
})
test('qwen21 is generate-only T2I with 25/1 defaults and no locks',()=>{
const q=validateRequest({mode:'generate',engine:'qwen21',folderId:'f',promptSections:{action:'a red bicycle'},settings:{aspect:'1:1'}})
assert.equal(q.engine,'qwen21');assert.equal(q.settings.steps,25);assert.equal(q.settings.cfg,1)
assert.equal(q.settings.width,1024);assert.equal(q.settings.height,1024)
assert.equal(q.lockFace,false);assert.equal(q.lockOutfit,false);assert.deepEqual(q.settings.loraStack,[])
assert.throws(()=>validateRequest({mode:'edit',engine:'qwen21',folderId:'f',imageAId:'s',promptSections:{action:'x'}}),/T2I in this build/)
})
+34
View File
@@ -0,0 +1,34 @@
import test from 'node:test'
import assert from 'node:assert/strict'
import {folderAssets,newestOutput,selectResult} from '../shared/studio2/viewer.mjs'
import {restoreAsset,resolveSize} from '../shared/studio2/input.mjs'
import {validateRequest} from '../shared/studio2/contracts.mjs'
import {applyResolvedImageSize} from '../shared/studio2/graphs.mjs'
const base={mode:'generate',engine:'flux',folderId:'f',prompt:'Walk'}
test('completed outputs join all folder assets and select only the output canvas',()=>{
const source={id:'source',folderId:'f'},output={id:'result',kind:'still',studio2:{savedAt:2}}
const jobs=[{request:{folderId:'f'},outputs:[output]}],library={stills:[source,{id:'other',folderId:'f'},{id:'result',folderId:'f'}],clips:[{id:'video',folderId:'f'}]}
const strip=folderAssets(library,jobs,'f');assert.equal(strip.length,4)
const state=selectResult({imageAId:'source',startClipId:'existing'},newestOutput(jobs,new Set(),'f'))
assert.equal(state.selected.id,'result');assert.equal(state.imageAId,'source');assert.equal(state.startClipId,'existing')
assert.equal(newestOutput(jobs,new Set(['result']),'f'),null)
})
test('Use as input binds the selected still, not its original parent input',()=>{
const next=restoreAsset({settings:{}},{id:'output',kind:'still',studio2:{imageAId:'old-parent',settings:{aspect:'1:1',seed:1}}})
assert.equal(next.imageAId,'output');assert.equal(next.startClipId,'');assert.equal(next.settings.aspect,'1:1');assert.equal(next.settings.seedMode,'random')
})
test('resolved aspect survives image graph default and megapixel scaling',()=>{
for(const aspect of ['auto','1:1','16:9','9:16']) {
const size=resolveSize(aspect,{width:1024,height:1024})
const request=validateRequest({...base,settings:{aspect,...size}})
const graph=applyResolvedImageSize({latent:{inputs:{width:1344,height:768}},scale:{class_type:'ImageScaleToTotalPixels',inputs:{image:['load',0],megapixels:1}}},request.settings)
assert.equal(graph.latent.inputs.width,size.width);assert.equal(graph.latent.inputs.height,size.height)
assert.equal(graph.scale.inputs.crop,'disabled');assert.equal(graph.scale.inputs.width,size.width)
if(['auto','1:1'].includes(aspect))assert.equal(size.width,size.height)
}
})
test('random Queues and shots differ; typed seed including 1 remains locked',()=>{
const a=validateRequest({...base,batch:['Turn','Sit']}),b=validateRequest(base)
assert.notEqual(a.settings.seed,b.settings.seed);assert.equal(new Set(a.shotSeeds).size,3);assert.ok(a.shotSeeds.every(s=>s!==1))
for(const seed of [1,987]){const raw={...base,batch:['Turn'],settings:{seedMode:'locked',seed}};assert.deepEqual(validateRequest(raw).shotSeeds,[seed,seed]);assert.equal(validateRequest(raw).settings.seed,seed)}
})
+15
View File
@@ -0,0 +1,15 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import {readFileSync} from 'node:fs';
import ts from 'typescript';
import {scopedFile} from '../shared/studio2/contracts.mjs';
const source=readFileSync(new URL('../server/utils/studio2/cleanup.ts',import.meta.url),'utf8').replace(/^import .*;\r?\n/gm,'').replace(/export /g,'');
function load(scope){const js=ts.transpileModule(source,{compilerOptions:{target:ts.ScriptTarget.ES2022}}).outputText;return new Function(...Object.keys(scope),js+';return {purge,retryCleanup}')( ...Object.values(scope));}
test('cleanup persists failures and retries only saved manifests after restart',async()=>{
const saved=[];let online=false,calls=[];
const file={filename:'hero.png',subfolder:'video/dev/studio2/job/0',type:'input'};
const r={id:'job',state:'complete',files:[file]};
const scope={scopedFile,useRuntimeConfig:()=>({comfyFilenamePrefix:'video/dev',comfyControlUrl:'http://local'}),sharedGpuHeaders:()=>({}),saveRecord:r=>saved.push(structuredClone(r)),records:()=>[r],withSharedGpuStart:async fn=>fn(),console:{warn(){}},fetch:async(url,opts)=>{calls.push(JSON.parse(opts.body));return {ok:online,status:online?200:404,json:async()=>({ok:online,cleared:1})}}};
await load(scope).purge(r);assert.equal(r.cleanupPending.length,1);assert.match(r.cleanupError,/404/);assert.equal(saved[0].cleanupPending.length,1);
r.files=[{...file,filename:'unsaved.png'}];online=true;await load(scope).retryCleanup();assert.deepEqual(calls[1].files,[file]);assert.deepEqual(r.cleanupPending,[]);assert.equal(r.files[0].filename,'unsaved.png');
});
+14 -3
View File
@@ -21,7 +21,7 @@ test('Use as input restores camera, sound, dialogue, music, LoRAs, aspect and lo
}) })
test('extend binds completed clip, inherits settings and leaves delta prompt alone',()=>{ test('extend binds completed clip, inherits settings and leaves delta prompt alone',()=>{
const form={...base,lockFace:true,lockOutfit:false,settings:{duration:5,width:1344,height:768},promptSections:{action:'Turn',cameraBeats:beats}} const form={...base,lockFace:true,lockOutfit:false,settings:{duration:5,width:1344,height:768},promptSections:{action:'Turn',cameraBeats:beats}}
const clip={id:'done',kind:'clip',studio2:{...base,lockFace:true,lockOutfit:false,settings:{duration:8,width:768,height:1344,aspect:'9:16',fps:24,steps:8,cfg:1.5,turbo:false,loraStack:[{name:'a',strengthModel:1,strengthClip:.5}]}}} const clip={id:'done',kind:'clip',studio2:{...base,lockFace:true,lockOutfit:false,settings:{duration:8,width:768,height:1344,aspect:'9:16',fps:24,steps:8,cfg:1.5,turbo:false,loraStack:[{name:'example.safetensors',strengthModel:1,strengthClip:.5}]}}}
const bound=bindClip(form,clip);assert.equal(bound.startClipId,'done');assert.deepEqual(bound.settings,clip.studio2.settings);assert.equal(bound.promptSections.action,'Turn');assert.equal(bound.identityStillId,'hero');assert.equal(bound.lockOutfit,false) const bound=bindClip(form,clip);assert.equal(bound.startClipId,'done');assert.deepEqual(bound.settings,clip.studio2.settings);assert.equal(bound.promptSections.action,'Turn');assert.equal(bound.identityStillId,'hero');assert.equal(bound.lockOutfit,false)
assert.equal(restoreAsset(form,clip).startClipId,'done');assert.equal(bindClip(form,{id:'legacy'}).lockOutfit,false) assert.equal(restoreAsset(form,clip).startClipId,'done');assert.equal(bindClip(form,{id:'legacy'}).lockOutfit,false)
const overridden={...bound,lockOutfit:false,settings:{...bound.settings,cfg:2.5}} const overridden={...bound,lockOutfit:false,settings:{...bound.settings,cfg:2.5}}
@@ -36,9 +36,20 @@ test('appearance roles distinguish wardrobe lock and both-off graph excludes her
const graph=nativeVideoGraph(template,r,{hero:'ignored.png',start:'start.png'},'preview') const graph=nativeVideoGraph(template,r,{hero:'ignored.png',start:'start.png'},'preview')
assert.equal(graph.hero,undefined);assert.equal(graph['155'].inputs['ref_images.ref_image_0'],undefined);assert.equal(graph.load_start.inputs.image,'start.png');assert.equal(r.identityStillId,'hero') assert.equal(graph.hero,undefined);assert.equal(graph['155'].inputs['ref_images.ref_image_0'],undefined);assert.equal(graph.load_start.inputs.image,'start.png');assert.equal(r.identityStillId,'hero')
}) })
test('video with locks off and no assets is MiniMax T2V empty first frame',()=>{
const r=validateRequest({mode:'video',engine:'minimax',folderId:'f',lockFace:false,lockOutfit:false,identityStillId:'',imageAId:'',promptSections:{action:'Wide empty street at dawn'}})
assert.equal(r.lockFace,false);assert.equal(r.lockOutfit,false);assert.equal(r.imageAId,'');assert.equal(r.startClipId,'')
const template=JSON.parse(fs.readFileSync(new URL('../server/assets/studio2_minimax_native.json',import.meta.url)))
const graph=nativeVideoGraph(template,r,{hero:'',start:''},'preview/t2v')
assert.equal(graph['155'].class_type,'MiniMaxH3ImageToVideo')
assert.equal(graph['155'].inputs.first_frame,undefined)
assert.equal(graph.load_start,undefined)
assert.equal(graph.hero,undefined)
assert.throws(()=>validateRequest({mode:'extend',engine:'minimax',folderId:'f',lockFace:false,lockOutfit:false,promptSections:{action:'Continue'}}),e=>/Previous clip|family strip/i.test(e.message))
})
test('sound/dialogue/music compile without empty invented audio',()=>{ test('sound/dialogue/music compile without empty invented audio',()=>{
const empty=compilePrompt({});assert.doesNotMatch(empty,/Soundscape:|<d>/);assert.match(empty,/non_diegetic_music: N\/A/) const empty=compilePrompt({},true);assert.doesNotMatch(empty,/Soundscape:|<d>/);assert.match(empty,/non_diegetic_music: N\/A/)
const text=compilePrompt({soundscapeChips:['footsteps'],dialogue:{mode:'spoken off-screen',line:'Hello'}});assert.match(text,/lips remain closed/);assert.match(text,/<Subject 1> \(S1\) says <d>\[English\]Hello<\/d>/) const text=compilePrompt({soundscapeChips:['footsteps'],dialogue:{mode:'spoken off-screen',line:'Hello'}},true);assert.match(text,/lips remain closed/);assert.match(text,/<Subject 1> \(S1\) says <d>\[English\]Hello<\/d>/)
}) })
test('aspect sizing follows source and named presets',()=>{ test('aspect sizing follows source and named presets',()=>{
assert.deepEqual(resolveSize('9:16',{width:1344,height:768}),{width:768,height:1344});const size=resolveSize('auto',{width:768,height:1024});assert.equal(size.width/size.height,.75) assert.deepEqual(resolveSize('9:16',{width:1344,height:768}),{width:768,height:1344});const size=resolveSize('auto',{width:768,height:1024});assert.equal(size.width/size.height,.75)
+143
View File
@@ -0,0 +1,143 @@
import test from 'node:test'
import assert from 'node:assert/strict'
import { createRequire } from 'node:module'
import { readFileSync, mkdtempSync, writeFileSync, existsSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import ts from 'typescript'
const require = createRequire(import.meta.url)
function load(path, resolve = require) {
const source = readFileSync(new URL(path, import.meta.url), 'utf8')
const code = ts.transpileModule(source, { compilerOptions: { target: ts.ScriptTarget.ES2022, module: ts.ModuleKind.CommonJS } }).outputText
const exports = {}
new Function('require', 'exports', code)(resolve, exports)
return exports
}
const { runFfmpeg } = load('../server/utils/ffmpeg.ts', id => id.startsWith('node:') ? require(id) : {})
const { imageDimensions } = load('../server/utils/resolution.ts')
const { fitStill, needsFit } = load('../server/utils/studio2/media.ts', id =>
id.startsWith('node:') ? require(id) : id === '../ffmpeg' ? { runFfmpeg } : id === '../resolution' ? { imageDimensions } : {}
)
const { isValidAnchorFile } = load('../server/utils/extensionFrame.ts', id =>
id.endsWith('.json') ? { default: {} } : id.startsWith('node:') ? require(id) : {}
)
// A fake PNG carrying a real signature + IHDR width/height for imageDimensions() to read.
// This is not a decodable image; tests never run ffmpeg against it (fit is either skipped
// or mocked), they only need imageDimensions() to report the right size.
function fakePng(width, height) {
const buf = Buffer.alloc(64) // isValidAnchorFile requires >= 64 bytes
buf.set([0x89, 0x50, 0x4e, 0x47], 0) // PNG signature start
buf.writeUInt32BE(width, 16)
buf.writeUInt32BE(height, 20)
return buf
}
const PNG_32x32 = fakePng(32, 32)
// A 16:9 phone photo -- same aspect as the 1344x768 job canvas but a much larger pixel size.
// This is the exact shape of the regression: same-aspect-different-size must still be fit.
const PNG_PHONE_16x9 = fakePng(4032, 2268)
const CANVAS_16x9 = { width: 1344, height: 768 }
const text = readFileSync(new URL('../server/utils/studio2/runner.ts', import.meta.url), 'utf8')
const tree = ts.createSourceFile('runner.ts', text, ts.ScriptTarget.Latest, true)
function extractUpload(scope) {
const nodes = tree.statements.filter(n =>
(ts.isFunctionDeclaration(n) && n.name?.text === 'upload') ||
(ts.isVariableStatement(n) && n.getText(tree).startsWith('const NEVER_FIT'))
)
const js = nodes.map(n => ts.transpileModule(n.getText(tree), { compilerOptions: { target: ts.ScriptTarget.ES2022 } }).outputText).join('\n')
return new Function(...Object.keys(scope), `${js};return upload`)(...Object.values(scope))
}
function fixture() {
const uploaded = []
const record = { id: 'r', index: 0, files: [], request: { settings: { width: 32, height: 32 } } }
const scope = {
fitStill: async (data) => { throw new Error('fitStill must not be called for an already-canvas-sized handoff frame') },
needsFit,
saveRecord() {},
useRuntimeConfig: () => ({ comfyFilenamePrefix: 'preview/MiniMax_H3' }),
comfyFetch: async (_url, opts) => {
uploaded.push(opts.body)
return { ok: true, json: async () => ({ name: 'start.png', subfolder: 'preview/MiniMax_H3/studio2/r/0' }) }
},
scopedFile: () => true
}
return { scope, record, uploaded }
}
test('upload() never fits the start/hero labels when the still already matches the job canvas', async () => {
const f = fixture()
const upload = extractUpload(f.scope)
await upload(f.record, 'start', PNG_32x32)
assert.equal(f.record.handoffFitted, false)
assert.equal(f.record.files.length, 1)
})
test('upload() never fits the hero label even when it mismatches the canvas aspect', async () => {
const f = fixture()
f.record.request.settings = { width: 1344, height: 768 }
const upload = extractUpload(f.scope)
// PNG_32x32 is square, canvas is 16:9 -- would trigger a fit for any other label.
await upload(f.record, 'hero', PNG_32x32)
assert.equal(f.record.handoffFitted, undefined) // only tracked for 'start'
})
test('upload() still fits a source still whose aspect truly differs from the job canvas', async () => {
const f = fixture()
f.record.request.settings = { width: 1344, height: 768 }
let fitCalled = false
f.scope.fitStill = async (data) => { fitCalled = true; return data }
const upload = extractUpload(f.scope)
await upload(f.record, 'compose', PNG_32x32)
assert.equal(fitCalled, true)
})
test('needsFit is false only for an exact width+height match; same-aspect-different-size still needs fitting', () => {
assert.equal(needsFit(PNG_32x32, 32, 32), false)
assert.equal(needsFit(PNG_32x32, 1344, 768), true)
// Same 16:9 aspect as the canvas, but 3x the pixel size -- must still fit.
assert.equal(needsFit(PNG_PHONE_16x9, CANVAS_16x9.width, CANVAS_16x9.height), true)
assert.equal(needsFit(PNG_PHONE_16x9, 4032, 2268), false)
assert.equal(needsFit(Buffer.alloc(64), CANVAS_16x9.width, CANVAS_16x9.height), true) // unreadable dims always fit
})
test('New Video (mode video, no startClipId): a same-aspect phone still uploaded as "source" is fitted to the job canvas', async () => {
const f = fixture()
f.record.request.settings = { ...CANVAS_16x9 }
let fitCalled = false
f.scope.fitStill = async (data, w, h) => { fitCalled = true; assert.equal(w, CANVAS_16x9.width); assert.equal(h, CANVAS_16x9.height); return Buffer.alloc(64) }
const upload = extractUpload(f.scope)
await upload(f.record, 'source', PNG_PHONE_16x9)
assert.equal(fitCalled, true, 'a 4032x2268 still against a 1344x768 canvas must still be fitted despite matching aspect')
})
test('Extend (startClipId set): an anchor already sized to the job canvas uploaded as "start" is not fitted', async () => {
const f = fixture()
f.record.request.settings = { ...CANVAS_16x9 }
const anchorAtCanvasSize = fakePng(CANVAS_16x9.width, CANVAS_16x9.height)
const upload = extractUpload(f.scope) // f.scope.fitStill throws if called -- see fixture()
await upload(f.record, 'start', anchorAtCanvasSize)
assert.equal(f.record.handoffFitted, false)
})
test('two extends at identical width/height: the anchor PNG reaches Comfy byte-identical, no ffmpeg fit', async () => {
const dir = mkdtempSync(join(tmpdir(), 'studio2-anchor-'))
const anchorPath = join(dir, 'anchor-last.png')
writeFileSync(anchorPath, PNG_32x32)
assert.ok(isValidAnchorFile(anchorPath))
for (const hop of [0, 1]) {
const f = fixture()
const upload = extractUpload(f.scope)
const anchorBytes = readFileSync(anchorPath)
const key = await upload(f.record, 'start', anchorBytes)
assert.equal(f.record.handoffFitted, false, `hop ${hop} must not resample the anchor frame`)
const sentBody = f.uploaded[0]
const sentBlob = sentBody.get('image')
const sentBytes = Buffer.from(await sentBlob.arrayBuffer())
assert.deepEqual(sentBytes, anchorBytes, `hop ${hop} must upload the anchor PNG byte-identical`)
assert.ok(key.length > 0)
}
})
+76
View File
@@ -0,0 +1,76 @@
import {kleinIdentityPlan,applyKleinIdentity,assertKleinSample} from '../shared/studio2/klein-identity.mjs'
import {stylePrompt} from '../shared/studio2/styles.mjs'
import test from 'node:test'
import assert from 'node:assert/strict'
import fs from 'node:fs'
import ts from 'typescript'
import {validateRequest,compilePrompt} from '../shared/studio2/contracts.mjs'
import {attachHeroReference,applyResolvedImageSize} from '../shared/studio2/graphs.mjs'
import {extendSelected,changeSource,restoreAsset} from '../shared/studio2/input.mjs'
const template=JSON.parse(fs.readFileSync(new URL('../server/assets/klein_v2_edit.json',import.meta.url)))
const source=fs.readFileSync(new URL('../server/utils/studio2/runner.ts',import.meta.url),'utf8')
const tree=ts.createSourceFile('runner.ts',source,ts.ScriptTarget.Latest,true)
const fn=tree.statements.find(n=>ts.isFunctionDeclaration(n)&&n.name.text==='prepareGraph')
const js=ts.transpileModule(fn.getText(tree),{compilerOptions:{target:ts.ScriptTarget.ES2022}}).outputText
async function prepare(request){
const uploads=[]
const scope={stylePrompt,useRuntimeConfig:()=>({comfyFilenamePrefix:'preview'}),upload:async(r,label)=>{uploads.push(label);return label+'.png'},readFileSync:()=>Buffer.from('fixture'),stillPath:()=>'',getStill:(_,id)=>({id}),resolveRequestSize:async()=>{},saveRecord(){},
buildImageV2Workflow:params=>{const graph=JSON.parse(fs.readFileSync(new URL('../server/assets/klein_v2_'+params.mode+'.json',import.meta.url)));if(graph['1']) graph['1'].inputs.image=params.imageAName;return {graph}},kleinIdentityPlan,applyKleinIdentity,attachHeroReference,applyResolvedImageSize,ensureComfyLoraNames:async()=>{},resolveGraphLoraNames(){}}
const run=new Function(...Object.keys(scope),`${js};return prepareGraph`)(...Object.values(scope))
const record={request,owner:'o',id:'job',index:0};return {graph:await run(record),record,uploads}
}
for(const sourceId of ['hero','source','']) test(`Klein edit samples source with hero ${sourceId || 'as sole source'}`,async()=>{
const q=validateRequest({folderId:'f',mode:'edit',engine:'flux',imageAId:sourceId,identityStillId:'hero',lockFace:true,lockOutfit:false,prompt:'she at a pool'})
const {graph,record,uploads}=await prepare(q)
assert.deepEqual(graph['19'].inputs.latent_image,['14',0]);assert.equal(graph['14'].class_type,'EmptyFlux2LatentImage')
assert.deepEqual(graph['19'].inputs.sigmas,['17',0]);assert.equal(graph.source_sigmas,undefined)
assert.equal(record.sourceStillId,sourceId||'hero');assert.equal(record.sampleLatent,'empty+reference')
assert.equal(uploads.includes('hero'),sourceId==='source');assert.equal(!!graph.hero_load,sourceId==='source')
assert.match(q.compiledPrompt,/face, glasses, hair/);assert.match(q.compiledPrompt,/do not restore the hero outfit/)
})
test('image requests omit video sections while form keeps them',()=>{
const sections={action:'sitting in a restaurant',cameraBeats:[{moves:['hold']}],soundscapeChips:['footsteps'],dialogue:{mode:'whisper',line:'Hello'},musicChip:'low underscore'}
const q=validateRequest({folderId:'f',mode:'edit',engine:'flux',identityStillId:'hero',promptSections:sections})
assert.doesNotMatch(q.compiledPrompt,/tripod|hold|soundscape|music|Hello|Picture 1/i)
assert.deepEqual(q.promptSections,{action:sections.action});assert.equal(sections.cameraBeats.length,1)
assert.equal(compilePrompt(sections,false,{engine:'krea',identityStillId:'hero'}),sections.action)
})
test('Extend normalizes engine and inherits clip without replacing action or camera',()=>{
const form={mode:'edit',engine:'flux',imageAId:'still',promptSections:{action:'New action',cameraBeats:[{moves:['hold']}]},settings:{loraStack:[]}}
const clip={id:'clip',kind:'clip',studio2:{engine:'minimax',identityStillId:'hero',lockOutfit:false,promptSections:{action:'Old action'},settings:{duration:8,aspect:'1:1',width:960,height:960,fps:24,turbo:false,steps:8,cfg:1.5,loraStack:[{name:'example.safetensors',strengthModel:.6,strengthClip:.3}]}}}
const next=extendSelected(form,clip);assert.equal(next.engine,'minimax');assert.equal(next.startClipId,'clip');assert.deepEqual(next.settings,clip.studio2.settings)
assert.deepEqual(next.promptSections,form.promptSections);assert.equal(next.lockOutfit,false)
assert.equal(changeSource({...next,imageAId:'new-still'}).startClipId,'clip');assert.equal(changeSource(next).mode,'extend')
assert.deepEqual(restoreAsset(form,clip).promptSections,form.promptSections)
assert.equal(extendSelected({...form,startClipId:'clip'},null).startClipId,'clip')
assert.throws(()=>validateRequest({folderId:'f',mode:'extend',engine:'minimax',identityStillId:'hero',prompt:'Go'}),/Select the clip in the family strip or Previous clip/)
})
for (const imageAId of ['', 'hero', 'other']) test('Generate samples hero exclusively: '+imageAId,async()=>{
const q=validateRequest({folderId:'f',mode:'generate',engine:'flux',imageAId,identityStillId:'hero',lockFace:true,lockOutfit:false,prompt:'she at a pool',settings:{loraStack:[]}})
const {graph,record,uploads}=await prepare(q)
assert.equal(record.graphId,'klein_v2_edit.json')
assert.equal(record.sourceStillId,'hero')
assert.equal(record.sampleLatent,'empty+reference')
assert.equal(record.sampleDenoise,null)
assert.deepEqual(graph['19'].inputs.latent_image,['14',0])
assert.deepEqual(uploads,['source'])
assert.equal(graph.hero_load,undefined)
assert.deepEqual(q.settings.loraStack,[])
assert.doesNotMatch(q.compiledPrompt,/Picture 1/)
})
test('reference sampling ignores face-lock denoise; text-only uses empty latent',async()=>{
const q=validateRequest({folderId:'f',mode:'edit',engine:'flux',imageAId:'source',lockFace:false,prompt:'pool'})
assert.equal((await prepare(q)).record.sampleDenoise,null)
const empty=validateRequest({folderId:'f',mode:'generate',engine:'flux',prompt:'pool'})
assert.equal((await prepare(empty)).record.sampleLatent,'empty')
})
test('validate live references and reject direct source sampling',()=>{
assert.equal(assertKleinSample(template,'hero'),'empty+reference')
const broken=structuredClone(template);broken['18'].inputs.positive=['9',0]
assert.throws(()=>assertKleinSample(broken,'hero'),/reference conditioning is missing/)
const copy=structuredClone(template);copy['19'].inputs.latent_image=['11',0]
assert.throws(()=>assertKleinSample(copy,'hero'),/not the source image latent/)
assert.equal(kleinIdentityPlan({engine:'krea',mode:'generate',identityStillId:'hero'}),null)
assert.equal(kleinIdentityPlan({engine:'minimax',mode:'extend'}),null)
})
+19
View File
@@ -0,0 +1,19 @@
import test from 'node:test'
import assert from 'node:assert/strict'
import {readFileSync} from 'node:fs'
import ts from 'typescript'
import {loraIdentityKey,loraFriendlyName,loraTriggerFor} from '../utils/loras.ts'
test('Studio 2 uses production labels/triggers and sees later changes without a restart',()=>{
const source=readFileSync(new URL('../server/utils/studio2/lora-catalog.ts',import.meta.url),'utf8').replace(/^import .*;\r?\n/gm,'').replace('export function','function')
const js=ts.transpileModule(source,{compilerOptions:{target:ts.ScriptTarget.ES2022}}).outputText
let exported={loraAliases:{'XAIgen-klein_portrait.safetensors':'My portrait'},loraTriggers:{'xaigen-klein_portrait.safetensors':'portrait token'}}
const read=new Function('readFileSync','getLoraAliases','getLoraTriggers','loraIdentityKey','join','studio2Root',js+';return studio2LoraCatalog')(()=>{if(!exported)throw new Error('missing');return JSON.stringify(exported)},()=>({'klein_portrait.safetensors':'Old local name'}),()=>({}),loraIdentityKey,(...parts)=>parts.join('/'),()=>'/dev/studio2')
let catalog=read('owner')
assert.equal(loraFriendlyName('folder/xaigen-klein_portrait.safetensors',catalog.aliases),'My portrait')
assert.equal(loraTriggerFor('klein_portrait.safetensors',catalog.triggers),'portrait token')
exported.loraAliases['XAIgen-klein_portrait.safetensors']='Updated portrait'
assert.equal(loraFriendlyName('klein_portrait.safetensors',read('owner').aliases),'Updated portrait')
exported=null
assert.equal(loraFriendlyName('klein_portrait.safetensors',read('owner').aliases),'Old local name')
})
+19
View File
@@ -0,0 +1,19 @@
import test from 'node:test'
import assert from 'node:assert/strict'
import {readFileSync} from 'node:fs'
import * as Vue from 'vue'
import {compile} from '@vue/compiler-dom'
import {renderToString} from '@vue/server-renderer'
import {loraFriendlyName,loraIdentityKey} from '../utils/loras.ts'
test('LoRA dropdown renders one clean label or alias, with filename only in tooltip',async()=>{
const source=readFileSync(new URL('../components/LoraSelect.vue',import.meta.url),'utf8')
const option=source.match(/<option\s+v-for="name in availableNames"[\s\S]*?<\/option>/)[0]
const render=new Function('Vue',compile('<select>'+option+'</select>',{mode:'function',prefixIdentifiers:true}).code)(Vue)
const names=['xaigen-minimax csot.safetensors','folder/XAIgen-klein-portrait.safetensors']
const aliases={[loraIdentityKey(names[1])]:'Portrait style'}
const html=await renderToString(Vue.createSSRApp({render,setup:()=>({availableNames:names,filenameSubtitle:true,labelFor:name=>loraFriendlyName(name,aliases)})}))
const options=[...html.matchAll(/<option[^>]*title="([^"]*)"[^>]*>([^<]*)<\/option>/g)]
assert.deepEqual(options.map(match=>match[1]),names)
assert.deepEqual(options.map(match=>match[2]),['xaigen-minimax csot','Portrait style'])
})
+69
View File
@@ -0,0 +1,69 @@
import test from 'node:test'
import assert from 'node:assert/strict'
import {reactive,watch} from 'vue'
import {inferLoraUse,filterLorasForUse,filterLorasForStudio,filterLoraStackForUse} from '../utils/loras.ts'
import {createLoraMemory,loraUse} from '../shared/studio2/lora-memory.mjs'
import {restoreAll,restoreAsset,bindClip,extendSelected,useAsHero,useAsSource} from '../shared/studio2/input.mjs'
const stack=name=>[{name:name+'.safetensors',strengthModel:.6,strengthClip:.35}]
test('engine token matrix uses paths and names, omits unknown and system picker entries',()=>{
const cases={'xaigen-klein_car':'flux','klein/style':'flux','XAIgen-flux_test':'flux','Flux2-Klein/test':'flux','MiniMax/car':'minimax','folder/LTX_style':'ltx','xiagen-Krea_look':'krea','xaigen-krea/style':'krea','unlabelled':null}
for(const [name,use] of Object.entries(cases)) {
assert.deepEqual(inferLoraUse(name+'.safetensors'),use?[use]:null)
for(const current of ['flux','krea','minimax','ltx']) assert.equal(filterLorasForUse([name+'.safetensors'],current).length,use===current?1:0)
}
const restricted=['xaigen-klein.safetensors','XAIgen-MiniMax.safetensors','xiagen-Krea.safetensors']
assert.deepEqual(filterLorasForStudio(restricted,false),[])
assert.deepEqual(filterLoraStackForUse(stack('xaigen-klein'),'flux',false),[])
assert.deepEqual(filterLoraStackForUse(stack('LTX'),'ltx',false),[])
for(const [name,use] of [['minimax_h3_fl2v_turbo_8step_v1.0_comfyui_bf16','minimax'],['ltx-2.3-22b-distilled','ltx'],['Flux2-Klein-9B-consistency-V2','flux']]) assert.deepEqual(filterLorasForUse([name+'.safetensors'],use),[])
assert.deepEqual(filterLoraStackForUse(stack('unlabelled'),'flux',true),stack('unlabelled'))
})
function bench() {
const data=new Map(),storage={getItem:k=>data.get(k),setItem:(k,v)=>data.set(k,v)}
const memory=createLoraMemory(true,storage)
const form=reactive({mode:'generate',engine:'flux',settings:{loraStack:[]}})
const stop=watch(()=>[loraUse(form),form.settings.loraStack],()=>memory.sync(form),{deep:true,flush:'sync'})
memory.sync(form)
return {form,memory,storage,stop}
}
test('reactive task switches remember four independent stacks and both strengths',()=>{
const {form,memory,storage,stop}=bench()
form.settings.loraStack=stack('klein_car_bee')
form.settings.loraStack[0].strengthClip=.8
form.mode='video';form.engine='minimax'
assert.deepEqual(form.settings.loraStack,[])
form.settings.loraStack=stack('MiniMax_style')
form.engine='ltx';assert.deepEqual(form.settings.loraStack,[])
form.settings.loraStack=stack('LTX_style')
form.mode='edit';form.engine='flux'
assert.equal(form.settings.loraStack[0].name,'klein_car_bee.safetensors')
assert.equal(form.settings.loraStack[0].strengthClip,.8)
form.engine='krea';assert.deepEqual(form.settings.loraStack,[])
form.settings.loraStack=stack('Krea_style')
form.mode='video';form.engine='minimax'
assert.deepEqual(form.settings.loraStack,stack('MiniMax_style'))
Object.assign(form,useAsHero(form,{id:'image'}),useAsSource(form,{id:'source'}))
assert.deepEqual(form.settings.loraStack,stack('MiniMax_style'))
const fresh=createLoraMemory(true,storage),other={mode:'generate',engine:'krea',settings:{loraStack:[]}}
fresh.sync(other);assert.deepEqual(other.settings.loraStack,stack('Krea_style'))
stop()
})
test('Use as input and Extend copy saved stacks, filter wrong engines, and fall back only when saved empty',()=>{
const {form,memory,stop}=bench()
form.settings.loraStack=stack('klein_remembered')
const still={id:'still',kind:'still',studio2:{engine:'flux',mode:'edit',settings:{loraStack:[...stack('klein_saved'),...stack('MiniMax_wrong')]}}}
Object.assign(form,restoreAll(form,still,true,memory))
assert.deepEqual(form.settings.loraStack,stack('klein_saved'))
const clip={id:'clip',kind:'clip',studio2:{engine:'minimax',settings:{loraStack:[...stack('MiniMax_saved'),...stack('klein_wrong')]}}}
Object.assign(form,extendSelected(form,clip,true,memory))
assert.deepEqual(form.settings.loraStack,stack('MiniMax_saved'))
Object.assign(form,restoreAll(form,{...still,studio2:{...still.studio2,settings:{loraStack:[]}}},true,memory))
assert.deepEqual(form.settings.loraStack,stack('klein_saved'))
Object.assign(form,bindClip(form,{id:'empty',settings:{loraStack:[]}},true,memory))
assert.deepEqual(form.settings.loraStack,stack('MiniMax_saved'))
const legacy=restoreAsset(form,{id:'old-still',kind:'still',engine:'krea',settings:{loraStack:stack('Krea_saved')}},true,memory)
assert.equal(legacy.engine,'krea');assert.deepEqual(legacy.settings.loraStack,stack('Krea_saved'))
stop()
})
+17
View File
@@ -0,0 +1,17 @@
import test from 'node:test'
import assert from 'node:assert/strict'
import {queueJobs,jobView,elapsedSeconds} from '../shared/studio2/queue-view.mjs'
test('queue shows every waiting job in order alongside the running job',()=>{
const jobs=[{id:'late',state:'waiting',queuedAt:30},{id:'done',state:'complete',queuedAt:1},{id:'early',state:'waiting',queuedAt:10},{id:'active',state:'rendering',queuedAt:20},{id:'held',state:'held',queuedAt:40}]
assert.deepEqual(queueJobs(jobs).map(j=>j.id),['active','early','late','held'])
assert.equal(queueJobs(Array.from({length:15},(_,i)=>({id:i,state:'waiting',queuedAt:i}))).length,15)
})
test('authoritative completion ends a stale rendering row and freezes its timer',()=>{
const record={state:'rendering',queuedAt:1000,startedAt:2000,updatedAt:5000}
for(const status of ['complete','error','cancelled']){
const result=jobView(record,{status,updatedAt:12000});assert.equal(elapsedSeconds(result,20000),10);assert.equal(elapsedSeconds(result,2000000),10);assert.equal(queueJobs([result]).length,0)
}
const live=jobView(record,null,{status:'complete'});assert.equal(live.state,'complete');assert.equal(elapsedSeconds(live,999999),3)
assert.equal(jobView(record,null,{status:'complete',saving:true}).state,'rendering')
assert.equal(elapsedSeconds({state:'complete',startedAt:2000,finishedAt:12000,updatedAt:200000},500000),10)
})
+14 -3
View File
@@ -1,3 +1,4 @@
import {queueSeeds} from '../shared/studio2/seed.mjs'
import test from 'node:test' import test from 'node:test'
import assert from 'node:assert/strict' import assert from 'node:assert/strict'
import {readFileSync,mkdtempSync,mkdirSync,writeFileSync,existsSync} from 'node:fs' import {readFileSync,mkdtempSync,mkdirSync,writeFileSync,existsSync} from 'node:fs'
@@ -16,12 +17,12 @@ function fixture(mode='video') {
const job={status:'running',clientId:'client',library:{}} const job={status:'running',clientId:'client',library:{}}
let id=0 let id=0
const scope={ const scope={
watchProgress:()=>()=>{},compilePrompt,saveRecord(){},update(r,state){r.state=state;phases.push(state)},acquireSharedGpu:async()=>true,ensureComfyReady:async()=>{},emitJob(){}, queueSeeds,watchProgress:()=>()=>{},compilePrompt,saveRecord(){},update(r,state){r.state=state;phases.push(state)},acquireSharedGpu:async()=>true,ensureComfyReady:async()=>{},emitJob(){},
prepareGraph:async r=>{queued.push({hero:r.request.identityStillId,start:r.request.startClipId,action:r.request.promptSections.action});return {}},queuePrompt:async()=>({prompt_id:`p${++id}`}), prepareGraph:async r=>{queued.push({hero:r.request.identityStillId,start:r.request.startClipId,action:r.request.promptSections.action});return {}},queuePrompt:async()=>({prompt_id:`p${++id}`}),
fetchHistory:async p=>({[p]:{status:{status_str:'success'},outputs:{}}}),comfyFetch:()=>{throw new Error('Unexpected network call')}, fetchHistory:async p=>({[p]:{status:{status_str:'success'},outputs:{}}}),comfyFetch:()=>{throw new Error('Unexpected network call')},
extractVideo:()=>({filename:'video.mp4',subfolder:'preview/studio2/r',type:'output'}),extractEditedImage:()=>({filename:'image.png',subfolder:'preview/studio2/r',type:'output'}), extractVideo:()=>({filename:'video.mp4',subfolder:'preview/studio2/r',type:'output'}),extractEditedImage:()=>({filename:'image.png',subfolder:'preview/studio2/r',type:'output'}),
downloadComfyVideo:async()=>Buffer.from('video'),downloadComfyImage:async()=>Buffer.from('image'),imageDimensions:()=>({width:960,height:960}), downloadComfyVideo:async()=>Buffer.alloc(128,'v'),downloadComfyImage:async()=>Buffer.from('image'),imageDimensions:()=>({width:960,height:960}),
videoSourcePaths:()=>['source'],clipVideoPath:()=>'/fake/source',studio2Root:()=>'/fake',join,mkdirSync(){},stitchExtension:async()=>Buffer.from('stitched'), videoSourcePaths:()=>['source'],clipVideoPath:()=>'/fake/source',studio2Root:()=>'/fake',join,mkdirSync(){},writeFileSync(){},unlinkSync(){},probeDuration:async()=>5,stitchExtension:async()=>Buffer.alloc(128,'s'),
saveClip:async p=>{const a={id:`asset${saved.length}`,...p};saved.push(a);return a},saveStill:async p=>{const a={id:`asset${saved.length}`,...p};saved.push(a);return a}, saveClip:async p=>{const a={id:`asset${saved.length}`,...p};saved.push(a);return a},saveStill:async p=>{const a={id:`asset${saved.length}`,...p};saved.push(a);return a},
attachStudio2Metadata:async(owner,id,data)=>{assert.ok(saved.some(a=>a.id===id));assert.equal(data.identityStillId,'hero')}, attachStudio2Metadata:async(owner,id,data)=>{assert.ok(saved.some(a=>a.id===id));assert.equal(data.identityStillId,'hero')},
purge:async r=>{assert.ok(saved.length>purged.length);purged.push(r.index);r.purgeResult='Cleared input + output'}, purge:async r=>{assert.ok(saved.length>purged.length);purged.push(r.index);r.purgeResult='Cleared input + output'},
@@ -39,6 +40,16 @@ for(const mode of ['video','iterate']) test(`${mode} runner saves each output be
test('failed library save never purges host media',async()=>{ test('failed library save never purges host media',async()=>{
const f=fixture();f.scope.saveClip=async()=>{throw new Error('Disk full')};await runner(f.scope)(f.record,f.job);assert.equal(f.record.state,'failed');assert.equal(f.purged.length,0) const f=fixture();f.scope.saveClip=async()=>{throw new Error('Disk full')};await runner(f.scope)(f.record,f.job);assert.equal(f.record.state,'failed');assert.equal(f.purged.length,0)
}) })
test('an empty Comfy video output fails the job with a real error and never saves or attaches a clip',async()=>{
const f=fixture();f.scope.downloadComfyVideo=async()=>Buffer.alloc(0)
await runner(f.scope)(f.record,f.job)
assert.equal(f.record.state,'failed');assert.match(f.record.error,/empty video file/);assert.equal(f.saved.length,0);assert.equal(f.record.outputs.length,0)
})
test('a stitched extension with no readable duration fails the job and never saves or attaches a clip',async()=>{
const f=fixture('video');f.scope.probeDuration=async()=>NaN
await runner(f.scope)(f.record,f.job)
assert.equal(f.record.state,'failed');assert.match(f.record.error,/no readable duration/);assert.equal(f.saved.length,0);assert.equal(f.record.outputs.length,0)
})
test('resume uses persisted Comfy prompt and never submits it again',async()=>{ test('resume uses persisted Comfy prompt and never submits it again',async()=>{
const f=fixture();f.record.request.batch=[];f.record.promptId='existing';f.record.files=[];f.record.index=0 const f=fixture();f.record.request.batch=[];f.record.promptId='existing';f.record.files=[];f.record.index=0
f.scope.queuePrompt=async()=>{throw new Error('Duplicate submit')};await runner(f.scope)(f.record,f.job) f.scope.queuePrompt=async()=>{throw new Error('Duplicate submit')};await runner(f.scope)(f.record,f.job)
+25
View File
@@ -0,0 +1,25 @@
import test from 'node:test'
import assert from 'node:assert/strict'
import {validateHeaderValue} from 'node:http'
import {gpuStatus} from '../shared/studio2/status.mjs'
import {inlineFilename} from '../shared/content-disposition.mjs'
import {readFileSync} from 'node:fs'
test('waiting records never replace the idle GPU status after completion',()=>{
const waiting={state:'waiting',request:{promptSections:{action:'Old waiting prompt'}}},running={state:'rendering',request:{promptSections:{action:'Current video'}}},health={comfy:{ok:true}};
assert.equal(gpuStatus([waiting,running],health).detail,'Current video');
assert.deepEqual(gpuStatus([waiting,{...running,state:'complete'}],health),{label:'Ready',tone:'ready',detail:'No job running'});
assert.equal(gpuStatus([waiting],health,[{status:'running',name:'Upscale current clip'}]).label,'Upscaling');
assert.equal(gpuStatus([waiting],{comfy:{asleep:true}}).label,'Sleeping');
assert.equal(gpuStatus([waiting],{comfy:{asleep:true},yue2:{busy:true}}).label,'Generating');
assert.equal(gpuStatus([waiting],{comfy:{asleep:true}},[],[{status:'running',name:'song',live:{message:'Planning'}}]).label,'Generating');
});
test('Unicode video titles produce valid response headers and retain UTF-8 filename',()=>{
for(const title of ['A scene… “hello” — 🎥.mp4','日本語.mp4','bad\r\nheader".mp4']){
const header=inlineFilename(title);assert.doesNotThrow(()=>validateHeaderValue('Content-Disposition',header));assert.match(header,/filename\*=UTF-8''/);assert.doesNotMatch(header,/[\r\n]/);
}
assert.match(inlineFilename('Scene….mp4'),/%E2%80%A6/);
});
test('Studio 2 upscale button delegates progress to the queue without stretching actions',()=>{
const bench=readFileSync(new URL('../components/studio2/Bench.vue',import.meta.url),'utf8'),control=readFileSync(new URL('../components/VideoUpscaleControl.vue',import.meta.url),'utf8');
assert.match(bench,/:clip-id="selected.id" queue-only/);assert.match(bench,/items-start gap-3/);assert.match(bench,/Job queue/);assert.match(control,/state\?\.job && !queueOnly/);
});
+14
View File
@@ -0,0 +1,14 @@
import test from 'node:test'
import assert from 'node:assert/strict'
import {applyUploadedStill} from '../shared/studio2/upload.mjs'
test('uploaded source is selected without changing hero or output',()=>{
const form={imageAId:'old',identityStillId:'hero',selected:'result',startClipId:'clip',mode:'extend',settings:{aspect:'16:9'}}
const next=applyUploadedStill(form,{still:{id:'uploaded'}},'source')
assert.equal(next.imageAId,'uploaded');assert.equal(next.identityStillId,'hero');assert.equal(next.selected,'result');assert.equal(next.startClipId,'clip');assert.equal(next.mode,'extend');assert.equal(next.settings.aspect,'16:9')
})
test('uploaded hero changes identity only and missing saves are reported',()=>{
const form={identityStillId:'old',imageAId:'source',settings:{aspect:'1:1'}}
const next=applyUploadedStill(form,{still:{id:'hero'}},'hero')
assert.equal(next.identityStillId,'hero');assert.equal(next.imageAId,'source');assert.equal(next.settings.aspect,'1:1')
assert.throws(()=>applyUploadedStill(form,{},'hero'),/did not return a saved still/)
})
+1 -1
View File
@@ -12,7 +12,7 @@ test('compose without B and refine without mask are 400',()=>{
for(const mode of ['compose','refine']) assert.throws(()=>validateRequest({...base,mode,engine:'flux',imageAId:'a'}),e=>e.statusCode===400) for(const mode of ['compose','refine']) assert.throws(()=>validateRequest({...base,mode,engine:'flux',imageAId:'a'}),e=>e.statusCode===400)
}) })
test('Use as input round-trips camera chip and never invents one for flat text',()=>{ test('Use as input round-trips camera chip and never invents one for flat text',()=>{
const r=validateRequest({...base,promptSections:{action:'walk',cameraChip:{family:'dolly in',amplitude:'subtle',speed:'slow'}}}); assert.deepEqual(restoreInput({studio2:r}).promptSections,r.promptSections);assert.equal(restoreInput({prompt:'old text'}).promptSections.cameraChip,undefined);assert.match(compilePrompt(r.promptSections),/dolly in/) const r=validateRequest({...base,promptSections:{action:'walk',cameraChip:{family:'dolly in',amplitude:'subtle',speed:'slow'}}}); assert.deepEqual(restoreInput({studio2:r}).promptSections,r.promptSections);assert.equal(restoreInput({prompt:'old text'}).promptSections.cameraChip,undefined);assert.match(compilePrompt(r.promptSections,true),/dolly in/)
}) })
test('purge accepts only exact preview path boundary',()=>{ test('purge accepts only exact preview path boundary',()=>{
assert.equal(scopedFile({subfolder:'video/MiniMax_H3_preview/job',filename:'a.png'},'video/MiniMax_H3_preview'),true) assert.equal(scopedFile({subfolder:'video/MiniMax_H3_preview/job',filename:'a.png'},'video/MiniMax_H3_preview'),true)
+47
View File
@@ -0,0 +1,47 @@
"""CPU-only YuE2 worker regression tests; never import torch or run inference."""
import importlib.util
import os
from pathlib import Path
import unittest
from types import SimpleNamespace
spec = importlib.util.spec_from_file_location('yue2_worker', Path(__file__).parents[1] / 'scripts/yue2-worker.py')
worker = importlib.util.module_from_spec(spec)
spec.loader.exec_module(worker)
class WorkerTests(unittest.TestCase):
def test_lyrics_allow_multiple_sections_and_wrap_plain_text(self):
multi = '[Verse 1]\nHello\n\n[Chorus]\nSing it'
self.assertEqual(worker.normalize_lyrics(multi), multi + '\n')
self.assertEqual(worker.normalize_lyrics('[Verse 1]\nHello'), '[Verse 1]\nHello\n')
self.assertEqual(worker.normalize_lyrics('No heading yet'), '[song]\nNo heading yet\n\n')
with self.assertRaises(ValueError):
worker.normalize_lyrics('')
with self.assertRaises(ValueError):
worker.normalize_lyrics(' ')
def test_attention_backend_never_selects_flash(self):
torch_mod = SimpleNamespace(
cuda=SimpleNamespace(is_available=lambda: True),
backends=SimpleNamespace(cudnn=SimpleNamespace(is_available=lambda: True)),
)
attention, pipeline = worker.resolve_attention_backend(torch_mod)
self.assertEqual(attention, 'sdpa')
self.assertEqual(pipeline, 'torch-eager')
self.assertNotEqual(attention, 'flash')
self.assertTrue(os.environ.get('PYTORCH_CUDA_ALLOC_CONF', '').startswith('expandable_segments'))
def test_target_length_maps_to_semantic_max_tokens(self):
# 48000 / 1920 = 25 latent frames per second.
self.assertEqual(max(200, min(9000, 30 * 25)), 750)
self.assertEqual(max(200, min(9000, 15 * 25)), 375)
self.assertEqual(max(200, min(9000, 150 * 25)), 3750)
def test_cuda_oom_detection(self):
self.assertTrue(worker.is_cuda_oom(RuntimeError('CUDA out of memory. Tried to allocate 2.49 GiB')))
self.assertFalse(worker.is_cuda_oom(RuntimeError('bad lyrics')))
if __name__ == '__main__':
unittest.main()
+101
View File
@@ -0,0 +1,101 @@
import test from 'node:test'
import assert from 'node:assert/strict'
import { EventEmitter } from 'node:events'
import { PassThrough } from 'node:stream'
import { mkdtempSync, writeFileSync, readFileSync, existsSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import ts from 'typescript'
import { createYue2Host, validateYue2Request } from '../scripts/yue2-host.mjs'
import { createYueGpHost, validateYueGpRequest } from '../scripts/yuegp-host.mjs'
const request = { id: 'test-song-123456', tags: 'pop, warm vocals', lyrics: '[Verse 1]\nA quiet morning, a new day', seed: 0 }
test('YuE2 defaults are 60 seconds and allow multiple lyric sections', () => {
assert.deepEqual(validateYue2Request(request), { ...request, duration: 60 })
assert.deepEqual(
validateYue2Request({ ...request, lyrics: '[Verse]\nA\n[Chorus]\nB' }).lyrics,
'[Verse]\nA\n[Chorus]\nB'
)
assert.throws(() => validateYue2Request({ ...request, lyrics: '' }))
assert.throws(() => validateYue2Request({ ...request, duration: 10 }))
assert.deepEqual(validateYue2Request({ ...request, duration: 15 }).duration, 15)
assert.throws(() => validateYue2Request({ ...request, id: '../escape' }))
})
test('yue still validates as YuEGP and stays separate from yue2', () => {
assert.equal(validateYueGpRequest(request).profile, 1)
assert.equal(validateYue2Request(request).duration, 60)
assert.notEqual(createYueGpHost.name, createYue2Host.name)
})
function fixture() {
const root = mkdtempSync(join(tmpdir(), 'aigen-yue2-'))
const python = join(root, 'python.exe')
writeFileSync(python, '')
writeFileSync(join(root, 'aigen-ready.json'), JSON.stringify({ model: 'm-a-p/YuE2-3B', vae: 'm-a-p/YuE2-Vae' }))
const child = new EventEmitter()
child.stdout = new PassThrough(); child.stderr = new PassThrough()
child.kill = () => { setImmediate(() => child.emit('close', 1)); return true }
const calls = []
const dataDir = join(root, 'jobs')
const host = createYue2Host({ root, python, dataDir, prepare: async () => {}, leaseValid: () => true,
spawnProcess: (...args) => { calls.push(args); return child } })
return { host, child, calls, root, dataDir }
}
test('host launches yue2-worker, preserves seed 0 and lyrics, and deduplicates POST retries', async () => {
const f = fixture()
await f.host.start(request, 'lease')
await f.host.start(request, 'lease')
assert.equal(f.calls.length, 1)
const [, args, opts] = f.calls[0]
assert.ok(String(args[1]).endsWith('yue2-worker.py'))
assert.equal(opts.windowsHide, true)
assert.equal(opts.shell, false)
const posted = JSON.parse(readFileSync(join(f.dataDir, request.id, 'request.json')))
assert.equal(posted.seed, 0)
assert.equal(posted.lyrics, request.lyrics)
assert.equal(posted.model, 'm-a-p/YuE2-3B')
assert.equal(posted.vae, 'm-a-p/YuE2-Vae')
await assert.rejects(f.host.start({ ...request, id: 'second-song-12345' }, 'lease'), /already running/)
await f.host.cancel(request.id)
assert.equal(f.host.busy(), false)
assert.equal(f.host.read(request.id).status, 'cancelled')
})
test('OOM stays on yue2 with no engine switch', async () => {
const f = fixture()
await f.host.start(request, 'lease')
f.child.stdout.write('AIGEN_EVENT {"stage":"error","error":"CUDA out of memory"}\n')
f.child.emit('close', 1)
assert.match(f.host.read(request.id).error, /out of memory/)
assert.equal(f.calls.length, 1)
})
test('music workflow refuses yue and yue2 while ACE graphs still build', () => {
const source = readFileSync(new URL('../server/utils/musicWorkflow.ts', import.meta.url), 'utf8')
const code = ts.transpileModule(source, { compilerOptions: { target: ts.ScriptTarget.ES2022, module: ts.ModuleKind.CommonJS } }).outputText
const exports = {}
new Function('require', 'exports', 'useRuntimeConfig', code)(id => {
if (id.endsWith('.json')) return { default: JSON.parse(readFileSync(new URL(`../server/assets/${id.split('/').pop()}`, import.meta.url))) }
if (id === '~/utils/music') return { parseMusicBpm: () => 120 }
throw new Error(id)
}, exports, () => ({}))
const params = { tags: 'pop', lyrics: '[Verse]\nhello', duration: 60, steps: 8, seed: 0, cfg: 1, lyricsStrength: 0.9 }
assert.throws(() => exports.buildMusicWorkflow({ ...params, engine: 'yue' }), /standalone YuEGP/)
assert.throws(() => exports.buildMusicWorkflow({ ...params, engine: 'yue2' }), /standalone YuE2/)
for (const engine of ['ace-step', 'ace-step-1.5']) {
const graph = exports.buildMusicWorkflow({ ...params, engine })
assert.ok(Object.values(graph).some(n => n.class_type === 'KSampler' && n.inputs.seed === 0))
}
assert.equal(existsSync(new URL('../server/assets/workflow_yue.json', import.meta.url)), false)
})
test('musicChain is YuE2-only', () => {
const source = readFileSync(new URL('../server/utils/musicChain.ts', import.meta.url), 'utf8')
assert.match(source, /startYue2Job/)
assert.match(source, /from '\.\/yue2'/)
assert.doesNotMatch(source, /startYueGpJob/)
assert.doesNotMatch(source, /buildMusicWorkflow/)
})

Some files were not shown because too many files have changed in this diff Show More