Add Image v2 as a sibling of Image edit, with separate Klein graphs.
Keep the v1 Image tab on its dual-branch template. The host agent follows Desktop's live port so Coolify stays on :8198. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Towsty
+2
-2
@@ -1,7 +1,7 @@
|
||||
PORT=3000
|
||||
HOST=0.0.0.0
|
||||
# ComfyUI on the Windows desktop. 8188 is often taken (SSH / other Comfy).
|
||||
# The host agent discovers the live instance (usually 8189) and proxies it on 8198.
|
||||
# ComfyUI on the Windows desktop. Desktop hops ports (8188/8189/8190/...).
|
||||
# The host agent (scripts/comfy-host-agent.mjs) finds the live instance and proxies it on 8198.
|
||||
COMFY_HOST=http://192.168.77.7:8198
|
||||
COMFY_PORT=8198
|
||||
# Optional: command used when this server runs on the Comfy host
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
# Image v1 vs Image v2
|
||||
|
||||
Image edit (v1) and Image v2 are siblings. v1 is unchanged. Do not mute-fix `workflow_flux2_klein_edit.json` (the dual-branch template that deletes `75:*` or `92:*` at runtime).
|
||||
|
||||
| | Image edit (v1) | Image v2 |
|
||||
|---|---|---|
|
||||
| Tab | Image edit | Image v2 |
|
||||
| Route | `POST /api/edit` | `POST /api/v2/generate` and `POST /v2/generate` |
|
||||
| Graphs | One file, prune unused branch | `klein_v2_edit.json` or `klein_v2_compose.json` |
|
||||
| Second still | Optional; switches branch | Compose only; required. Mismatch is a 400 |
|
||||
| LoRAs | Optional user picker | SNOFS + Consistency, four strengths |
|
||||
| Defaults | 20 steps, CFG 1 | 24 steps, CFG 4 (turbo: 8 / 1) |
|
||||
|
||||
Poll v2 jobs the same way as v1: `GET /api/generate/{id}/stream` and `GET /api/generate/{id}`.
|
||||
|
||||
## Modes
|
||||
|
||||
- **edit** — one image + text. Task is `scene`. Sending `image_b` is rejected.
|
||||
- **compose** — two images + text. `image_b` is required. No silent one-image fallback.
|
||||
|
||||
Compose tasks:
|
||||
|
||||
- **identity** — person from A; B reinforces the face; prompt changes pose/scene
|
||||
- **outfit** — person, body, pose, background from A; clothing only from B
|
||||
- **face_lock** — body/pose/scene from A; face from B
|
||||
- **scene** — A is the edit image; B is style/background
|
||||
|
||||
Dummy test: Compose + two unrelated photos + “keep everything the same” must change the output. If it matches old v1 gens, still B is not connected.
|
||||
|
||||
## Nodes the mapper patches
|
||||
|
||||
Both graphs:
|
||||
|
||||
| Node | Role |
|
||||
|------|------|
|
||||
| `1` | Load Image A (`inputs.image`) |
|
||||
| `2` / `23` | Scale to MP, lanczos (`megapixels`) |
|
||||
| `7` | SNOFS LoRA (`lora_name`, `strength_model`, `strength_clip`) |
|
||||
| `8` | Consistency LoRA (`lora_name`, `strength_model`, `strength_clip`) |
|
||||
| `9` | Positive `CLIPTextEncode.text` (role header + user prompt) |
|
||||
| `10` | Negative `CLIPTextEncode.text` |
|
||||
| `15` | Seed |
|
||||
| `17` | Steps (Flux2Scheduler; Klein-native, euler sampler) |
|
||||
| `18` | CFG |
|
||||
| `21` | SaveImage prefix |
|
||||
|
||||
Compose only:
|
||||
|
||||
| Node | Role |
|
||||
|------|------|
|
||||
| `22` | Load Image B |
|
||||
| `23` | Scale B |
|
||||
| `24` | VAE encode B |
|
||||
| `25` / `26` | Second ReferenceLatent (B fused into pos/neg) |
|
||||
|
||||
If `image_b` is sent and the executed graph has fewer than two `LoadImage` nodes, the job fails.
|
||||
|
||||
## Default sliders
|
||||
|
||||
| Slider | Default |
|
||||
|--------|---------|
|
||||
| SNOFS Model | 0.65 |
|
||||
| SNOFS CLIP | 0.35 |
|
||||
| Consistency Model | 0.70 |
|
||||
| Consistency CLIP | 0.70 |
|
||||
| Steps | 24 (turbo 8) |
|
||||
| CFG | 4 (turbo 1) |
|
||||
| Scale to MP | 1.0 lanczos |
|
||||
|
||||
There is no Denoise slider. This is reference-latent Klein, not inpaint.
|
||||
|
||||
### NSFW identity vs outfit swap
|
||||
|
||||
- **identity / face_lock** — keep Consistency at 0.70 / 0.70 so the face from A (identity) or B (face_lock) holds. Do not drop SNOFS CLIP below ~0.30 or the skin/body read falls apart.
|
||||
- **outfit** — keep SNOFS Model ~0.65 so cloth reads; do not raise it past ~0.85 or it starts rewriting the body from A. Consistency stays at 0.70 so the person in A does not become the person in B.
|
||||
- **scene / edit** — start at the table defaults. Turbo is for drafts only.
|
||||
|
||||
Presets on this tab store sliders + mode + task only. They do not load v1 LoRA stacks or a cached latent.
|
||||
+345
-32
@@ -54,9 +54,21 @@
|
||||
>
|
||||
Image edit
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
class="rounded-full border px-3 py-1 text-xs font-medium"
|
||||
:class="studioMode === 'editv2' ? 'border-amber-300/70 bg-amber-400/10 text-amber-100' : 'border-white/10 text-zinc-400 hover:text-white'"
|
||||
@click="studioMode = 'editv2'; outputFocus = 'edit'"
|
||||
>
|
||||
Image v2
|
||||
</button>
|
||||
</div>
|
||||
<h2 class="font-display text-2xl font-bold">Input</h2>
|
||||
<p class="text-sm text-zinc-400">{{ studioMode === 'edit'
|
||||
<p class="text-sm text-zinc-400">{{ studioMode === 'editv2'
|
||||
? (v2Mode === 'compose'
|
||||
? 'Compose: still A is the person/body. Still B is the face or outfit. Two stills required — no silent one-image fallback.'
|
||||
: 'Edit: one still plus text. Compose is a separate graph if you need a second still.')
|
||||
: studioMode === 'edit'
|
||||
? 'Drop the still to edit. An optional second still applies image 2 onto image 1. Files leave the desktop after save.'
|
||||
: 'Drop a still, describe the motion, then send the job. Generate stays available — a long shot list takes one queue slot, and you can line up more while it runs.' }}</p>
|
||||
</div>
|
||||
@@ -80,14 +92,45 @@
|
||||
<button type="button" class="relative mt-3 rounded-full bg-zinc-950/80 px-3 py-1 text-xs" @click.stop="resetImage">Reset</button>
|
||||
</div>
|
||||
<div v-else class="flex min-h-44 flex-col items-center justify-center text-center">
|
||||
<p class="font-medium">{{ studioMode === 'edit' ? 'Image 1' : (textToVideo ? 'Start still · optional' : 'Choose an image') }}</p>
|
||||
<p class="font-medium">{{ studioMode === 'edit' || studioMode === 'editv2' ? (studioMode === 'editv2' ? 'Still A' : 'Image 1') : (textToVideo ? 'Start still · optional' : 'Choose an image') }}</p>
|
||||
<p class="mt-1 text-sm text-zinc-500">{{ textToVideo ? 'Text-to-video does not need a still. Shot 2+ will still extend from the last frame.' : 'Browse the library or upload a new PNG, JPG, or WEBP' }}</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div v-if="studioMode === 'edit'" class="rounded-2xl border border-dashed border-white/15 bg-zinc-950/50 p-4">
|
||||
<div v-if="studioMode === 'editv2'" class="flex flex-wrap gap-2">
|
||||
<button
|
||||
type="button"
|
||||
class="rounded-full border px-3 py-1 text-xs font-medium"
|
||||
:class="v2Mode === 'edit' ? 'border-amber-300/70 bg-amber-400/10 text-amber-100' : 'border-white/10 text-zinc-400 hover:text-white'"
|
||||
@click="v2Mode = 'edit'; v2Task = 'scene'"
|
||||
>
|
||||
Edit
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
class="rounded-full border px-3 py-1 text-xs font-medium"
|
||||
:class="v2Mode === 'compose' ? 'border-amber-300/70 bg-amber-400/10 text-amber-100' : 'border-white/10 text-zinc-400 hover:text-white'"
|
||||
@click="v2Mode = 'compose'"
|
||||
>
|
||||
Compose
|
||||
</button>
|
||||
<label v-if="v2Mode === 'compose'" class="ml-auto text-xs">
|
||||
<span class="mr-2 text-zinc-500">Task</span>
|
||||
<select
|
||||
v-model="v2Task"
|
||||
class="rounded-lg border border-white/10 bg-zinc-950 px-2 py-1 text-xs text-zinc-300 outline-none ring-amber-300/40 focus:ring-2"
|
||||
>
|
||||
<option value="scene">scene</option>
|
||||
<option value="identity">identity</option>
|
||||
<option value="outfit">outfit</option>
|
||||
<option value="face_lock">face_lock</option>
|
||||
</select>
|
||||
</label>
|
||||
</div>
|
||||
|
||||
<div v-if="studioMode === 'edit' || (studioMode === 'editv2' && v2Mode === 'compose')" class="rounded-2xl border border-dashed border-white/15 bg-zinc-950/50 p-4">
|
||||
<div class="mb-2 flex items-center justify-between gap-2">
|
||||
<p class="text-sm font-medium text-zinc-300">Image 2 · optional</p>
|
||||
<p class="text-sm font-medium text-zinc-300">{{ studioMode === 'editv2' ? 'Still B · required' : 'Image 2 · optional' }}</p>
|
||||
<button
|
||||
v-if="editRefFile"
|
||||
type="button"
|
||||
@@ -116,19 +159,21 @@
|
||||
class="max-h-40 w-full object-contain bg-black/40"
|
||||
>
|
||||
<span v-else class="flex min-h-28 items-center justify-center px-3 py-6 text-center text-sm text-zinc-500">
|
||||
Apply this still onto image 1
|
||||
{{ studioMode === 'editv2' ? stillBHint : 'Apply this still onto image 1' }}
|
||||
</span>
|
||||
</button>
|
||||
<p class="mt-2 text-xs text-zinc-500">
|
||||
{{ editRefFile
|
||||
? 'Two-image branch on. Image 2 is applied onto image 1.'
|
||||
: 'One-image branch on. Add a second still to transfer details from image 2.' }}
|
||||
{{ studioMode === 'editv2'
|
||||
? (editRefFile ? stillBHint : 'Compose will reject this job until still B is loaded.')
|
||||
: (editRefFile
|
||||
? 'Two-image branch on. Image 2 is applied onto image 1.'
|
||||
: 'One-image branch on. Add a second still to transfer details from image 2.') }}
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div>
|
||||
<div class="mb-2 flex flex-wrap items-center justify-between gap-2">
|
||||
<label class="text-sm font-medium text-zinc-300">{{ studioMode === 'edit' ? 'Edit prompt' : 'Motion & scene prompt' }}</label>
|
||||
<label class="text-sm font-medium text-zinc-300">{{ studioMode === 'edit' || studioMode === 'editv2' ? 'Edit prompt' : 'Motion & scene prompt' }}</label>
|
||||
<div class="flex flex-wrap items-center gap-3">
|
||||
<button
|
||||
v-if="identityPrompting"
|
||||
@@ -175,7 +220,7 @@
|
||||
</p>
|
||||
<div
|
||||
class="grid gap-3"
|
||||
:class="studioMode === 'edit' ? 'md:grid-cols-2' : ''"
|
||||
:class="studioMode === 'edit' || studioMode === 'editv2' ? 'md:grid-cols-2' : ''"
|
||||
>
|
||||
<div class="min-w-0">
|
||||
<textarea
|
||||
@@ -194,9 +239,17 @@
|
||||
<span class="mt-2 inline-block text-xs text-amber-200">{{ prompt.trim() ? 'View / edit prompt' : 'Write prompt' }}</span>
|
||||
</button>
|
||||
</div>
|
||||
<label v-if="studioMode === 'edit'" class="block min-w-0 text-sm">
|
||||
<label v-if="studioMode === 'edit' || studioMode === 'editv2'" class="block min-w-0 text-sm">
|
||||
<span class="mb-2 block font-medium text-zinc-300">Negative prompt</span>
|
||||
<textarea
|
||||
v-if="studioMode === 'editv2'"
|
||||
v-model="v2Negative"
|
||||
rows="2"
|
||||
class="min-h-[4.5rem] w-full rounded-2xl border border-white/10 bg-zinc-950/70 px-4 py-3 text-sm outline-none ring-amber-300/40 placeholder:text-zinc-600 [overflow-wrap:anywhere] focus:ring-2 md:h-[calc(100%-1.75rem)] md:min-h-[8.5rem]"
|
||||
placeholder="What to avoid…"
|
||||
/>
|
||||
<textarea
|
||||
v-else
|
||||
v-model="editNegative"
|
||||
rows="2"
|
||||
class="min-h-[4.5rem] w-full rounded-2xl border border-white/10 bg-zinc-950/70 px-4 py-3 text-sm outline-none ring-amber-300/40 placeholder:text-zinc-600 [overflow-wrap:anywhere] focus:ring-2 md:h-[calc(100%-1.75rem)] md:min-h-[8.5rem]"
|
||||
@@ -617,7 +670,62 @@
|
||||
@remove="deleteGenerationPreset"
|
||||
/>
|
||||
</div>
|
||||
<div v-else class="space-y-3">
|
||||
<div v-else-if="studioMode === 'editv2'" class="space-y-3">
|
||||
<p class="text-xs text-zinc-500">Klein v2 · 9B Base · euler. Edit and Compose are separate graphs. Shares the desktop GPU with video.</p>
|
||||
<div class="grid grid-cols-2 gap-3">
|
||||
<label class="block text-sm">
|
||||
<span class="mb-1 block text-zinc-400">SNOFS Model</span>
|
||||
<input v-model.number="v2SnofsModel" type="number" min="0" max="2" step="0.05" class="w-full rounded-xl border border-white/10 bg-zinc-950 px-3 py-2 text-sm outline-none ring-amber-300/40 focus:ring-2">
|
||||
</label>
|
||||
<label class="block text-sm">
|
||||
<span class="mb-1 block text-zinc-400">SNOFS CLIP</span>
|
||||
<input v-model.number="v2SnofsClip" type="number" min="0" max="2" step="0.05" class="w-full rounded-xl border border-white/10 bg-zinc-950 px-3 py-2 text-sm outline-none ring-amber-300/40 focus:ring-2">
|
||||
</label>
|
||||
<label class="block text-sm">
|
||||
<span class="mb-1 block text-zinc-400">Consistency Model</span>
|
||||
<input v-model.number="v2ConsistencyModel" type="number" min="0" max="2" step="0.05" class="w-full rounded-xl border border-white/10 bg-zinc-950 px-3 py-2 text-sm outline-none ring-amber-300/40 focus:ring-2">
|
||||
</label>
|
||||
<label class="block text-sm">
|
||||
<span class="mb-1 block text-zinc-400">Consistency CLIP</span>
|
||||
<input v-model.number="v2ConsistencyClip" type="number" min="0" max="2" step="0.05" class="w-full rounded-xl border border-white/10 bg-zinc-950 px-3 py-2 text-sm outline-none ring-amber-300/40 focus:ring-2">
|
||||
</label>
|
||||
<label class="block text-sm">
|
||||
<span class="mb-1 block text-zinc-400">Steps</span>
|
||||
<input v-model.number="v2Steps" type="number" :min="IMAGE_STEPS_MIN" :max="IMAGE_STEPS_MAX" step="1" :disabled="v2Turbo" class="w-full rounded-xl border border-white/10 bg-zinc-950 px-3 py-2 text-sm outline-none ring-amber-300/40 focus:ring-2 disabled:opacity-50">
|
||||
</label>
|
||||
<label class="block text-sm">
|
||||
<span class="mb-1 block text-zinc-400">CFG</span>
|
||||
<input v-model.number="v2Cfg" type="number" :min="IMAGE_CFG_MIN" :max="IMAGE_CFG_MAX" :step="IMAGE_CFG_STEP" :disabled="v2Turbo" class="w-full rounded-xl border border-white/10 bg-zinc-950 px-3 py-2 text-sm outline-none ring-amber-300/40 focus:ring-2 disabled:opacity-50">
|
||||
</label>
|
||||
<label class="block text-sm">
|
||||
<span class="mb-1 block text-zinc-400">Seed</span>
|
||||
<input v-model="seedInput" class="w-full rounded-xl border border-white/10 bg-zinc-950 px-3 py-2 text-sm" placeholder="random">
|
||||
</label>
|
||||
<label class="block text-sm">
|
||||
<span class="mb-1 block text-zinc-400">Scale to MP</span>
|
||||
<input v-model.number="v2Megapixels" type="number" :min="IMAGE_SCALE_MP_MIN" :max="IMAGE_SCALE_MP_MAX" :step="IMAGE_SCALE_MP_STEP" class="w-full rounded-xl border border-white/10 bg-zinc-950 px-3 py-2 text-sm outline-none ring-amber-300/40 focus:ring-2">
|
||||
</label>
|
||||
</div>
|
||||
<label class="flex cursor-pointer items-center gap-2 text-xs text-zinc-400">
|
||||
<span class="relative inline-flex h-5 w-9 shrink-0 items-center">
|
||||
<input v-model="v2Turbo" type="checkbox" class="peer sr-only" role="switch">
|
||||
<span class="absolute inset-0 rounded-full bg-zinc-700 transition peer-checked:bg-amber-400" />
|
||||
<span class="absolute left-0.5 top-0.5 h-4 w-4 rounded-full bg-white transition peer-checked:translate-x-4" />
|
||||
</span>
|
||||
Turbo · 8 steps, CFG 1
|
||||
</label>
|
||||
<div class="max-w-md">
|
||||
<GenerationPresetBar
|
||||
:presets="imageV2GenerationPresets"
|
||||
v-model:selected-id="imageV2PresetId"
|
||||
v-model:name="imageV2PresetName"
|
||||
@load="loadGenerationPreset"
|
||||
@save="saveGenerationPreset"
|
||||
@remove="deleteGenerationPreset"
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
<div v-else-if="studioMode === 'edit'" class="space-y-3">
|
||||
<p class="text-xs text-zinc-500">Flux.2 Klein 9B. Shares the desktop GPU with video.</p>
|
||||
<div class="max-w-md space-y-2">
|
||||
<LoraSelect v-model="imageLoraStack" :names="imageLoraOptions" label="LoRA" empty-label="None" compact />
|
||||
@@ -795,10 +903,10 @@
|
||||
<button
|
||||
type="button"
|
||||
class="flex-1 rounded-2xl bg-amber-400 px-5 py-3 font-semibold text-zinc-950 transition hover:bg-amber-300 disabled:cursor-not-allowed disabled:opacity-50"
|
||||
:disabled="studioMode === 'edit' ? editBlocked : videoGenerateDisabled"
|
||||
@click="studioMode === 'edit' ? editImage() : generate()"
|
||||
:disabled="studioMode === 'editv2' ? editV2Blocked : (studioMode === 'edit' ? editBlocked : videoGenerateDisabled)"
|
||||
@click="studioMode === 'editv2' ? editImageV2() : (studioMode === 'edit' ? editImage() : generate())"
|
||||
>
|
||||
{{ studioMode === 'edit' ? editSubmitLabel : generateLabel }}
|
||||
{{ studioMode === 'editv2' ? editV2SubmitLabel : (studioMode === 'edit' ? editSubmitLabel : generateLabel) }}
|
||||
</button>
|
||||
<button
|
||||
v-if="videoBusy || editBusy"
|
||||
@@ -810,9 +918,10 @@
|
||||
</button>
|
||||
</div>
|
||||
<p v-if="studioMode === 'video' && videoBusy" class="text-xs text-zinc-400">The current job keeps running in Output. This submit waits in the job queue.</p>
|
||||
<p v-else-if="studioMode === 'edit' && (videoBusy || editBusy)" class="text-xs text-zinc-400">The current job keeps running in Output. This submit waits in the job queue.</p>
|
||||
<p v-else-if="(studioMode === 'edit' || studioMode === 'editv2') && (videoBusy || editBusy)" class="text-xs text-zinc-400">The current job keeps running in Output. This submit waits in the job queue.</p>
|
||||
<p v-if="studioMode === 'video' && videoBlockReason" class="text-xs text-zinc-400">{{ videoBlockReason }}</p>
|
||||
<p v-if="studioMode === 'edit' && editBlockReason" class="text-xs text-zinc-400">{{ editBlockReason }}</p>
|
||||
<p v-if="studioMode === 'editv2' && editV2BlockReason" class="text-xs text-zinc-400">{{ editV2BlockReason }}</p>
|
||||
|
||||
<div v-if="studioJobs.length" class="rounded-2xl border border-white/10 bg-zinc-950/40 p-4">
|
||||
<div class="flex items-start justify-between gap-3">
|
||||
@@ -836,7 +945,7 @@
|
||||
<div class="min-w-0 flex-1">
|
||||
<p class="truncate text-sm font-medium text-zinc-100">{{ job.name }}</p>
|
||||
<p class="mt-0.5 text-xs text-zinc-500">
|
||||
<span v-if="job.kind === 'edit'" class="mr-1 text-amber-200/80">{{ (job.shotCount || 1) > 1 ? 'EDIT' : 'IMAGE' }} ·</span>
|
||||
<span v-if="job.kind === 'edit'" class="mr-1 text-amber-200/80">{{ job.imagePipeline === 'v2' ? 'IMAGE V2' : ((job.shotCount || 1) > 1 ? 'EDIT' : 'IMAGE') }} ·</span>
|
||||
{{ studioJobStatusLabel(job) }}
|
||||
· {{ studioJobShotLine(job) }}
|
||||
</p>
|
||||
@@ -1545,7 +1654,7 @@
|
||||
<PromptDrawer
|
||||
:open="promptDrawerOpen"
|
||||
v-model:text="prompt"
|
||||
:title="studioMode === 'edit' ? 'Edit prompt' : (shotScriptMode ? 'Shot script' : 'Prompt')"
|
||||
:title="studioMode === 'edit' || studioMode === 'editv2' ? 'Edit prompt' : (shotScriptMode ? 'Shot script' : 'Prompt')"
|
||||
:placeholder="promptPlaceholder"
|
||||
@update:open="promptDrawerOpen = $event"
|
||||
/>
|
||||
@@ -1671,6 +1780,18 @@ import {
|
||||
VIDEO_CFG_STEP,
|
||||
type GenerationPreset
|
||||
} from '~/utils/generationPresets'
|
||||
import {
|
||||
IMAGE_V2_CFG_DEFAULT,
|
||||
IMAGE_V2_CONSISTENCY_CLIP,
|
||||
IMAGE_V2_CONSISTENCY_MODEL,
|
||||
IMAGE_V2_SNOFS_CLIP,
|
||||
IMAGE_V2_SNOFS_MODEL,
|
||||
IMAGE_V2_STEPS_DEFAULT,
|
||||
clampImageV2Strength,
|
||||
type ImageV2Mode,
|
||||
type ImageV2PresetSettings,
|
||||
type ImageV2Task
|
||||
} from '~/utils/imageV2'
|
||||
import {
|
||||
coerceVideoWorkflow,
|
||||
composeVideoWorkflow,
|
||||
@@ -1863,10 +1984,13 @@ const shotLoraStacks = ref<Record<number, LoraStackItem[]>>({})
|
||||
const extendLoraStack = ref<LoraStackItem[]>([])
|
||||
const videoGenerationPresets = ref<GenerationPreset[]>([])
|
||||
const imageGenerationPresets = ref<GenerationPreset[]>([])
|
||||
const imageV2GenerationPresets = ref<GenerationPreset[]>([])
|
||||
const videoPresetId = ref('')
|
||||
const imagePresetId = ref('')
|
||||
const imageV2PresetId = ref('')
|
||||
const videoPresetName = ref('')
|
||||
const imagePresetName = ref('')
|
||||
const imageV2PresetName = ref('')
|
||||
const CFG_MIN = VIDEO_CFG_MIN
|
||||
const CFG_MAX = VIDEO_CFG_MAX
|
||||
const CFG_STEP = VIDEO_CFG_STEP
|
||||
@@ -1933,6 +2057,17 @@ const editScaleToTotalPixels = ref(false)
|
||||
const editScaleMegapixels = ref(1)
|
||||
const editRefFile = ref<File | null>(null)
|
||||
const editRefPreview = ref('')
|
||||
const v2Mode = ref<ImageV2Mode>('edit')
|
||||
const v2Task = ref<ImageV2Task>('scene')
|
||||
const v2Negative = ref('')
|
||||
const v2SnofsModel = ref(IMAGE_V2_SNOFS_MODEL)
|
||||
const v2SnofsClip = ref(IMAGE_V2_SNOFS_CLIP)
|
||||
const v2ConsistencyModel = ref(IMAGE_V2_CONSISTENCY_MODEL)
|
||||
const v2ConsistencyClip = ref(IMAGE_V2_CONSISTENCY_CLIP)
|
||||
const v2Steps = ref(IMAGE_V2_STEPS_DEFAULT)
|
||||
const v2Cfg = ref(IMAGE_V2_CFG_DEFAULT)
|
||||
const v2Megapixels = ref(1)
|
||||
const v2Turbo = ref(false)
|
||||
const editChainStep = ref(1)
|
||||
const editChainTotal = ref(1)
|
||||
const editChainLabel = ref('')
|
||||
@@ -2010,6 +2145,7 @@ const studioJobs = ref<Array<{
|
||||
lastError?: string
|
||||
completedCount?: number
|
||||
pendingCount?: number
|
||||
imagePipeline?: string
|
||||
}>>([])
|
||||
const outputStudioJobId = ref('')
|
||||
const videoAwaitingBeast = ref(false)
|
||||
@@ -2044,8 +2180,8 @@ const LOCKS_STORE = 'aigen-global-locks'
|
||||
const REFS_STORE = 'aigen-permanence-refs'
|
||||
const pickerHeading = computed(() => {
|
||||
if (pickerKind.value === 'permanence') return 'Choose a still to name'
|
||||
if (pickerSlot.value === 'main') return studioMode.value === 'edit' ? 'Choose image 1' : 'Choose an image'
|
||||
if (pickerSlot.value === 'editRef') return 'Choose image 2'
|
||||
if (pickerSlot.value === 'main') return studioMode.value === 'edit' || studioMode.value === 'editv2' ? (studioMode.value === 'editv2' ? 'Choose still A' : 'Choose image 1') : 'Choose an image'
|
||||
if (pickerSlot.value === 'editRef') return studioMode.value === 'editv2' ? 'Choose still B' : 'Choose image 2'
|
||||
return `Choose Picture ${Number(pickerSlot.value) + 2}`
|
||||
})
|
||||
const permanencePicker = computed(() => pickerKind.value === 'permanence')
|
||||
@@ -2299,6 +2435,32 @@ const editBlockReason = computed(() => {
|
||||
if (!comfyOk.value && !imageComfyOk.value) return 'Desktop GPU is offline.'
|
||||
return ''
|
||||
})
|
||||
const stillBHint = computed(() => {
|
||||
if (v2Task.value === 'outfit') return 'Clothing only from still B. Person, body, pose, and background stay on still A.'
|
||||
if (v2Task.value === 'face_lock') return 'Exact face from still B. Body, pose, and scene stay on still A.'
|
||||
if (v2Task.value === 'identity') return 'Still B reinforces the face. Still A is the person.'
|
||||
return 'Still A is the edit image. Still B is style or background.'
|
||||
})
|
||||
const editV2Blocked = computed(() => {
|
||||
if (!file.value || !prompt.value.trim() || !folderId.value) return true
|
||||
if (v2Mode.value === 'compose' && !editRefFile.value) return true
|
||||
if (!comfyOk.value && !imageComfyOk.value) return true
|
||||
return false
|
||||
})
|
||||
const editV2BlockReason = computed(() => {
|
||||
if (!file.value) return 'Load still A first.'
|
||||
if (v2Mode.value === 'compose' && !editRefFile.value) return 'Compose requires still B. This will not fall back to one-image generation.'
|
||||
if (!prompt.value.trim()) return 'Write a prompt first.'
|
||||
if (!folderId.value) return 'Choose a library folder before generating.'
|
||||
if (!imageComfyConfigured.value && !comfyOk.value) return 'Image v2 is not configured. Set COMFY_HOST.'
|
||||
if (!comfyOk.value && !imageComfyOk.value) return 'Desktop GPU is offline.'
|
||||
return ''
|
||||
})
|
||||
const editV2SubmitLabel = computed(() => {
|
||||
const occupied = videoBusy.value || editBusy.value || studioJobs.value.some(job => job.status === 'running')
|
||||
const label = v2Mode.value === 'compose' ? 'Compose image' : 'Edit image'
|
||||
return occupied ? `Queue ${label.toLowerCase()}` : label
|
||||
})
|
||||
const generateLabel = computed(() => {
|
||||
const n = queuedExtensionCount.value
|
||||
if (videoBusy.value) return n ? `Queue ${n + 1} shots` : 'Queue job'
|
||||
@@ -2353,6 +2515,12 @@ const composedIdentityPrompt = computed(() => {
|
||||
return buildIdentityPrompt(identityActionPrompt.value, extraIdentityPictures.value)
|
||||
})
|
||||
const promptPlaceholder = computed(() => {
|
||||
if (studioMode.value === 'editv2') {
|
||||
if (v2Mode.value === 'compose' && v2Task.value === 'outfit') return 'Keep everything the same except the clothes from still B.'
|
||||
if (v2Mode.value === 'compose' && v2Task.value === 'face_lock') return 'Keep the body and scene from still A. Use the face from still B.'
|
||||
if (v2Mode.value === 'compose' && v2Task.value === 'identity') return 'Same person as still A. Change the pose and scene.'
|
||||
return 'Make the lighting warmer. Keep the face unchanged.'
|
||||
}
|
||||
if (studioMode.value === 'edit') {
|
||||
return editRefFile.value
|
||||
? 'Apply the dress in image 2 to the person in image 1. Preserve the exact features, proportions, pose, and setting of image 1.'
|
||||
@@ -3064,18 +3232,41 @@ function applyLoraSelection(stack?: unknown, shotStacks?: Array<unknown>) {
|
||||
shotLoraStacks.value = next
|
||||
}
|
||||
|
||||
const currentPresetKind = computed(() => studioMode.value === 'edit' ? 'image' : 'video')
|
||||
const currentPresetKind = computed(() => (
|
||||
studioMode.value === 'editv2' ? 'imagev2' : studioMode.value === 'edit' ? 'image' : 'video'
|
||||
))
|
||||
|
||||
async function loadGenerationPresets() {
|
||||
const [video, image] = await Promise.all([
|
||||
const [video, image, imagev2] = await Promise.all([
|
||||
$fetch<{ presets?: GenerationPreset[] }>('/api/generation-presets', { query: { kind: 'video' } }).catch(() => ({ presets: [] })),
|
||||
$fetch<{ presets?: GenerationPreset[] }>('/api/generation-presets', { query: { kind: 'image' } }).catch(() => ({ presets: [] }))
|
||||
$fetch<{ presets?: GenerationPreset[] }>('/api/generation-presets', { query: { kind: 'image' } }).catch(() => ({ presets: [] })),
|
||||
$fetch<{ presets?: GenerationPreset[] }>('/api/generation-presets', { query: { kind: 'imagev2' } }).catch(() => ({ presets: [] }))
|
||||
])
|
||||
videoGenerationPresets.value = video.presets || []
|
||||
imageGenerationPresets.value = image.presets || []
|
||||
imageV2GenerationPresets.value = imagev2.presets || []
|
||||
}
|
||||
|
||||
function currentPresetSnapshot() {
|
||||
if (studioMode.value === 'editv2') {
|
||||
return {
|
||||
kind: 'imagev2' as const,
|
||||
loraStack: [],
|
||||
settings: {
|
||||
mode: v2Mode.value,
|
||||
task: v2Mode.value === 'compose' ? v2Task.value : 'scene',
|
||||
negative: v2Negative.value,
|
||||
snofsModel: clampImageV2Strength(v2SnofsModel.value, IMAGE_V2_SNOFS_MODEL),
|
||||
snofsClip: clampImageV2Strength(v2SnofsClip.value, IMAGE_V2_SNOFS_CLIP),
|
||||
consistencyModel: clampImageV2Strength(v2ConsistencyModel.value, IMAGE_V2_CONSISTENCY_MODEL),
|
||||
consistencyClip: clampImageV2Strength(v2ConsistencyClip.value, IMAGE_V2_CONSISTENCY_CLIP),
|
||||
steps: clampImageSteps(v2Steps.value, IMAGE_V2_STEPS_DEFAULT),
|
||||
cfg: clampImageCfg(v2Cfg.value, IMAGE_V2_CFG_DEFAULT),
|
||||
megapixels: clampImageScaleMegapixels(v2Megapixels.value, 1),
|
||||
turbo: v2Turbo.value === true
|
||||
}
|
||||
}
|
||||
}
|
||||
if (studioMode.value === 'edit') {
|
||||
return {
|
||||
kind: 'image' as const,
|
||||
@@ -3109,7 +3300,22 @@ function currentPresetSnapshot() {
|
||||
function applyGenerationPreset(preset: GenerationPreset) {
|
||||
const stack = filterLoraStackForStudio(preset.loraStack, ltxEnabled.value)
|
||||
const skipped = preset.loraStack.length - stack.length
|
||||
if (preset.kind === 'image') {
|
||||
if (preset.kind === 'imagev2') {
|
||||
const settings = preset.settings as ImageV2PresetSettings
|
||||
v2Mode.value = settings.mode === 'compose' ? 'compose' : 'edit'
|
||||
v2Task.value = v2Mode.value === 'compose' ? (settings.task || 'scene') : 'scene'
|
||||
if (typeof settings.negative === 'string') v2Negative.value = settings.negative
|
||||
v2SnofsModel.value = clampImageV2Strength(settings.snofsModel, IMAGE_V2_SNOFS_MODEL)
|
||||
v2SnofsClip.value = clampImageV2Strength(settings.snofsClip, IMAGE_V2_SNOFS_CLIP)
|
||||
v2ConsistencyModel.value = clampImageV2Strength(settings.consistencyModel, IMAGE_V2_CONSISTENCY_MODEL)
|
||||
v2ConsistencyClip.value = clampImageV2Strength(settings.consistencyClip, IMAGE_V2_CONSISTENCY_CLIP)
|
||||
v2Steps.value = clampImageSteps(settings.steps, IMAGE_V2_STEPS_DEFAULT)
|
||||
v2Cfg.value = clampImageCfg(settings.cfg, IMAGE_V2_CFG_DEFAULT)
|
||||
v2Megapixels.value = clampImageScaleMegapixels(settings.megapixels, 1)
|
||||
v2Turbo.value = settings.turbo === true
|
||||
imageV2PresetId.value = preset.id
|
||||
imageV2PresetName.value = preset.name
|
||||
} else if (preset.kind === 'image') {
|
||||
imageLoraStack.value = stack
|
||||
const settings = preset.settings as {
|
||||
steps?: number
|
||||
@@ -3168,20 +3374,31 @@ function applyGenerationPreset(preset: GenerationPreset) {
|
||||
})
|
||||
}
|
||||
|
||||
function currentPresetList() {
|
||||
if (currentPresetKind.value === 'imagev2') return imageV2GenerationPresets.value
|
||||
if (currentPresetKind.value === 'image') return imageGenerationPresets.value
|
||||
return videoGenerationPresets.value
|
||||
}
|
||||
|
||||
function loadGenerationPreset(id: string) {
|
||||
const list = currentPresetKind.value === 'image' ? imageGenerationPresets.value : videoGenerationPresets.value
|
||||
const preset = list.find(item => item.id === id)
|
||||
const preset = currentPresetList().find(item => item.id === id)
|
||||
if (!preset) return
|
||||
applyGenerationPreset(preset)
|
||||
}
|
||||
|
||||
async function saveGenerationPreset() {
|
||||
const name = (currentPresetKind.value === 'image' ? imagePresetName.value : videoPresetName.value).trim()
|
||||
const name = (
|
||||
currentPresetKind.value === 'imagev2'
|
||||
? imageV2PresetName.value
|
||||
: currentPresetKind.value === 'image'
|
||||
? imagePresetName.value
|
||||
: videoPresetName.value
|
||||
).trim()
|
||||
if (!name) {
|
||||
toast('Name the preset before saving.')
|
||||
return
|
||||
}
|
||||
const list = currentPresetKind.value === 'image' ? imageGenerationPresets.value : videoGenerationPresets.value
|
||||
const list = currentPresetList()
|
||||
const existing = list.find(item => item.name.toLowerCase() === name.toLowerCase())
|
||||
if (existing && !window.confirm(`Overwrite preset “${existing.name}”?`)) return
|
||||
try {
|
||||
@@ -3190,7 +3407,11 @@ async function saveGenerationPreset() {
|
||||
method: 'POST',
|
||||
body
|
||||
})
|
||||
if (saved.preset.kind === 'image') {
|
||||
if (saved.preset.kind === 'imagev2') {
|
||||
imageV2GenerationPresets.value = saved.presets
|
||||
imageV2PresetId.value = saved.preset.id
|
||||
imageV2PresetName.value = saved.preset.name
|
||||
} else if (saved.preset.kind === 'image') {
|
||||
imageGenerationPresets.value = saved.presets
|
||||
imagePresetId.value = saved.preset.id
|
||||
imagePresetName.value = saved.preset.name
|
||||
@@ -3206,8 +3427,12 @@ async function saveGenerationPreset() {
|
||||
}
|
||||
|
||||
async function deleteGenerationPreset() {
|
||||
const id = currentPresetKind.value === 'image' ? imagePresetId.value : videoPresetId.value
|
||||
const list = currentPresetKind.value === 'image' ? imageGenerationPresets.value : videoGenerationPresets.value
|
||||
const id = currentPresetKind.value === 'imagev2'
|
||||
? imageV2PresetId.value
|
||||
: currentPresetKind.value === 'image'
|
||||
? imagePresetId.value
|
||||
: videoPresetId.value
|
||||
const list = currentPresetList()
|
||||
const preset = list.find(item => item.id === id)
|
||||
if (!id || !preset) return
|
||||
if (!window.confirm(`Delete preset “${preset.name}”?`)) return
|
||||
@@ -3216,7 +3441,10 @@ async function deleteGenerationPreset() {
|
||||
method: 'DELETE',
|
||||
query: { kind: currentPresetKind.value }
|
||||
})
|
||||
if (currentPresetKind.value === 'image') {
|
||||
if (currentPresetKind.value === 'imagev2') {
|
||||
imageV2GenerationPresets.value = saved.presets
|
||||
imageV2PresetId.value = ''
|
||||
} else if (currentPresetKind.value === 'image') {
|
||||
imageGenerationPresets.value = saved.presets
|
||||
imagePresetId.value = ''
|
||||
} else {
|
||||
@@ -4869,6 +5097,91 @@ async function editImage() {
|
||||
}
|
||||
}
|
||||
|
||||
async function editImageV2() {
|
||||
if (editV2Blocked.value) {
|
||||
toast(editV2BlockReason.value || 'Load still A first.')
|
||||
return
|
||||
}
|
||||
if (v2Mode.value === 'compose' && !editRefFile.value) {
|
||||
toast('Compose requires still B. This will not fall back to one-image generation.')
|
||||
return
|
||||
}
|
||||
const hideOut = hideThumbnail.value
|
||||
try {
|
||||
const body = new FormData()
|
||||
body.append('mode', v2Mode.value)
|
||||
body.append('task', v2Mode.value === 'compose' ? v2Task.value : 'scene')
|
||||
body.append('image_a', file.value as File)
|
||||
if (v2Mode.value === 'compose' && editRefFile.value) body.append('image_b', editRefFile.value)
|
||||
body.append('prompt', prompt.value.trim())
|
||||
body.append('negative', v2Negative.value)
|
||||
body.append('snofs_model', String(clampImageV2Strength(v2SnofsModel.value, IMAGE_V2_SNOFS_MODEL)))
|
||||
body.append('snofs_clip', String(clampImageV2Strength(v2SnofsClip.value, IMAGE_V2_SNOFS_CLIP)))
|
||||
body.append('consistency_model', String(clampImageV2Strength(v2ConsistencyModel.value, IMAGE_V2_CONSISTENCY_MODEL)))
|
||||
body.append('consistency_clip', String(clampImageV2Strength(v2ConsistencyClip.value, IMAGE_V2_CONSISTENCY_CLIP)))
|
||||
body.append('steps', String(clampImageSteps(v2Steps.value, IMAGE_V2_STEPS_DEFAULT)))
|
||||
body.append('cfg', String(clampImageCfg(v2Cfg.value, IMAGE_V2_CFG_DEFAULT)))
|
||||
body.append('megapixels', String(clampImageScaleMegapixels(v2Megapixels.value, 1)))
|
||||
body.append('turbo', String(v2Turbo.value === true))
|
||||
body.append('seed', seedInput.value || 'random')
|
||||
body.append('folderId', folderId.value)
|
||||
body.append('hideInput', String(showPrivacyToggles.value && hideInputPreview.value))
|
||||
body.append('hideThumbnail', String(showPrivacyToggles.value && hideThumbnail.value))
|
||||
body.append('name', clipName.value.trim())
|
||||
const downloadName = clipFileName(clipName.value.trim() || prompt.value.split(/[.!?\n]/)[0] || 'aigen-v2')
|
||||
const started = await $fetch<{
|
||||
jobId: string
|
||||
studioJobId?: string
|
||||
queued?: boolean
|
||||
steps: number
|
||||
hideThumbnail?: boolean
|
||||
folderLocked?: boolean
|
||||
}>('/api/v2/generate', {
|
||||
method: 'POST',
|
||||
body
|
||||
})
|
||||
if (started.queued) {
|
||||
toast('Queued Klein v2 behind the current job.')
|
||||
await refreshStudioQueue()
|
||||
return
|
||||
}
|
||||
editStatusBusy.value = false
|
||||
editBusy.value = true
|
||||
stopListen('edit')
|
||||
editSettledUi = false
|
||||
editProgress.value = 2
|
||||
editResultUrl.value = ''
|
||||
editAwaitingReveal.value = false
|
||||
concealEditOutput.value = hideOut
|
||||
editLockedSave.value = false
|
||||
editStatusMessage.value = 'Checking Beast ComfyUI...'
|
||||
editMaxStep.value = started.steps
|
||||
editChainStep.value = 1
|
||||
editChainTotal.value = 1
|
||||
editChainLabel.value = ''
|
||||
editOverallProgress.value = 0
|
||||
editCompletedChainStep.value = 0
|
||||
editActiveChainPlan.value = [{ label: v2Mode.value === 'compose' ? 'Compose' : 'Edit', prompt: prompt.value.trim() }]
|
||||
startTimer('edit')
|
||||
editDownloadName.value = downloadName
|
||||
editJobId.value = started.jobId
|
||||
outputStudioJobId.value = started.studioJobId || ''
|
||||
persistActiveJob('edit', started.jobId, started.hideThumbnail === true || hideOut, started.folderLocked === true)
|
||||
listen('edit', started.jobId, started.hideThumbnail === true || hideOut, started.folderLocked === true)
|
||||
void pollComfyHealth()
|
||||
void refreshStudioQueue()
|
||||
} catch (error: any) {
|
||||
editSettledUi = true
|
||||
const message = error?.data?.statusMessage || error?.statusMessage || error?.message || 'Image v2 failed'
|
||||
editStatusMessage.value = message
|
||||
toast(message)
|
||||
editBusy.value = false
|
||||
editStatusBusy.value = false
|
||||
stopTimer('edit')
|
||||
void pollComfyHealth()
|
||||
}
|
||||
}
|
||||
|
||||
async function useEditAsInput() {
|
||||
if (!editResultUrl.value) return
|
||||
try {
|
||||
|
||||
+5
-1
@@ -457,6 +457,7 @@ type StudioJobRow = {
|
||||
shots?: Queue | null
|
||||
plannedShots?: { prompt: string; duration: number; loraName?: string; loraStack?: LoraStackItem[] }[]
|
||||
completedCount?: number
|
||||
imagePipeline?: string
|
||||
}
|
||||
|
||||
type PendingRemove = {
|
||||
@@ -759,7 +760,10 @@ function listenJob(job: StudioJobRow) {
|
||||
}
|
||||
|
||||
function jobKindLabel(job: StudioJobRow) {
|
||||
if (job.kind === 'edit') return (job.shotCount || 1) > 1 ? 'EDIT' : 'IMAGE'
|
||||
if (job.kind === 'edit') {
|
||||
if (job.imagePipeline === 'v2') return 'IMAGE V2'
|
||||
return (job.shotCount || 1) > 1 ? 'EDIT' : 'IMAGE'
|
||||
}
|
||||
return 'Job'
|
||||
}
|
||||
|
||||
|
||||
@@ -23,36 +23,73 @@ function authorized(req) {
|
||||
return header === `Bearer ${token}`
|
||||
}
|
||||
|
||||
function candidatePorts() {
|
||||
const ports = new Set([8188, 8189])
|
||||
try {
|
||||
const url = new URL(defaultHttp.includes('://') ? defaultHttp : `http://${defaultHttp}`)
|
||||
if (url.port) ports.add(Number(url.port))
|
||||
} catch { /* ignore */ }
|
||||
function lockPorts() {
|
||||
const ports = []
|
||||
try {
|
||||
const dir = join(process.env.APPDATA || '', 'Comfy Desktop', 'port-locks')
|
||||
for (const name of readdirSync(dir)) {
|
||||
const match = name.match(/^port-(\d+)\.json$/)
|
||||
if (match) ports.add(Number(match[1]))
|
||||
if (match) ports.push(Number(match[1]))
|
||||
}
|
||||
} catch { /* ignore */ }
|
||||
return [...ports]
|
||||
return ports
|
||||
}
|
||||
|
||||
async function probe(portNum) {
|
||||
function candidatePorts() {
|
||||
const skip = new Set([port, proxyPort])
|
||||
const ordered = []
|
||||
const seen = new Set()
|
||||
const add = (value) => {
|
||||
const next = Number(value)
|
||||
if (!Number.isInteger(next) || next < 1 || next > 65535) return
|
||||
if (skip.has(next) || seen.has(next)) return
|
||||
seen.add(next)
|
||||
ordered.push(next)
|
||||
}
|
||||
for (const next of lockPorts()) add(next)
|
||||
try {
|
||||
const url = new URL(defaultHttp.includes('://') ? defaultHttp : `http://${defaultHttp}`)
|
||||
if (url.port) add(url.port)
|
||||
} catch { /* ignore */ }
|
||||
for (let next = 8188; next <= 8210; next++) add(next)
|
||||
return ordered
|
||||
}
|
||||
|
||||
async function probeStats(portNum) {
|
||||
try {
|
||||
const res = await fetch(`http://127.0.0.1:${portNum}/system_stats`, { signal: AbortSignal.timeout(2500) })
|
||||
return res.ok
|
||||
if (!res.ok) return null
|
||||
const stats = await res.json()
|
||||
if (!stats?.system) return null
|
||||
return stats
|
||||
} catch {
|
||||
return false
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
function advertisedPort(stats) {
|
||||
const argv = stats?.system?.argv
|
||||
if (!Array.isArray(argv)) return 0
|
||||
const index = argv.findIndex(item => String(item) === '--port')
|
||||
if (index >= 0) return Number(argv[index + 1]) || 0
|
||||
const flag = argv.find(item => /^--port=\d+$/.test(String(item)))
|
||||
if (flag) return Number(String(flag).split('=')[1]) || 0
|
||||
return 0
|
||||
}
|
||||
|
||||
async function findHealthyPort() {
|
||||
const found = []
|
||||
for (const next of candidatePorts()) {
|
||||
if (await probe(next)) return next
|
||||
const stats = await probeStats(next)
|
||||
if (!stats) continue
|
||||
found.push({ port: next, advertised: advertisedPort(stats) })
|
||||
}
|
||||
return 0
|
||||
const native = found.find(item => item.advertised && item.advertised === item.port)
|
||||
if (native) return native.port
|
||||
for (const item of found) {
|
||||
if (item.advertised && found.some(other => other.port === item.advertised)) return item.advertised
|
||||
}
|
||||
return found[0]?.port || 0
|
||||
}
|
||||
|
||||
let proxyServer = null
|
||||
@@ -60,12 +97,11 @@ let proxyTarget = 0
|
||||
|
||||
function ensureProxy(targetPort) {
|
||||
if (!targetPort) return
|
||||
if (proxyServer && proxyTarget === targetPort) return
|
||||
proxyTarget = targetPort
|
||||
if (proxyServer) {
|
||||
try { proxyServer.close() } catch { /* ignore */ }
|
||||
proxyServer = null
|
||||
if (proxyTarget !== targetPort) {
|
||||
proxyTarget = targetPort
|
||||
console.log(JSON.stringify({ src: 'comfy-host-agent', event: 'proxy-target', listen: proxyPort, target: proxyTarget }))
|
||||
}
|
||||
if (proxyServer) return
|
||||
proxyServer = net.createServer((client) => {
|
||||
const upstream = net.connect(proxyTarget, '127.0.0.1')
|
||||
const fail = () => {
|
||||
@@ -85,6 +121,12 @@ function ensureProxy(targetPort) {
|
||||
})
|
||||
}
|
||||
|
||||
async function syncProxy() {
|
||||
const healthyPort = await findHealthyPort()
|
||||
if (healthyPort) ensureProxy(healthyPort)
|
||||
return healthyPort
|
||||
}
|
||||
|
||||
async function processUp() {
|
||||
try {
|
||||
const { stdout } = await execFileAsync('tasklist', ['/FO', 'CSV', '/NH'], { windowsHide: true, timeout: 4000 })
|
||||
@@ -231,8 +273,7 @@ const server = http.createServer(async (req, res) => {
|
||||
if (!authorized(req)) return json(res, 401, { ok: false, error: 'unauthorized' })
|
||||
const url = new URL(req.url || '/', 'http://localhost')
|
||||
if (req.method === 'GET' && url.pathname === '/status') {
|
||||
const [healthyPort, process] = await Promise.all([findHealthyPort(), processUp()])
|
||||
if (healthyPort) ensureProxy(healthyPort)
|
||||
const [healthyPort, process] = await Promise.all([syncProxy(), processUp()])
|
||||
return json(res, 200, {
|
||||
ok: true,
|
||||
http: Boolean(healthyPort),
|
||||
@@ -242,9 +283,8 @@ const server = http.createServer(async (req, res) => {
|
||||
})
|
||||
}
|
||||
if (req.method === 'POST' && url.pathname === '/start') {
|
||||
const healthyPort = await findHealthyPort()
|
||||
const healthyPort = await syncProxy()
|
||||
if (healthyPort) {
|
||||
ensureProxy(healthyPort)
|
||||
return json(res, 200, { ok: true, started: false, already: true, port: healthyPort, proxyPort })
|
||||
}
|
||||
const launched = startComfy()
|
||||
@@ -258,8 +298,7 @@ const server = http.createServer(async (req, res) => {
|
||||
})
|
||||
|
||||
server.listen(port, '0.0.0.0', async () => {
|
||||
const healthyPort = await findHealthyPort()
|
||||
if (healthyPort) ensureProxy(healthyPort)
|
||||
const healthyPort = await syncProxy()
|
||||
console.log(JSON.stringify({
|
||||
src: 'comfy-host-agent',
|
||||
event: 'listen',
|
||||
@@ -268,4 +307,7 @@ server.listen(port, '0.0.0.0', async () => {
|
||||
comfyPort: healthyPort || null,
|
||||
candidates: candidatePorts()
|
||||
}))
|
||||
setInterval(() => {
|
||||
syncProxy().catch(() => {})
|
||||
}, 3000)
|
||||
})
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
@echo off
|
||||
cd /d "%~dp0.."
|
||||
node "%~dp0comfy-host-agent.mjs"
|
||||
@@ -0,0 +1,255 @@
|
||||
import { addStudioJob, kickStudioQueue, listStudioJobs, videoJobsBusy } from '~/server/utils/studioQueue'
|
||||
import { comfyConfigured } from '~/server/utils/comfy'
|
||||
import { imageDimensions } from '~/server/utils/resolution'
|
||||
import { clampImageCfg, clampImageScaleMegapixels, clampImageSteps } from '~/utils/generationPresets'
|
||||
import {
|
||||
IMAGE_V2_CFG_DEFAULT,
|
||||
IMAGE_V2_CONSISTENCY_CLIP,
|
||||
IMAGE_V2_CONSISTENCY_MODEL,
|
||||
IMAGE_V2_SNOFS_CLIP,
|
||||
IMAGE_V2_SNOFS_MODEL,
|
||||
IMAGE_V2_STEPS_DEFAULT,
|
||||
IMAGE_V2_TURBO_CFG,
|
||||
IMAGE_V2_TURBO_STEPS,
|
||||
clampImageV2Strength,
|
||||
parseImageV2Mode,
|
||||
parseImageV2Task,
|
||||
type ImageV2Mode,
|
||||
type ImageV2Task
|
||||
} from '~/utils/imageV2'
|
||||
import { getStill, rememberInputStill, stillPath } from '~/server/utils/library'
|
||||
import { existsSync, readFileSync } from 'node:fs'
|
||||
|
||||
type ImageFile = { filename: string; data: Buffer; type?: string }
|
||||
|
||||
function parseBool(raw: unknown) {
|
||||
return raw === true || raw === 'true' || raw === '1' || raw === 1
|
||||
}
|
||||
|
||||
function isHttpUrl(value: string) {
|
||||
return /^https?:\/\//i.test(value)
|
||||
}
|
||||
|
||||
async function fileFromUrl(url: string): Promise<ImageFile> {
|
||||
const res = await fetch(url, { signal: AbortSignal.timeout(20_000) })
|
||||
if (!res.ok) {
|
||||
throw createError({ statusCode: 400, statusMessage: `Could not fetch image (${res.status})` })
|
||||
}
|
||||
const mime = String(res.headers.get('content-type') || 'image/png').split(';')[0]
|
||||
if (!/^image\//i.test(mime)) {
|
||||
throw createError({ statusCode: 400, statusMessage: 'image_a / image_b URL must be an image' })
|
||||
}
|
||||
const data = Buffer.from(await res.arrayBuffer())
|
||||
if (data.length > 40 * 1024 * 1024) {
|
||||
throw createError({ statusCode: 400, statusMessage: 'Image is too large' })
|
||||
}
|
||||
const ext = mime.includes('jpeg') || mime.includes('jpg') ? '.jpg' : mime.includes('webp') ? '.webp' : '.png'
|
||||
return { filename: `remote${ext}`, data, type: mime }
|
||||
}
|
||||
|
||||
function fileFromStill(ownerKey: string, id: string): ImageFile {
|
||||
const still = getStill(ownerKey, id)
|
||||
const path = stillPath(ownerKey, still.id)
|
||||
if (!existsSync(path)) {
|
||||
throw createError({ statusCode: 400, statusMessage: 'That still is missing from the library' })
|
||||
}
|
||||
return {
|
||||
filename: still.filename || `${still.id}.png`,
|
||||
data: readFileSync(path),
|
||||
type: 'image/png'
|
||||
}
|
||||
}
|
||||
|
||||
async function resolveImageRef(ownerKey: string, raw: unknown, uploaded: ImageFile | null) {
|
||||
if (uploaded) return uploaded
|
||||
const value = String(raw || '').trim()
|
||||
if (!value || value === 'null') return null
|
||||
if (isHttpUrl(value)) return fileFromUrl(value)
|
||||
return fileFromStill(ownerKey, value)
|
||||
}
|
||||
|
||||
function readMultipart(parts: Array<{ name?: string; filename?: string; type?: string; data?: Buffer }> | null) {
|
||||
const fields: Record<string, string> = {}
|
||||
let imageA: ImageFile | null = null
|
||||
let imageB: ImageFile | null = null
|
||||
for (const part of parts || []) {
|
||||
if ((part.name === 'image_a' || part.name === 'image') && part.filename && part.data?.length) {
|
||||
imageA = { filename: part.filename, data: part.data, type: part.type }
|
||||
} else if ((part.name === 'image_b' || part.name === 'image2') && part.filename && part.data?.length) {
|
||||
imageB = { filename: part.filename, data: part.data, type: part.type }
|
||||
} else if (part.name && part.data) {
|
||||
fields[part.name] = part.data.toString('utf8')
|
||||
}
|
||||
}
|
||||
return { fields, imageA, imageB }
|
||||
}
|
||||
|
||||
export default defineEventHandler(async (event) => {
|
||||
const contentType = String(getHeader(event, 'content-type') || '')
|
||||
let fields: Record<string, unknown> = {}
|
||||
let uploadedA: ImageFile | null = null
|
||||
let uploadedB: ImageFile | null = null
|
||||
|
||||
if (contentType.includes('multipart/form-data')) {
|
||||
const form = await readMultipartFormData(event).catch(() => null)
|
||||
const parsed = readMultipart(form || [])
|
||||
fields = parsed.fields
|
||||
uploadedA = parsed.imageA
|
||||
uploadedB = parsed.imageB
|
||||
} else {
|
||||
fields = await readBody<Record<string, unknown>>(event).catch(() => ({}))
|
||||
}
|
||||
|
||||
const mode = parseImageV2Mode(fields.mode)
|
||||
if (!mode) {
|
||||
throw createError({ statusCode: 400, statusMessage: 'mode must be edit or compose' })
|
||||
}
|
||||
const task = parseImageV2Task(fields.task, 'scene')
|
||||
const prompt = String(fields.prompt || '').trim()
|
||||
if (!prompt) {
|
||||
throw createError({ statusCode: 400, statusMessage: 'A prompt is required' })
|
||||
}
|
||||
if (!comfyConfigured()) {
|
||||
throw createError({
|
||||
statusCode: 503,
|
||||
statusMessage: 'Beast ComfyUI is not configured. Set COMFY_HOST.'
|
||||
})
|
||||
}
|
||||
|
||||
if (mode === 'edit' && (task === 'identity' || task === 'outfit' || task === 'face_lock')) {
|
||||
throw createError({
|
||||
statusCode: 400,
|
||||
statusMessage: `${task} requires Compose and image B. Edit is one image only.`
|
||||
})
|
||||
}
|
||||
|
||||
const ownerKey = libraryOwnerKey(event)
|
||||
const imageA = await resolveImageRef(ownerKey, fields.image_a, uploadedA)
|
||||
const imageB = await resolveImageRef(ownerKey, fields.image_b, uploadedB)
|
||||
|
||||
if (!imageA) {
|
||||
throw createError({ statusCode: 400, statusMessage: 'image_a is required' })
|
||||
}
|
||||
if (mode === 'edit' && imageB) {
|
||||
throw createError({
|
||||
statusCode: 400,
|
||||
statusMessage: 'Edit mode takes one image. Use Compose for two stills.'
|
||||
})
|
||||
}
|
||||
if (mode === 'compose' && !imageB) {
|
||||
throw createError({
|
||||
statusCode: 400,
|
||||
statusMessage: 'Compose requires image_b. Refusing to fall back to one-image generation.'
|
||||
})
|
||||
}
|
||||
|
||||
const library = publicLibrary(event)
|
||||
const folderId = library.folders.some(folder => folder.id === String(fields.folderId || ''))
|
||||
? String(fields.folderId)
|
||||
: library.folders[0]?.id
|
||||
if (!folderId) {
|
||||
throw createError({ statusCode: 400, statusMessage: 'Create a library folder before generating' })
|
||||
}
|
||||
assertFolderExists(event, folderId)
|
||||
|
||||
const destFolder = library.folders.find(folder => folder.id === folderId)
|
||||
const folderLocked = Boolean(destFolder?.protected && !destFolder.unlocked)
|
||||
const hideInput = parseBool(fields.hideInput)
|
||||
const hideThumbnail = parseBool(fields.hideThumbnail)
|
||||
const turbo = parseBool(fields.turbo)
|
||||
const steps = turbo ? IMAGE_V2_TURBO_STEPS : clampImageSteps(fields.steps, IMAGE_V2_STEPS_DEFAULT)
|
||||
const cfg = turbo ? IMAGE_V2_TURBO_CFG : clampImageCfg(fields.cfg, IMAGE_V2_CFG_DEFAULT)
|
||||
const megapixels = clampImageScaleMegapixels(fields.megapixels ?? fields.scaleMegapixels, 1)
|
||||
const seed = fields.seed && String(fields.seed) !== 'random'
|
||||
? Number(fields.seed)
|
||||
: Math.floor(Math.random() * 2_147_483_647)
|
||||
const size = imageDimensions(imageA.data)
|
||||
const clipName = String(fields.name || '').trim().slice(0, 80)
|
||||
const v2Mode = mode as ImageV2Mode
|
||||
const v2Task = (mode === 'edit' ? 'scene' : task) as ImageV2Task
|
||||
|
||||
const still = await rememberInputStill({
|
||||
ownerKey,
|
||||
folderId,
|
||||
filename: imageA.filename,
|
||||
data: imageA.data,
|
||||
width: size?.width || 0,
|
||||
height: size?.height || 0,
|
||||
hideInput
|
||||
})
|
||||
const savedRef = imageB
|
||||
? await rememberInputStill({
|
||||
ownerKey,
|
||||
folderId,
|
||||
filename: imageB.filename,
|
||||
data: imageB.data,
|
||||
hideInput
|
||||
})
|
||||
: null
|
||||
|
||||
const studio = await addStudioJob({
|
||||
ownerKey,
|
||||
familyId: crypto.randomUUID(),
|
||||
kind: 'edit',
|
||||
payload: {
|
||||
prompt,
|
||||
name: clipName,
|
||||
folderId,
|
||||
aspect: 'auto',
|
||||
width: size?.width || 0,
|
||||
height: size?.height || 0,
|
||||
steps,
|
||||
turbo,
|
||||
seed,
|
||||
cfg,
|
||||
fps: 24,
|
||||
samplerName: 'euler',
|
||||
scheduler: 'simple',
|
||||
duration: 0,
|
||||
sound: false,
|
||||
workflow: 'v1',
|
||||
useIdentityRefs: false,
|
||||
stillId: still?.id,
|
||||
stillFilename: still?.filename,
|
||||
hideThumbnail,
|
||||
hideInput,
|
||||
folderLocked,
|
||||
referenceStillIds: [null, null, null, null],
|
||||
extensions: [],
|
||||
queueAutoRun: false,
|
||||
negative: String(fields.negative || '').trim(),
|
||||
referenceStillId: savedRef?.id,
|
||||
referenceStillFilename: savedRef?.filename,
|
||||
scaleToTotalPixels: true,
|
||||
scaleMegapixels: megapixels,
|
||||
imagePipeline: 'v2',
|
||||
v2Mode,
|
||||
v2Task,
|
||||
snofsModel: clampImageV2Strength(fields.snofs_model, IMAGE_V2_SNOFS_MODEL),
|
||||
snofsClip: clampImageV2Strength(fields.snofs_clip, IMAGE_V2_SNOFS_CLIP),
|
||||
consistencyModel: clampImageV2Strength(fields.consistency_model, IMAGE_V2_CONSISTENCY_MODEL),
|
||||
consistencyClip: clampImageV2Strength(fields.consistency_clip, IMAGE_V2_CONSISTENCY_CLIP)
|
||||
}
|
||||
})
|
||||
await kickStudioQueue()
|
||||
let latest = listStudioJobs(ownerKey).find(item => item.id === studio.id)
|
||||
if (!latest?.liveJobId && !(await videoJobsBusy())) {
|
||||
await kickStudioQueue()
|
||||
latest = listStudioJobs(ownerKey).find(item => item.id === studio.id)
|
||||
}
|
||||
const liveJobId = latest?.liveJobId || ''
|
||||
|
||||
return {
|
||||
jobId: liveJobId || studio.id,
|
||||
studioJobId: studio.id,
|
||||
queued: !liveJobId,
|
||||
seed,
|
||||
steps,
|
||||
cfg,
|
||||
mode: v2Mode,
|
||||
task: v2Task,
|
||||
workflow: v2Mode === 'compose' ? 'klein_v2_compose.json' : 'klein_v2_edit.json',
|
||||
hideThumbnail,
|
||||
folderLocked
|
||||
}
|
||||
})
|
||||
@@ -0,0 +1,210 @@
|
||||
{
|
||||
"1": {
|
||||
"inputs": { "image": "" },
|
||||
"class_type": "LoadImage",
|
||||
"_meta": { "title": "Load Image A" }
|
||||
},
|
||||
"22": {
|
||||
"inputs": { "image": "" },
|
||||
"class_type": "LoadImage",
|
||||
"_meta": { "title": "Load Image B" }
|
||||
},
|
||||
"2": {
|
||||
"inputs": {
|
||||
"upscale_method": "lanczos",
|
||||
"megapixels": 1,
|
||||
"resolution_steps": 1,
|
||||
"image": ["1", 0]
|
||||
},
|
||||
"class_type": "ImageScaleToTotalPixels",
|
||||
"_meta": { "title": "Scale Image A" }
|
||||
},
|
||||
"23": {
|
||||
"inputs": {
|
||||
"upscale_method": "lanczos",
|
||||
"megapixels": 1,
|
||||
"resolution_steps": 1,
|
||||
"image": ["22", 0]
|
||||
},
|
||||
"class_type": "ImageScaleToTotalPixels",
|
||||
"_meta": { "title": "Scale Image B" }
|
||||
},
|
||||
"3": {
|
||||
"inputs": { "image": ["2", 0] },
|
||||
"class_type": "GetImageSize",
|
||||
"_meta": { "title": "Get Image Size" }
|
||||
},
|
||||
"4": {
|
||||
"inputs": {
|
||||
"unet_name": "flux-2-klein-base-9b-fp8.safetensors",
|
||||
"weight_dtype": "default"
|
||||
},
|
||||
"class_type": "UNETLoader",
|
||||
"_meta": { "title": "Load Flux.2 Klein 9B Base" }
|
||||
},
|
||||
"5": {
|
||||
"inputs": {
|
||||
"clip_name": "qwen_3_8b_fp8mixed.safetensors",
|
||||
"type": "flux2",
|
||||
"device": "default"
|
||||
},
|
||||
"class_type": "CLIPLoader",
|
||||
"_meta": { "title": "Load Qwen 3 8B CLIP" }
|
||||
},
|
||||
"6": {
|
||||
"inputs": { "vae_name": "full_encoder_small_decoder.safetensors" },
|
||||
"class_type": "VAELoader",
|
||||
"_meta": { "title": "Load Klein VAE" }
|
||||
},
|
||||
"7": {
|
||||
"inputs": {
|
||||
"lora_name": "klein_snofs_v1_4.safetensors",
|
||||
"strength_model": 0.65,
|
||||
"strength_clip": 0.35,
|
||||
"model": ["4", 0],
|
||||
"clip": ["5", 0]
|
||||
},
|
||||
"class_type": "LoraLoader",
|
||||
"_meta": { "title": "SNOFS" }
|
||||
},
|
||||
"8": {
|
||||
"inputs": {
|
||||
"lora_name": "Flux2-Klein-9B-consistency-V2.safetensors",
|
||||
"strength_model": 0.7,
|
||||
"strength_clip": 0.7,
|
||||
"model": ["7", 0],
|
||||
"clip": ["7", 1]
|
||||
},
|
||||
"class_type": "LoraLoader",
|
||||
"_meta": { "title": "Consistency" }
|
||||
},
|
||||
"9": {
|
||||
"inputs": {
|
||||
"text": "",
|
||||
"clip": ["8", 1]
|
||||
},
|
||||
"class_type": "CLIPTextEncode",
|
||||
"_meta": { "title": "Positive Prompt" }
|
||||
},
|
||||
"10": {
|
||||
"inputs": {
|
||||
"text": "",
|
||||
"clip": ["8", 1]
|
||||
},
|
||||
"class_type": "CLIPTextEncode",
|
||||
"_meta": { "title": "Negative Prompt" }
|
||||
},
|
||||
"11": {
|
||||
"inputs": {
|
||||
"pixels": ["2", 0],
|
||||
"vae": ["6", 0]
|
||||
},
|
||||
"class_type": "VAEEncode",
|
||||
"_meta": { "title": "VAE Encode A" }
|
||||
},
|
||||
"24": {
|
||||
"inputs": {
|
||||
"pixels": ["23", 0],
|
||||
"vae": ["6", 0]
|
||||
},
|
||||
"class_type": "VAEEncode",
|
||||
"_meta": { "title": "VAE Encode B" }
|
||||
},
|
||||
"12": {
|
||||
"inputs": {
|
||||
"conditioning": ["9", 0],
|
||||
"latent": ["11", 0]
|
||||
},
|
||||
"class_type": "ReferenceLatent",
|
||||
"_meta": { "title": "Reference Latent A+" }
|
||||
},
|
||||
"25": {
|
||||
"inputs": {
|
||||
"conditioning": ["12", 0],
|
||||
"latent": ["24", 0]
|
||||
},
|
||||
"class_type": "ReferenceLatent",
|
||||
"_meta": { "title": "Reference Latent B+" }
|
||||
},
|
||||
"13": {
|
||||
"inputs": {
|
||||
"conditioning": ["10", 0],
|
||||
"latent": ["11", 0]
|
||||
},
|
||||
"class_type": "ReferenceLatent",
|
||||
"_meta": { "title": "Reference Latent A-" }
|
||||
},
|
||||
"26": {
|
||||
"inputs": {
|
||||
"conditioning": ["13", 0],
|
||||
"latent": ["24", 0]
|
||||
},
|
||||
"class_type": "ReferenceLatent",
|
||||
"_meta": { "title": "Reference Latent B-" }
|
||||
},
|
||||
"14": {
|
||||
"inputs": {
|
||||
"width": ["3", 0],
|
||||
"height": ["3", 1],
|
||||
"batch_size": 1
|
||||
},
|
||||
"class_type": "EmptyFlux2LatentImage",
|
||||
"_meta": { "title": "Empty Flux 2 Latent" }
|
||||
},
|
||||
"15": {
|
||||
"inputs": { "noise_seed": 1 },
|
||||
"class_type": "RandomNoise",
|
||||
"_meta": { "title": "RandomNoise" }
|
||||
},
|
||||
"16": {
|
||||
"inputs": { "sampler_name": "euler" },
|
||||
"class_type": "KSamplerSelect",
|
||||
"_meta": { "title": "KSamplerSelect" }
|
||||
},
|
||||
"17": {
|
||||
"inputs": {
|
||||
"steps": 24,
|
||||
"width": ["3", 0],
|
||||
"height": ["3", 1]
|
||||
},
|
||||
"class_type": "Flux2Scheduler",
|
||||
"_meta": { "title": "Flux2Scheduler" }
|
||||
},
|
||||
"18": {
|
||||
"inputs": {
|
||||
"cfg": 4,
|
||||
"model": ["8", 0],
|
||||
"positive": ["25", 0],
|
||||
"negative": ["26", 0]
|
||||
},
|
||||
"class_type": "CFGGuider",
|
||||
"_meta": { "title": "CFG Guider" }
|
||||
},
|
||||
"19": {
|
||||
"inputs": {
|
||||
"noise": ["15", 0],
|
||||
"guider": ["18", 0],
|
||||
"sampler": ["16", 0],
|
||||
"sigmas": ["17", 0],
|
||||
"latent_image": ["14", 0]
|
||||
},
|
||||
"class_type": "SamplerCustomAdvanced",
|
||||
"_meta": { "title": "SamplerCustomAdvanced" }
|
||||
},
|
||||
"20": {
|
||||
"inputs": {
|
||||
"samples": ["19", 0],
|
||||
"vae": ["6", 0]
|
||||
},
|
||||
"class_type": "VAEDecode",
|
||||
"_meta": { "title": "VAE Decode" }
|
||||
},
|
||||
"21": {
|
||||
"inputs": {
|
||||
"filename_prefix": "aigen-v2",
|
||||
"images": ["20", 0]
|
||||
},
|
||||
"class_type": "SaveImage",
|
||||
"_meta": { "title": "Save Image" }
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,171 @@
|
||||
{
|
||||
"1": {
|
||||
"inputs": { "image": "" },
|
||||
"class_type": "LoadImage",
|
||||
"_meta": { "title": "Load Image A" }
|
||||
},
|
||||
"2": {
|
||||
"inputs": {
|
||||
"upscale_method": "lanczos",
|
||||
"megapixels": 1,
|
||||
"resolution_steps": 1,
|
||||
"image": ["1", 0]
|
||||
},
|
||||
"class_type": "ImageScaleToTotalPixels",
|
||||
"_meta": { "title": "Scale Image A" }
|
||||
},
|
||||
"3": {
|
||||
"inputs": { "image": ["2", 0] },
|
||||
"class_type": "GetImageSize",
|
||||
"_meta": { "title": "Get Image Size" }
|
||||
},
|
||||
"4": {
|
||||
"inputs": {
|
||||
"unet_name": "flux-2-klein-base-9b-fp8.safetensors",
|
||||
"weight_dtype": "default"
|
||||
},
|
||||
"class_type": "UNETLoader",
|
||||
"_meta": { "title": "Load Flux.2 Klein 9B Base" }
|
||||
},
|
||||
"5": {
|
||||
"inputs": {
|
||||
"clip_name": "qwen_3_8b_fp8mixed.safetensors",
|
||||
"type": "flux2",
|
||||
"device": "default"
|
||||
},
|
||||
"class_type": "CLIPLoader",
|
||||
"_meta": { "title": "Load Qwen 3 8B CLIP" }
|
||||
},
|
||||
"6": {
|
||||
"inputs": { "vae_name": "full_encoder_small_decoder.safetensors" },
|
||||
"class_type": "VAELoader",
|
||||
"_meta": { "title": "Load Klein VAE" }
|
||||
},
|
||||
"7": {
|
||||
"inputs": {
|
||||
"lora_name": "klein_snofs_v1_4.safetensors",
|
||||
"strength_model": 0.65,
|
||||
"strength_clip": 0.35,
|
||||
"model": ["4", 0],
|
||||
"clip": ["5", 0]
|
||||
},
|
||||
"class_type": "LoraLoader",
|
||||
"_meta": { "title": "SNOFS" }
|
||||
},
|
||||
"8": {
|
||||
"inputs": {
|
||||
"lora_name": "Flux2-Klein-9B-consistency-V2.safetensors",
|
||||
"strength_model": 0.7,
|
||||
"strength_clip": 0.7,
|
||||
"model": ["7", 0],
|
||||
"clip": ["7", 1]
|
||||
},
|
||||
"class_type": "LoraLoader",
|
||||
"_meta": { "title": "Consistency" }
|
||||
},
|
||||
"9": {
|
||||
"inputs": {
|
||||
"text": "",
|
||||
"clip": ["8", 1]
|
||||
},
|
||||
"class_type": "CLIPTextEncode",
|
||||
"_meta": { "title": "Positive Prompt" }
|
||||
},
|
||||
"10": {
|
||||
"inputs": {
|
||||
"text": "",
|
||||
"clip": ["8", 1]
|
||||
},
|
||||
"class_type": "CLIPTextEncode",
|
||||
"_meta": { "title": "Negative Prompt" }
|
||||
},
|
||||
"11": {
|
||||
"inputs": {
|
||||
"pixels": ["2", 0],
|
||||
"vae": ["6", 0]
|
||||
},
|
||||
"class_type": "VAEEncode",
|
||||
"_meta": { "title": "VAE Encode A" }
|
||||
},
|
||||
"12": {
|
||||
"inputs": {
|
||||
"conditioning": ["9", 0],
|
||||
"latent": ["11", 0]
|
||||
},
|
||||
"class_type": "ReferenceLatent",
|
||||
"_meta": { "title": "Reference Latent A+" }
|
||||
},
|
||||
"13": {
|
||||
"inputs": {
|
||||
"conditioning": ["10", 0],
|
||||
"latent": ["11", 0]
|
||||
},
|
||||
"class_type": "ReferenceLatent",
|
||||
"_meta": { "title": "Reference Latent A-" }
|
||||
},
|
||||
"14": {
|
||||
"inputs": {
|
||||
"width": ["3", 0],
|
||||
"height": ["3", 1],
|
||||
"batch_size": 1
|
||||
},
|
||||
"class_type": "EmptyFlux2LatentImage",
|
||||
"_meta": { "title": "Empty Flux 2 Latent" }
|
||||
},
|
||||
"15": {
|
||||
"inputs": { "noise_seed": 1 },
|
||||
"class_type": "RandomNoise",
|
||||
"_meta": { "title": "RandomNoise" }
|
||||
},
|
||||
"16": {
|
||||
"inputs": { "sampler_name": "euler" },
|
||||
"class_type": "KSamplerSelect",
|
||||
"_meta": { "title": "KSamplerSelect" }
|
||||
},
|
||||
"17": {
|
||||
"inputs": {
|
||||
"steps": 24,
|
||||
"width": ["3", 0],
|
||||
"height": ["3", 1]
|
||||
},
|
||||
"class_type": "Flux2Scheduler",
|
||||
"_meta": { "title": "Flux2Scheduler" }
|
||||
},
|
||||
"18": {
|
||||
"inputs": {
|
||||
"cfg": 4,
|
||||
"model": ["8", 0],
|
||||
"positive": ["12", 0],
|
||||
"negative": ["13", 0]
|
||||
},
|
||||
"class_type": "CFGGuider",
|
||||
"_meta": { "title": "CFG Guider" }
|
||||
},
|
||||
"19": {
|
||||
"inputs": {
|
||||
"noise": ["15", 0],
|
||||
"guider": ["18", 0],
|
||||
"sampler": ["16", 0],
|
||||
"sigmas": ["17", 0],
|
||||
"latent_image": ["14", 0]
|
||||
},
|
||||
"class_type": "SamplerCustomAdvanced",
|
||||
"_meta": { "title": "SamplerCustomAdvanced" }
|
||||
},
|
||||
"20": {
|
||||
"inputs": {
|
||||
"samples": ["19", 0],
|
||||
"vae": ["6", 0]
|
||||
},
|
||||
"class_type": "VAEDecode",
|
||||
"_meta": { "title": "VAE Decode" }
|
||||
},
|
||||
"21": {
|
||||
"inputs": {
|
||||
"filename_prefix": "aigen-v2",
|
||||
"images": ["20", 0]
|
||||
},
|
||||
"class_type": "SaveImage",
|
||||
"_meta": { "title": "Save Image" }
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
export { default } from '../../api/v2/generate.post'
|
||||
@@ -15,8 +15,8 @@ export function comfyConfigured() {
|
||||
function comfyBase() {
|
||||
if (comfyHostOverride) return comfyHostOverride
|
||||
const config = useRuntimeConfig()
|
||||
let host = String(config.comfyHost || process.env.COMFY_HOST || '').trim().replace(/\/$/, '')
|
||||
const port = String(config.comfyPort || process.env.COMFY_PORT || '').trim()
|
||||
let host = String(process.env.COMFY_HOST || config.comfyHost || '').trim().replace(/\/$/, '')
|
||||
const port = String(process.env.COMFY_PORT || config.comfyPort || '').trim()
|
||||
if (!host) {
|
||||
throw createError({ statusCode: 500, statusMessage: 'COMFY_HOST is not configured' })
|
||||
}
|
||||
|
||||
@@ -14,6 +14,18 @@ import {
|
||||
type ImagePresetSettings,
|
||||
type VideoPresetSettings
|
||||
} from '~/utils/generationPresets'
|
||||
import {
|
||||
IMAGE_V2_CFG_DEFAULT,
|
||||
IMAGE_V2_CONSISTENCY_CLIP,
|
||||
IMAGE_V2_CONSISTENCY_MODEL,
|
||||
IMAGE_V2_SNOFS_CLIP,
|
||||
IMAGE_V2_SNOFS_MODEL,
|
||||
IMAGE_V2_STEPS_DEFAULT,
|
||||
clampImageV2Strength,
|
||||
parseImageV2Mode,
|
||||
parseImageV2Task,
|
||||
type ImageV2PresetSettings
|
||||
} from '~/utils/imageV2'
|
||||
import { coerceVideoWorkflow, isXaigenStudio, parseVideoWorkflow } from '~/utils/videoModels'
|
||||
|
||||
const writeChains = new Map<string, Promise<unknown>>()
|
||||
@@ -96,6 +108,31 @@ function sanitizeImageSettings(raw: unknown): ImagePresetSettings {
|
||||
}
|
||||
}
|
||||
|
||||
function sanitizeImageV2Settings(raw: unknown): ImageV2PresetSettings {
|
||||
const rec = raw && typeof raw === 'object' ? raw as Record<string, unknown> : {}
|
||||
const mode = parseImageV2Mode(rec.mode) || 'edit'
|
||||
const turbo = rec.turbo === true
|
||||
return {
|
||||
mode,
|
||||
task: mode === 'compose' ? parseImageV2Task(rec.task, 'scene') : 'scene',
|
||||
negative: String(rec.negative || '').slice(0, 2000),
|
||||
snofsModel: clampImageV2Strength(rec.snofsModel ?? rec.snofs_model, IMAGE_V2_SNOFS_MODEL),
|
||||
snofsClip: clampImageV2Strength(rec.snofsClip ?? rec.snofs_clip, IMAGE_V2_SNOFS_CLIP),
|
||||
consistencyModel: clampImageV2Strength(rec.consistencyModel ?? rec.consistency_model, IMAGE_V2_CONSISTENCY_MODEL),
|
||||
consistencyClip: clampImageV2Strength(rec.consistencyClip ?? rec.consistency_clip, IMAGE_V2_CONSISTENCY_CLIP),
|
||||
steps: turbo ? 8 : clampImageSteps(rec.steps, IMAGE_V2_STEPS_DEFAULT),
|
||||
cfg: turbo ? 1 : clampImageCfg(rec.cfg, IMAGE_V2_CFG_DEFAULT),
|
||||
megapixels: clampImageScaleMegapixels(rec.megapixels ?? rec.scaleMegapixels, 1),
|
||||
turbo
|
||||
}
|
||||
}
|
||||
|
||||
function sanitizeSettings(kind: GenerationPresetKind, raw: unknown) {
|
||||
if (kind === 'image') return sanitizeImageSettings(raw)
|
||||
if (kind === 'imagev2') return sanitizeImageV2Settings(raw)
|
||||
return sanitizeVideoSettings(raw)
|
||||
}
|
||||
|
||||
function normalizeStoredPreset(raw: unknown): GenerationPreset | null {
|
||||
if (!raw || typeof raw !== 'object') return null
|
||||
const rec = raw as Record<string, unknown>
|
||||
@@ -111,7 +148,7 @@ function normalizeStoredPreset(raw: unknown): GenerationPreset | null {
|
||||
createdAt,
|
||||
updatedAt: Number(rec.updatedAt) || createdAt,
|
||||
loraStack: normalizeLoraStack(rec.loraStack),
|
||||
settings: kind === 'image' ? sanitizeImageSettings(rec.settings) : sanitizeVideoSettings(rec.settings)
|
||||
settings: sanitizeSettings(kind, rec.settings)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -147,7 +184,7 @@ export async function saveGenerationPreset(owner: string, input: {
|
||||
const kind = parseGenerationPresetKind(input.kind)
|
||||
const xaigen = isXaigenStudio()
|
||||
const loraStack = filterLoraStackForStudio(input.loraStack, xaigen)
|
||||
const settings = kind === 'image' ? sanitizeImageSettings(input.settings) : sanitizeVideoSettings(input.settings)
|
||||
const settings = sanitizeSettings(kind, input.settings)
|
||||
const now = Date.now()
|
||||
return mutate(owner, (presets) => {
|
||||
const existing = presets.find(item => item.kind === kind && item.name.toLowerCase() === name.toLowerCase())
|
||||
|
||||
@@ -0,0 +1,168 @@
|
||||
import { createJob, emitJob, type Job } from '~/server/utils/jobs'
|
||||
import { ensureComfyReady } from '~/server/utils/comfyLifecycle'
|
||||
import { assertImageScaleToTotalPixelsNode, getComfyHost, uploadImage, queuePrompt, purgeComfyArtifacts } from '~/server/utils/comfy'
|
||||
import { withImageComfyHost, waitForImageEdit, downloadEditedImage } from '~/server/utils/imageComfy'
|
||||
import { buildImageV2Workflow } from '~/server/utils/imageWorkflowV2'
|
||||
import { ensureComfyLoraNames } from '~/server/utils/loras'
|
||||
import { imageDimensions } from '~/server/utils/resolution'
|
||||
import { emitChainJob } from '~/server/utils/watch'
|
||||
import { saveStill } from '~/server/utils/library'
|
||||
import type { ImageV2Mode, ImageV2Task } from '~/utils/imageV2'
|
||||
import type { EditImageFile } from '~/server/utils/imageChain'
|
||||
|
||||
export type EditV2RunParams = {
|
||||
mode: ImageV2Mode
|
||||
task: ImageV2Task
|
||||
image: EditImageFile
|
||||
reference: EditImageFile | null
|
||||
prompt: string
|
||||
negative: string
|
||||
steps: number
|
||||
seed: number
|
||||
cfg: number
|
||||
snofsModel: number
|
||||
snofsClip: number
|
||||
consistencyModel: number
|
||||
consistencyClip: number
|
||||
megapixels: number
|
||||
}
|
||||
|
||||
export async function runEditV2(job: Job, params: EditV2RunParams) {
|
||||
const library = job.library
|
||||
if (!library) throw new Error('Edit job is missing library metadata')
|
||||
if (params.mode === 'compose' && !params.reference) {
|
||||
throw new Error('Compose requires image B. Refusing to fall back to one-image generation.')
|
||||
}
|
||||
if (params.mode === 'edit' && params.reference) {
|
||||
throw new Error('Edit mode takes one image. Use Compose for two stills.')
|
||||
}
|
||||
|
||||
try {
|
||||
await ensureComfyReady((status) => {
|
||||
emitChainJob(job, {
|
||||
type: status.state === 'busy' ? 'busy' : 'status',
|
||||
message: status.message,
|
||||
progress: status.state === 'online' ? Math.max(job.progress, 6) : Math.max(job.progress, 3),
|
||||
busy: status.state === 'busy'
|
||||
})
|
||||
}, { skipBusyWait: true })
|
||||
if (job.status === 'cancelled') throw new Error('Job interrupted.')
|
||||
job.imageComfyHost = getComfyHost()
|
||||
|
||||
await withImageComfyHost(job.imageComfyHost, async () => {
|
||||
emitChainJob(job, {
|
||||
type: 'status',
|
||||
message: params.mode === 'compose' ? 'Uploading stills A and B to Beast...' : 'Uploading still A to Beast...',
|
||||
progress: 8
|
||||
})
|
||||
const uploaded = await uploadImage(params.image, job.id)
|
||||
const uploadedRef = params.reference
|
||||
? await uploadImage({
|
||||
...params.reference,
|
||||
filename: `ref_${params.reference.filename || 'image_b.png'}`
|
||||
}, job.id)
|
||||
: null
|
||||
if (job.status === 'cancelled') throw new Error('Job interrupted.')
|
||||
|
||||
emitChainJob(job, {
|
||||
type: 'status',
|
||||
message: params.mode === 'compose'
|
||||
? `Queueing Klein v2 compose (${params.task}) on Beast...`
|
||||
: 'Queueing Klein v2 edit on Beast...',
|
||||
progress: 12
|
||||
})
|
||||
await ensureComfyLoraNames('image')
|
||||
await assertImageScaleToTotalPixelsNode()
|
||||
const built = buildImageV2Workflow({
|
||||
mode: params.mode,
|
||||
task: params.task,
|
||||
prompt: params.prompt,
|
||||
negative: params.negative,
|
||||
imageAName: uploaded.name,
|
||||
imageBName: uploadedRef?.name,
|
||||
snofsModel: params.snofsModel,
|
||||
snofsClip: params.snofsClip,
|
||||
consistencyModel: params.consistencyModel,
|
||||
consistencyClip: params.consistencyClip,
|
||||
steps: params.steps,
|
||||
cfg: params.cfg,
|
||||
seed: params.seed,
|
||||
megapixels: params.megapixels,
|
||||
filenamePrefix: `aigen_v2_${job.id.slice(0, 8)}`
|
||||
})
|
||||
const queued = await queuePrompt(built.graph, job.clientId)
|
||||
job.promptId = queued.prompt_id
|
||||
job.status = 'running'
|
||||
emitChainJob(job, {
|
||||
type: 'status',
|
||||
message: `Running ${built.workflowFile}...`,
|
||||
progress: 18,
|
||||
maxStep: params.steps
|
||||
})
|
||||
|
||||
const output = await waitForImageEdit({
|
||||
promptId: queued.prompt_id,
|
||||
clientId: job.clientId,
|
||||
timeoutMs: 10 * 60 * 1000,
|
||||
onProgress: (event) => {
|
||||
emitChainJob(job, {
|
||||
type: 'status',
|
||||
message: event.message,
|
||||
progress: event.progress,
|
||||
step: event.step,
|
||||
maxStep: event.maxStep || params.steps,
|
||||
node: event.node
|
||||
})
|
||||
},
|
||||
isCancelled: () => job.status === 'cancelled'
|
||||
})
|
||||
|
||||
emitChainJob(job, { type: 'status', message: 'Saving Klein v2 still...', progress: 94 })
|
||||
const buffer = await downloadEditedImage(output)
|
||||
const size = imageDimensions(buffer)
|
||||
const still = await saveStill({
|
||||
ownerKey: library.ownerKey,
|
||||
folderId: library.folderId,
|
||||
filename: output.filename,
|
||||
data: buffer,
|
||||
width: size?.width || 0,
|
||||
height: size?.height || 0,
|
||||
hideInput: job.hideThumbnail === true,
|
||||
role: 'output',
|
||||
name: library.name || undefined,
|
||||
prompt: built.prompt
|
||||
})
|
||||
job.stillId = still?.id
|
||||
await purgeComfyArtifacts({
|
||||
video: { filename: output.filename, subfolder: output.subfolder, type: output.type },
|
||||
imageName: uploaded.name,
|
||||
imageSubfolder: uploaded.subfolder,
|
||||
extraImageNames: uploadedRef?.name ? [uploadedRef.name] : [],
|
||||
promptId: job.promptId
|
||||
})
|
||||
|
||||
job.status = 'complete'
|
||||
emitChainJob(job, {
|
||||
type: 'complete',
|
||||
message: 'Klein v2 finished on Beast',
|
||||
progress: 100,
|
||||
stillId: still?.id,
|
||||
filename: output.filename,
|
||||
subfolder: output.subfolder,
|
||||
mediaType: 'image',
|
||||
hideThumbnail: job.hideThumbnail,
|
||||
folderLocked: library.folderLocked
|
||||
})
|
||||
})
|
||||
} catch (error) {
|
||||
if (job.status !== 'error' && job.status !== 'cancelled') {
|
||||
const message = error instanceof Error ? error.message : String(error)
|
||||
job.status = 'error'
|
||||
job.error = message
|
||||
emitJob(job, { type: 'error', error: message, message })
|
||||
}
|
||||
} finally {
|
||||
const { onLiveVideoSettled } = await import('~/server/utils/studioQueue')
|
||||
await onLiveVideoSettled(job)
|
||||
}
|
||||
}
|
||||
@@ -292,7 +292,7 @@ export async function waitForImageEdit(opts: {
|
||||
if (payload.type === 'executing' && payload.data?.node) {
|
||||
opts.onProgress?.({
|
||||
message: 'Running Flux.2 Klein...',
|
||||
progress: payload.data.node === '9' || payload.data.node === '94' ? 92 : 30,
|
||||
progress: payload.data.node === '9' || payload.data.node === '94' || payload.data.node === '21' ? 92 : 30,
|
||||
node: payload.data.node
|
||||
})
|
||||
}
|
||||
|
||||
@@ -0,0 +1,193 @@
|
||||
import editTemplate from '../assets/klein_v2_edit.json'
|
||||
import composeTemplate from '../assets/klein_v2_compose.json'
|
||||
import { IMAGE_SCALE_TO_TOTAL_PIXELS } from '~/server/utils/comfy'
|
||||
import { cachedComfyLoraNames } from '~/server/utils/loras'
|
||||
import { resolveComfyLoraName, loraIdentityKey } from '~/utils/loras'
|
||||
import { clampImageCfg, clampImageScaleMegapixels, clampImageSteps } from '~/utils/generationPresets'
|
||||
import {
|
||||
IMAGE_V2_CONSISTENCY_CLIP,
|
||||
IMAGE_V2_CONSISTENCY_LORA,
|
||||
IMAGE_V2_CONSISTENCY_MODEL,
|
||||
IMAGE_V2_SNOFS_CLIP,
|
||||
IMAGE_V2_SNOFS_LORA,
|
||||
IMAGE_V2_SNOFS_MODEL,
|
||||
clampImageV2Strength,
|
||||
composeImageV2Prompt,
|
||||
type ImageV2Mode,
|
||||
type ImageV2Task
|
||||
} from '~/utils/imageV2'
|
||||
|
||||
type WorkflowNode = { class_type: string; inputs: Record<string, unknown>; _meta?: { title?: string } }
|
||||
type WorkflowGraph = Record<string, WorkflowNode>
|
||||
|
||||
const LOAD_A = '1'
|
||||
const LOAD_B = '22'
|
||||
const SCALE_A = '2'
|
||||
const SCALE_B = '23'
|
||||
const PROMPT = '9'
|
||||
const NEGATIVE = '10'
|
||||
const NOISE = '15'
|
||||
const SCHEDULER = '17'
|
||||
const CFG = '18'
|
||||
const SAVE = '21'
|
||||
const SNOFS = '7'
|
||||
const CONSISTENCY = '8'
|
||||
|
||||
export const IMAGE_V2_EDIT_WORKFLOW = 'klein_v2_edit.json'
|
||||
export const IMAGE_V2_COMPOSE_WORKFLOW = 'klein_v2_compose.json'
|
||||
|
||||
export interface ImageV2BuildParams {
|
||||
mode: ImageV2Mode
|
||||
task: ImageV2Task
|
||||
prompt: string
|
||||
negative?: string
|
||||
imageAName: string
|
||||
imageBName?: string
|
||||
snofsModel?: number
|
||||
snofsClip?: number
|
||||
consistencyModel?: number
|
||||
consistencyClip?: number
|
||||
steps: number
|
||||
cfg: number
|
||||
seed: number
|
||||
megapixels?: number
|
||||
filenamePrefix?: string
|
||||
}
|
||||
|
||||
function setInput(graph: WorkflowGraph, id: string, key: string, value: unknown) {
|
||||
if (graph[id]) graph[id].inputs[key] = value
|
||||
}
|
||||
|
||||
function loadImageNames(graph: WorkflowGraph) {
|
||||
return Object.entries(graph)
|
||||
.filter(([, node]) => node.class_type === 'LoadImage')
|
||||
.map(([id, node]) => ({
|
||||
id,
|
||||
title: String(node._meta?.title || id),
|
||||
image: String(node.inputs.image || '')
|
||||
}))
|
||||
}
|
||||
|
||||
function resolveRequiredLora(wanted: string, label: string) {
|
||||
const names = cachedComfyLoraNames('image')
|
||||
const resolved = names.length ? resolveComfyLoraName(wanted, names) : wanted
|
||||
const hit = names.some(name => loraIdentityKey(name) === loraIdentityKey(wanted) || loraIdentityKey(name) === loraIdentityKey(resolved))
|
||||
if (names.length && !hit) {
|
||||
throw createError({ statusCode: 503, statusMessage: `Missing ${label} LoRA (${wanted}) on Beast Comfy` })
|
||||
}
|
||||
return resolved
|
||||
}
|
||||
|
||||
function patchScaleMegapixels(graph: WorkflowGraph, megapixels: number) {
|
||||
const mp = clampImageScaleMegapixels(megapixels)
|
||||
for (const node of Object.values(graph)) {
|
||||
if (node.class_type !== IMAGE_SCALE_TO_TOTAL_PIXELS) continue
|
||||
node.inputs.megapixels = mp
|
||||
node.inputs.upscale_method = 'lanczos'
|
||||
}
|
||||
}
|
||||
|
||||
export function assertImageV2Graph(graph: WorkflowGraph, mode: ImageV2Mode, imageBName?: string) {
|
||||
const loaders = loadImageNames(graph)
|
||||
if (mode === 'compose') {
|
||||
if (loaders.length < 2) {
|
||||
throw createError({
|
||||
statusCode: 500,
|
||||
statusMessage: 'Compose graph has no second image input. Refusing to run a one-image fallback.'
|
||||
})
|
||||
}
|
||||
const b = graph[LOAD_B]
|
||||
if (!b || b.class_type !== 'LoadImage' || !String(b.inputs.image || '').trim()) {
|
||||
throw createError({
|
||||
statusCode: 500,
|
||||
statusMessage: 'Compose job is missing Load Image B. Refusing to run.'
|
||||
})
|
||||
}
|
||||
}
|
||||
if (imageBName && loaders.length < 2) {
|
||||
throw createError({
|
||||
statusCode: 500,
|
||||
statusMessage: 'image_b was sent but the executed graph has no second image input.'
|
||||
})
|
||||
}
|
||||
const promptNode = graph[PROMPT]
|
||||
if (!promptNode || promptNode.class_type !== 'CLIPTextEncode') {
|
||||
throw createError({ statusCode: 500, statusMessage: 'v2 graph is missing the positive CLIPTextEncode node.' })
|
||||
}
|
||||
if (Array.isArray(promptNode.inputs.text)) {
|
||||
throw createError({ statusCode: 500, statusMessage: 'v2 prompt is a subgraph link. Refusing to run with a leftover widget prompt.' })
|
||||
}
|
||||
if (!String(promptNode.inputs.text || '').trim()) {
|
||||
throw createError({ statusCode: 400, statusMessage: 'v2 prompt was not patched onto the graph.' })
|
||||
}
|
||||
}
|
||||
|
||||
export function buildImageV2Workflow(params: ImageV2BuildParams) {
|
||||
const compose = params.mode === 'compose'
|
||||
const graph = structuredClone(compose ? composeTemplate : editTemplate) as WorkflowGraph
|
||||
const prompt = composeImageV2Prompt(params.mode, params.task, params.prompt)
|
||||
const negative = String(params.negative || '')
|
||||
const snofsModel = clampImageV2Strength(params.snofsModel, IMAGE_V2_SNOFS_MODEL)
|
||||
const snofsClip = clampImageV2Strength(params.snofsClip, IMAGE_V2_SNOFS_CLIP)
|
||||
const consistencyModel = clampImageV2Strength(params.consistencyModel, IMAGE_V2_CONSISTENCY_MODEL)
|
||||
const consistencyClip = clampImageV2Strength(params.consistencyClip, IMAGE_V2_CONSISTENCY_CLIP)
|
||||
const steps = clampImageSteps(params.steps, 24)
|
||||
const cfg = clampImageCfg(params.cfg, 4)
|
||||
const workflowFile = compose ? IMAGE_V2_COMPOSE_WORKFLOW : IMAGE_V2_EDIT_WORKFLOW
|
||||
|
||||
setInput(graph, LOAD_A, 'image', params.imageAName)
|
||||
if (compose) setInput(graph, LOAD_B, 'image', params.imageBName || '')
|
||||
setInput(graph, PROMPT, 'text', prompt)
|
||||
setInput(graph, NEGATIVE, 'text', negative)
|
||||
setInput(graph, NOISE, 'noise_seed', params.seed)
|
||||
setInput(graph, SCHEDULER, 'steps', steps)
|
||||
setInput(graph, CFG, 'cfg', cfg)
|
||||
setInput(graph, SAVE, 'filename_prefix', params.filenamePrefix || 'aigen-v2')
|
||||
patchScaleMegapixels(graph, params.megapixels ?? 1)
|
||||
|
||||
setInput(graph, SNOFS, 'lora_name', resolveRequiredLora(IMAGE_V2_SNOFS_LORA, 'SNOFS'))
|
||||
setInput(graph, SNOFS, 'strength_model', snofsModel)
|
||||
setInput(graph, SNOFS, 'strength_clip', snofsClip)
|
||||
setInput(graph, CONSISTENCY, 'lora_name', resolveRequiredLora(IMAGE_V2_CONSISTENCY_LORA, 'Consistency'))
|
||||
setInput(graph, CONSISTENCY, 'strength_model', consistencyModel)
|
||||
setInput(graph, CONSISTENCY, 'strength_clip', consistencyClip)
|
||||
|
||||
assertImageV2Graph(graph, params.mode, params.imageBName)
|
||||
|
||||
const loaders = loadImageNames(graph)
|
||||
console.log(JSON.stringify({
|
||||
src: 'image-v2',
|
||||
workflow: workflowFile,
|
||||
mode: params.mode,
|
||||
task: params.task,
|
||||
loadImage: Object.fromEntries(loaders.map(item => [item.id, { title: item.title, file: item.image }])),
|
||||
loras: {
|
||||
snofs: { name: graph[SNOFS]?.inputs.lora_name, model: snofsModel, clip: snofsClip },
|
||||
consistency: { name: graph[CONSISTENCY]?.inputs.lora_name, model: consistencyModel, clip: consistencyClip }
|
||||
},
|
||||
steps,
|
||||
cfg,
|
||||
seed: params.seed,
|
||||
megapixels: clampImageScaleMegapixels(params.megapixels ?? 1)
|
||||
}))
|
||||
|
||||
return { graph, workflowFile, loaders, prompt }
|
||||
}
|
||||
|
||||
export const IMAGE_V2_NODE_LABELS: Record<string, string> = {
|
||||
'1': 'Loading image A',
|
||||
'22': 'Loading image B',
|
||||
'2': 'Scaling image A',
|
||||
'23': 'Scaling image B',
|
||||
'4': 'Loading Flux.2 Klein 9B',
|
||||
'5': 'Loading CLIP',
|
||||
'6': 'Loading VAE',
|
||||
'7': 'Applying SNOFS',
|
||||
'8': 'Applying Consistency',
|
||||
'9': 'Encoding prompt',
|
||||
'11': 'Encoding image A',
|
||||
'24': 'Encoding image B',
|
||||
'19': 'Sampling Klein v2',
|
||||
'20': 'Decoding still',
|
||||
'21': 'Saving still'
|
||||
}
|
||||
@@ -51,6 +51,13 @@ export interface StudioJobPayload {
|
||||
referenceStillFilename?: string
|
||||
scaleToTotalPixels?: boolean
|
||||
scaleMegapixels?: number
|
||||
imagePipeline?: 'v1' | 'v2'
|
||||
v2Mode?: 'edit' | 'compose'
|
||||
v2Task?: 'scene' | 'identity' | 'outfit' | 'face_lock'
|
||||
snofsModel?: number
|
||||
snofsClip?: number
|
||||
consistencyModel?: number
|
||||
consistencyClip?: number
|
||||
}
|
||||
|
||||
export interface StudioJob {
|
||||
@@ -210,6 +217,7 @@ export function summarizeStudioJob(job: StudioJob) {
|
||||
liveJobId: job.liveJobId,
|
||||
stillId: job.payload.stillId,
|
||||
workflow: job.payload.workflow,
|
||||
imagePipeline: job.payload.imagePipeline || 'v1',
|
||||
duration: job.payload.duration,
|
||||
hideThumbnail: job.payload.hideThumbnail,
|
||||
folderLocked: job.payload.folderLocked === true,
|
||||
@@ -665,6 +673,7 @@ async function startStudioEditJob(item: StudioJob) {
|
||||
const { stillPath } = await import('~/server/utils/library')
|
||||
const { existsSync, readFileSync } = await import('node:fs')
|
||||
const { createEditLiveJob, runEdit } = await import('~/server/utils/imageChain')
|
||||
const { runEditV2 } = await import('~/server/utils/imageChainV2')
|
||||
|
||||
const payload = item.payload
|
||||
if (!payload.stillId || !existsSync(stillPath(item.ownerKey, payload.stillId))) {
|
||||
@@ -684,6 +693,66 @@ async function startStudioEditJob(item: StudioJob) {
|
||||
}
|
||||
: null
|
||||
|
||||
if (payload.imagePipeline === 'v2') {
|
||||
const mode = payload.v2Mode === 'compose' ? 'compose' : 'edit'
|
||||
if (mode === 'compose' && !reference) {
|
||||
throw new Error('Compose requires image B. Refusing to fall back to one-image generation.')
|
||||
}
|
||||
if (mode === 'edit' && reference) {
|
||||
throw new Error('Edit mode takes one image. Use Compose for two stills.')
|
||||
}
|
||||
live = createEditLiveJob({
|
||||
steps: payload.steps,
|
||||
hideThumbnail: payload.hideThumbnail,
|
||||
library: {
|
||||
ownerKey: item.ownerKey,
|
||||
folderId: payload.folderId,
|
||||
hideThumbnail: payload.hideThumbnail,
|
||||
hideInput: payload.hideInput,
|
||||
folderLocked: payload.folderLocked,
|
||||
name: payload.name,
|
||||
prompt: payload.prompt,
|
||||
aspect: payload.aspect || 'auto',
|
||||
width: payload.width,
|
||||
height: payload.height,
|
||||
steps: payload.steps,
|
||||
turbo: payload.turbo === true,
|
||||
seed: payload.seed || Math.floor(Math.random() * 2_147_483_647),
|
||||
cfg: payload.cfg,
|
||||
stillId: payload.stillId,
|
||||
stillFilename: payload.stillFilename,
|
||||
familyId: item.familyId,
|
||||
chainIndex: 0,
|
||||
chainStep: 1,
|
||||
chainTotal: 1
|
||||
}
|
||||
})
|
||||
await markStudioLive(item.ownerKey, item.id, live.id)
|
||||
void runEditV2(live, {
|
||||
mode,
|
||||
task: payload.v2Task || 'scene',
|
||||
image,
|
||||
reference,
|
||||
prompt: payload.prompt,
|
||||
negative: payload.negative || '',
|
||||
steps: payload.steps,
|
||||
seed: live.library?.seed || payload.seed,
|
||||
cfg: payload.cfg,
|
||||
snofsModel: payload.snofsModel ?? 0.65,
|
||||
snofsClip: payload.snofsClip ?? 0.35,
|
||||
consistencyModel: payload.consistencyModel ?? 0.7,
|
||||
consistencyClip: payload.consistencyClip ?? 0.7,
|
||||
megapixels: payload.scaleMegapixels ?? 1
|
||||
}).catch((error) => {
|
||||
const message = error instanceof Error ? error.message : String(error)
|
||||
if (live && live.status !== 'error' && live.status !== 'cancelled' && live.status !== 'deferred') {
|
||||
live.status = 'error'
|
||||
live.error = message
|
||||
}
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
const passes = payload.passes || []
|
||||
live = createEditLiveJob({
|
||||
steps: payload.steps,
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
import type { LoraStackItem } from '~/utils/loras'
|
||||
import type { VideoWorkflowId } from '~/utils/videoModels'
|
||||
import type { ImageV2PresetSettings } from '~/utils/imageV2'
|
||||
|
||||
export type GenerationPresetKind = 'video' | 'image'
|
||||
export type GenerationPresetKind = 'video' | 'image' | 'imagev2'
|
||||
|
||||
export type VideoPresetSettings = {
|
||||
workflow?: VideoWorkflowId
|
||||
@@ -75,11 +76,13 @@ export type GenerationPreset = {
|
||||
createdAt: number
|
||||
updatedAt: number
|
||||
loraStack: LoraStackItem[]
|
||||
settings: VideoPresetSettings | ImagePresetSettings
|
||||
settings: VideoPresetSettings | ImagePresetSettings | ImageV2PresetSettings
|
||||
}
|
||||
|
||||
export function parseGenerationPresetKind(raw: unknown): GenerationPresetKind {
|
||||
return raw === 'image' ? 'image' : 'video'
|
||||
if (raw === 'image') return 'image'
|
||||
if (raw === 'imagev2') return 'imagev2'
|
||||
return 'video'
|
||||
}
|
||||
|
||||
export function normalizePresetName(raw: unknown) {
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
export const IMAGE_V2_MODES = ['edit', 'compose'] as const
|
||||
export const IMAGE_V2_TASKS = ['scene', 'identity', 'outfit', 'face_lock'] as const
|
||||
|
||||
export type ImageV2Mode = (typeof IMAGE_V2_MODES)[number]
|
||||
export type ImageV2Task = (typeof IMAGE_V2_TASKS)[number]
|
||||
|
||||
export const IMAGE_V2_STEPS_DEFAULT = 24
|
||||
export const IMAGE_V2_CFG_DEFAULT = 4
|
||||
export const IMAGE_V2_TURBO_STEPS = 8
|
||||
export const IMAGE_V2_TURBO_CFG = 1
|
||||
export const IMAGE_V2_SNOFS_MODEL = 0.65
|
||||
export const IMAGE_V2_SNOFS_CLIP = 0.35
|
||||
export const IMAGE_V2_CONSISTENCY_MODEL = 0.7
|
||||
export const IMAGE_V2_CONSISTENCY_CLIP = 0.7
|
||||
export const IMAGE_V2_STRENGTH_MIN = 0
|
||||
export const IMAGE_V2_STRENGTH_MAX = 2
|
||||
export const IMAGE_V2_STRENGTH_STEP = 0.05
|
||||
|
||||
export const IMAGE_V2_SNOFS_LORA = 'klein_snofs_v1_4.safetensors'
|
||||
export const IMAGE_V2_CONSISTENCY_LORA = 'Flux2-Klein-9B-consistency-V2.safetensors'
|
||||
|
||||
export const IMAGE_V2_ROLE_HEADERS: Record<Exclude<ImageV2Task, 'scene'>, string> = {
|
||||
outfit: 'Person, face, body, pose, and background from image 1. Clothing only from image 2. Fit the outfit from image 2 to the body in image 1. Do not copy image 2’s face, body shape, or pose.',
|
||||
face_lock: 'Body, pose, and scene from image 1. Exact face from image 2.',
|
||||
identity: 'Same person as image 1. Use image 2 only to reinforce the face. Follow the user’s pose/scene prompt.'
|
||||
}
|
||||
|
||||
export function parseImageV2Mode(raw: unknown): ImageV2Mode | null {
|
||||
const value = String(raw || '').trim().toLowerCase()
|
||||
return IMAGE_V2_MODES.includes(value as ImageV2Mode) ? value as ImageV2Mode : null
|
||||
}
|
||||
|
||||
export function parseImageV2Task(raw: unknown, fallback: ImageV2Task = 'scene'): ImageV2Task {
|
||||
const value = String(raw || '').trim().toLowerCase()
|
||||
return IMAGE_V2_TASKS.includes(value as ImageV2Task) ? value as ImageV2Task : fallback
|
||||
}
|
||||
|
||||
export function clampImageV2Strength(raw: unknown, fallback: number) {
|
||||
const value = Number(raw)
|
||||
if (!Number.isFinite(value)) return fallback
|
||||
const snapped = Math.round(value / IMAGE_V2_STRENGTH_STEP) * IMAGE_V2_STRENGTH_STEP
|
||||
return Math.min(IMAGE_V2_STRENGTH_MAX, Math.max(IMAGE_V2_STRENGTH_MIN, Math.round(snapped * 100) / 100))
|
||||
}
|
||||
|
||||
export function composeImageV2Prompt(mode: ImageV2Mode, task: ImageV2Task, prompt: string) {
|
||||
const body = String(prompt || '').trim()
|
||||
if (mode !== 'compose' || task === 'scene') return body
|
||||
const header = IMAGE_V2_ROLE_HEADERS[task]
|
||||
return header ? `${header}\n\n${body}` : body
|
||||
}
|
||||
|
||||
export type ImageV2PresetSettings = {
|
||||
mode?: ImageV2Mode
|
||||
task?: ImageV2Task
|
||||
negative?: string
|
||||
snofsModel?: number
|
||||
snofsClip?: number
|
||||
consistencyModel?: number
|
||||
consistencyClip?: number
|
||||
steps?: number
|
||||
cfg?: number
|
||||
megapixels?: number
|
||||
turbo?: boolean
|
||||
}
|
||||
Reference in New Issue
Block a user