Add Krea as a second Image v2 Generate engine beside Flux.

Generate can queue krea_v2_generate.json without sharing Klein encoders or falling back if Krea assets are missing.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Towsty
2026-08-29 07:05:01 -05:00
co-authored by Cursor
parent be5211f488
commit f6663444f5
11 changed files with 567 additions and 108 deletions
+32 -19
View File
@@ -6,12 +6,12 @@ Image edit (v1) and Image v2 are siblings. v1 is unchanged. Do not mute-fix `wor
|---|---|---|
| Tab | Image edit | Image v2 |
| Route | `POST /api/edit` | `POST /api/v2/generate` and `POST /v2/generate` |
| Graphs | One file, prune unused branch | `klein_v2_edit.json`, `klein_v2_compose.json`, `klein_v2_refine.json`, or `klein_v2_generate.json` |
| Graphs | One file, prune unused branch | `klein_v2_edit.json`, `klein_v2_compose.json`, `klein_v2_refine.json`, `klein_v2_generate.json`, or `krea_v2_generate.json` |
| Second still | Optional; switches branch | Compose only; required. Mismatch is a 400 |
| Mask | — | Refine only; required. Missing mask is a 400, not an Edit fallback |
| Input still | Required | Edit / Compose / Refine required. Generate ignores stills |
| LoRAs | Optional user picker | SNOFS + Consistency, four strengths. Generate defaults Consistency off |
| Defaults | 20 steps, CFG 1 | 24 steps, CFG 4 (turbo: 8 / 1) |
| LoRAs | Optional user picker | Concept LoRA + Consistency (Flux). Generate Consistency off. Krea Generate has no Consistency |
| Defaults | 20 steps, CFG 1 | Flux: 24 steps, CFG 4 (turbo: 8 / 1). Krea Generate: 8 steps, CFG 1 |
Poll v2 jobs the same way as v1: `GET /api/generate/{id}/stream` and `GET /api/generate/{id}`.
@@ -20,7 +20,7 @@ Poll v2 jobs the same way as v1: `GET /api/generate/{id}/stream` and `GET /api/g
- **edit** — one image + text. Task is `scene`. Sending `image_b` is rejected. Use this to change lighting, background, or the whole frame from a single still.
- **compose** — two images + text. `image_b` is required. No silent one-image fallback. Use this to lock a person from A and pull a face or outfit from B.
- **refine** — one canvas + a painted mask + text. `image_a` and `mask` are required. `image_b` is ignored. Prompt only what should change inside the mask. Strength is denoise in that region (not CFG). Use this for face / hand / chest fixes without opening Comfy.
- **generate** — text only. No `image_a`, `image_b`, or mask. Size is width × height (768 / 1024 / 1280), not megapixels from a still. Use this to make a new Klein still from a prompt.
- **generate** — text only. No `image_a`, `image_b`, or mask. Size is width × height (768 / 1024 / 1280), not megapixels from a still. Engine is Flux (`klein_v2_generate.json`) or Krea (`krea_v2_generate.json`). Edit / Compose / Refine stay Flux.
Compose tasks:
@@ -35,7 +35,8 @@ Dummy tests:
- Compose + two unrelated photos + “keep everything the same” must change the output. If it matches old v1 gens, still B is not connected.
- Refine + last good gen + mask on the left breast + “red X painted on the left breast” at strength 0.4. Pass = X on the breast, face/pose/background stay. Fail = whole image regenerates, or nothing changes.
- Generate + “a red cube on a white table, studio light” and no stills. Pass = a new image of that. Fail = missing-image error, or the output equals the last Edit/Compose still.
- Generate + Flux + “a red cube on a white table, studio light” and no stills. Pass = a new image of that. Fail = missing-image error, or the output equals the last Edit/Compose still.
- Generate + Krea + the same prompt. Pass = `krea_v2_generate.json` is queued. Fail = Flux graph runs under a Krea label, or a missing Krea model falls back to Flux.
## Nodes the mapper patches
@@ -45,7 +46,7 @@ Edit and Compose:
|------|------|
| `1` | Load Image A (`inputs.image`) |
| `2` / `23` | Scale to MP, lanczos (`megapixels`) |
| `7` | SNOFS LoRA (`lora_name`, `strength_model`, `strength_clip`) |
| `7` | Concept LoRA (`lora_name`, `strength_model`, `strength_clip`) |
| `8` | Consistency LoRA (`lora_name`, `strength_model`, `strength_clip`) |
| `9` | Positive `CLIPTextEncode.text` (role header + user prompt on Compose) |
| `10` | Negative `CLIPTextEncode.text` |
@@ -75,16 +76,29 @@ Refine only (`klein_v2_refine.json`):
| `17` | `BasicScheduler.denoise` — this is Strength, not CFG |
| `19` | Sampler starts from the masked canvas latent, not an empty Flux2 latent |
Generate only (`klein_v2_generate.json`):
Generate Flux (`klein_v2_generate.json`):
| Node | Role |
|------|------|
| `14` | `EmptyFlux2LatentImage` at request width × height |
| `17` | `Flux2Scheduler` steps + the same size |
| `7` | SNOFS LoRA |
| `7` | Concept LoRA (Klein file) |
| `8` | Consistency LoRA — removed at runtime when both strengths are 0 |
| `9` / `10` | Prompt / negative from the API (no leftover widget text) |
Generate Krea (`krea_v2_generate.json`):
| Node | Role |
|------|------|
| `4` | `UNETLoader` — `krea2_turbo_mxfp8`, then `nvfp4`, then `fp8_scaled` |
| `5` | `CLIPLoader` type `krea2` — `qwen3vl_4b_fp8_scaled.safetensors` |
| `6` | `VAELoader` — `qwen_image_vae.safetensors` |
| `14` | `EmptyLatentImage` at request width × height |
| `15` | `KSampler` euler / simple, default 8 steps, CFG 1 |
| `9` / `10` | Prompt / negative from the API |
Krea does not load Klein UNET, Klein CLIP, Klein VAE, Klein Consistency, or the Klein concept LoRA. Missing Krea diffusion / text encoder / VAE fails the job. Concept LoRA on Krea is off unless `NUXT_KREA_CONCEPT_LORA` is set.
There is no `LoadImage`. If the executed Generate graph has a required LoadImage, the job fails. v1 node 145 (muted T2I with a baked prompt) is not used.
If `image_b` is sent on Edit/Compose and the executed graph has fewer than two `LoadImage` nodes, the job fails. If Refine runs without a mask input or without denoise on the scheduler, the job fails.
@@ -93,10 +107,10 @@ If `image_b` is sent on Edit/Compose and the executed graph has fewer than two `
| Slider | Default |
|--------|---------|
| SNOFS Model | 0.65 |
| SNOFS CLIP | 0.35 |
| Consistency Model | 0.70 |
| Consistency CLIP | 0.70 |
| Concept LoRA — Model | 0.65 on Flux. 0 on Krea |
| Concept LoRA — CLIP | 0.35 on Flux. 0 on Krea |
| Consistency — Model | 0.70 on Flux Edit/Compose/Refine. Off on Generate |
| Consistency — CLIP | 0.70 on Flux Edit/Compose/Refine. Off on Generate |
| Steps | 24 (turbo 8) |
| CFG | 4 (turbo 1) |
| Scale to MP | 1.0 lanczos |
@@ -108,17 +122,16 @@ If `image_b` is sent on Edit/Compose and the executed graph has fewer than two `
| Preset | Strength | SNOFS model / CLIP | Consistency | Use for |
|--------|----------|--------------------|-------------|---------|
| Face | 0.25–0.35 (chip 0.28) | 0.55 / 0.25 | 0.75 / 0.80 | Glasses, cheeks, jaw. Keep the rest of the canvas. |
| Hand / chest | 0.35–0.45 (chip 0.40) | 0.65 / 0.30 | 0.70 / 0.70 | Hands, breasts, local contact. Dummy test uses 0.40. |
| Hand / chest | 0.35–0.45 (chip 0.40) | 0.65 / 0.30 | 0.70 / 0.70 | Hands or chest, local contact. Dummy test uses 0.40. |
| Heavy | 0.55 | current sliders | current sliders | Stubborn region that barely moved at 0.40. |
Do not raise Strength to rewrite the whole frame. If the face or background moves, the mask is too big or Strength is too high.
### NSFW identity vs outfit swap
- **identity / face_lock** — keep Consistency at 0.70 / 0.70 so the face from A (identity) or B (face_lock) holds. Do not drop SNOFS CLIP below ~0.30 or the skin/body read falls apart.
- **outfit** — keep SNOFS Model ~0.65 so cloth reads; do not raise it past ~0.85 or it starts rewriting the body from A. Consistency stays at 0.70 so the person in A does not become the person in B.
- **identity / face_lock** — keep Consistency at 0.70 / 0.70 so the face from A (identity) or B (face_lock) holds. Do not drop Concept LoRA CLIP below ~0.30 or the skin/body read falls apart.
- **outfit** — keep Concept LoRA Model ~0.65 so cloth reads; do not raise it past ~0.85 or it starts rewriting the body from A. Consistency stays at 0.70 so the person in A does not become the person in B.
- **scene / edit** — start at the table defaults. Turbo is for drafts only.
- **refine** — prompt only the masked change. Face vs hand/chest chips set Strength and LoRAs together.
- **generate** — SFW scene = SNOFS 0 / 0. SNOF T2I = SNOFS 0.65 / 0.35. Consistency stays 0. Turbo is 8 steps / CFG 1.
- **generate / Flux** — Concept LoRA defaults 0.65 / 0.35. Consistency stays 0. Turbo is 8 steps / CFG 1.
- **generate / Krea** — 8 steps, CFG 1, no Consistency. Concept LoRA stays 0 unless a Krea concept file is configured.
Presets on this tab store sliders + mode + task (+ Strength when Refine, + size when Generate). They do not load v1 LoRA stacks or a cached latent.
Presets on this tab store sliders + mode + task + engine (+ Strength when Refine, + size when Generate). They do not load v1 LoRA stacks or a cached latent. Missing engine on an old preset is Flux.
+3 -1
View File
@@ -53,12 +53,14 @@ export default defineNuxtConfig({
authPassword: process.env.AUTH_PASSWORD || '',
purgeComfyOutputs: process.env.PURGE_COMFY_OUTPUTS !== 'false',
comfyFilenamePrefix: process.env.COMFY_FILENAME_PREFIX || 'video/MiniMax_H3',
kreaConceptLora: process.env.NUXT_KREA_CONCEPT_LORA || '',
public: {
authEnabled: process.env.NUXT_PUBLIC_AUTH_ENABLED === 'true',
authMode: process.env.NUXT_PUBLIC_AUTH_MODE || process.env.AUTH_MODE || '',
instanceName: process.env.NUXT_PUBLIC_INSTANCE_NAME || 'AIGen',
// NUXT_PUBLIC_STUDIO=xaigen on the xAIGen Coolify app (LTX Pink Cherry + xAIGen-prefixed LoRAs)
studio: process.env.NUXT_PUBLIC_STUDIO || ''
studio: process.env.NUXT_PUBLIC_STUDIO || '',
kreaConceptLora: process.env.NUXT_PUBLIC_KREA_CONCEPT_LORA || process.env.NUXT_KREA_CONCEPT_LORA || ''
}
},
nitro: {
+97 -48
View File
@@ -66,7 +66,7 @@
<h2 class="font-display text-2xl font-bold">Input</h2>
<p class="text-sm text-zinc-400">{{ studioMode === 'editv2'
? (v2Mode === 'generate'
? 'Generate: no reference image. Prompt only. Klein 9B Base text-to-image.'
? 'Generate: no reference image. Prompt only.'
: v2Mode === 'refine'
? 'Refine: paint a mask on Still A. Prompt only what should change in the painted area. Strength is denoise in that region.'
: v2Mode === 'compose'
@@ -710,10 +710,33 @@
/>
</div>
<div v-else-if="studioMode === 'editv2'" class="space-y-3">
<p class="text-xs text-zinc-500">Klein v2 · 9B Base · euler. Edit, Compose, Refine, and Generate are separate graphs. Shares the desktop GPU with video.</p>
<div v-if="v2Mode === 'generate'" class="space-y-2 rounded-2xl border border-white/10 bg-zinc-950/40 p-3">
<p class="text-xs text-zinc-500">Edit, Compose, Refine, and Generate are separate graphs. Shares the desktop GPU with video.</p>
<div v-if="v2Mode === 'generate'" class="space-y-3 rounded-2xl border border-white/10 bg-zinc-950/40 p-3">
<div>
<p class="text-xs text-zinc-400">Engine</p>
<div class="mt-2 flex flex-wrap gap-2">
<button
type="button"
class="rounded-full border px-2.5 py-1 text-xs"
:class="v2Engine === 'flux' ? 'border-amber-300/70 text-amber-100' : 'border-white/10 text-zinc-400'"
@click="setV2Engine('flux')"
>
Flux
</button>
<button
type="button"
class="rounded-full border px-2.5 py-1 text-xs"
:class="v2Engine === 'krea' ? 'border-amber-300/70 text-amber-100' : 'border-white/10 text-zinc-400'"
@click="setV2Engine('krea')"
>
Krea
</button>
</div>
<p class="mt-2 text-xs text-zinc-500">Flux is the current edit and compose engine. Krea is a separate photograph engine.</p>
</div>
<div>
<p class="text-xs text-zinc-400">Size · near 1 MP</p>
<div class="flex flex-wrap gap-2">
<div class="mt-2 flex flex-wrap gap-2">
<button
v-for="(pair, aspect) in IMAGE_V2_GENERATE_ASPECTS"
:key="aspect"
@@ -725,23 +748,6 @@
{{ aspect }} · {{ pair[0] }}×{{ pair[1] }}
</button>
</div>
<div class="flex flex-wrap gap-2">
<button
type="button"
class="rounded-full border px-2.5 py-1 text-xs"
:class="v2SnofsModel === 0 && v2SnofsClip === 0 ? 'border-amber-300/70 text-amber-100' : 'border-white/10 text-zinc-400'"
@click="applyGenerateLook('sfw')"
>
SFW scene
</button>
<button
type="button"
class="rounded-full border px-2.5 py-1 text-xs"
:class="v2SnofsModel === IMAGE_V2_GENERATE_SNOFS.snofsModel && v2SnofsClip === IMAGE_V2_GENERATE_SNOFS.snofsClip ? 'border-amber-300/70 text-amber-100' : 'border-white/10 text-zinc-400'"
@click="applyGenerateLook('snofs')"
>
SNOF T2I
</button>
</div>
</div>
<div v-if="v2Mode === 'refine'" class="space-y-2 rounded-2xl border border-white/10 bg-zinc-950/40 p-3">
@@ -788,19 +794,36 @@
</div>
<div class="grid grid-cols-2 gap-3">
<label class="block text-sm">
<span class="mb-1 block text-zinc-400">SNOFS Model</span>
<input v-model.number="v2SnofsModel" type="number" min="0" max="2" step="0.05" class="w-full rounded-xl border border-white/10 bg-zinc-950 px-3 py-2 text-sm outline-none ring-amber-300/40 focus:ring-2">
<span class="mb-1 block text-zinc-400">Concept LoRA — Model</span>
<input
v-model.number="v2SnofsModel"
type="number"
min="0"
max="2"
step="0.05"
:disabled="v2Engine === 'krea' && !kreaConceptConfigured"
class="w-full rounded-xl border border-white/10 bg-zinc-950 px-3 py-2 text-sm outline-none ring-amber-300/40 focus:ring-2 disabled:opacity-50"
>
</label>
<label class="block text-sm">
<span class="mb-1 block text-zinc-400">SNOFS CLIP</span>
<input v-model.number="v2SnofsClip" type="number" min="0" max="2" step="0.05" class="w-full rounded-xl border border-white/10 bg-zinc-950 px-3 py-2 text-sm outline-none ring-amber-300/40 focus:ring-2">
<span class="mb-1 block text-zinc-400">Concept LoRA — CLIP</span>
<input
v-model.number="v2SnofsClip"
type="number"
min="0"
max="2"
step="0.05"
:disabled="v2Engine === 'krea' && !kreaConceptConfigured"
class="w-full rounded-xl border border-white/10 bg-zinc-950 px-3 py-2 text-sm outline-none ring-amber-300/40 focus:ring-2 disabled:opacity-50"
>
</label>
<label v-if="v2Mode !== 'generate'" class="block text-sm">
<span class="mb-1 block text-zinc-400">Consistency Model</span>
<p v-if="v2Engine === 'krea' && !kreaConceptConfigured" class="col-span-2 text-xs text-zinc-500">No Krea concept LoRA is configured.</p>
<label v-if="v2Mode !== 'generate' && v2Engine !== 'krea'" class="block text-sm">
<span class="mb-1 block text-zinc-400">Consistency — Model</span>
<input v-model.number="v2ConsistencyModel" type="number" min="0" max="2" step="0.05" class="w-full rounded-xl border border-white/10 bg-zinc-950 px-3 py-2 text-sm outline-none ring-amber-300/40 focus:ring-2">
</label>
<label v-if="v2Mode !== 'generate'" class="block text-sm">
<span class="mb-1 block text-zinc-400">Consistency CLIP</span>
<label v-if="v2Mode !== 'generate' && v2Engine !== 'krea'" class="block text-sm">
<span class="mb-1 block text-zinc-400">Consistency — CLIP</span>
<input v-model.number="v2ConsistencyClip" type="number" min="0" max="2" step="0.05" class="w-full rounded-xl border border-white/10 bg-zinc-950 px-3 py-2 text-sm outline-none ring-amber-300/40 focus:ring-2">
</label>
<label class="block text-sm">
@@ -820,7 +843,7 @@
<input v-model.number="v2Megapixels" type="number" :min="IMAGE_SCALE_MP_MIN" :max="IMAGE_SCALE_MP_MAX" :step="IMAGE_SCALE_MP_STEP" class="w-full rounded-xl border border-white/10 bg-zinc-950 px-3 py-2 text-sm outline-none ring-amber-300/40 focus:ring-2">
</label>
</div>
<label class="flex cursor-pointer items-center gap-2 text-xs text-zinc-400">
<label v-if="v2Engine !== 'krea'" class="flex cursor-pointer items-center gap-2 text-xs text-zinc-400">
<span class="relative inline-flex h-5 w-9 shrink-0 items-center">
<input v-model="v2Turbo" type="checkbox" class="peer sr-only" role="switch">
<span class="absolute inset-0 rounded-full bg-zinc-700 transition peer-checked:bg-amber-400" />
@@ -1959,9 +1982,10 @@ import {
IMAGE_V2_DENOISE_STEP,
IMAGE_V2_GENERATE_ASPECTS,
IMAGE_V2_GENERATE_HEIGHT,
IMAGE_V2_GENERATE_SNOFS,
IMAGE_V2_GENERATE_SFW,
IMAGE_V2_GENERATE_WIDTH,
IMAGE_V2_KREA_CFG,
IMAGE_V2_KREA_CONCEPT_LORA,
IMAGE_V2_KREA_STEPS,
IMAGE_V2_REFINE_FACE,
IMAGE_V2_REFINE_FACE_PROMPT,
IMAGE_V2_REFINE_HAND,
@@ -1972,6 +1996,8 @@ import {
clampImageV2Denoise,
clampImageV2Size,
clampImageV2Strength,
parseImageV2Engine,
type ImageV2Engine,
type ImageV2GenerateAspect,
type ImageV2Mode,
type ImageV2PresetSettings,
@@ -2245,7 +2271,9 @@ const editScaleMegapixels = ref(1)
const editRefFile = ref<File | null>(null)
const editRefPreview = ref('')
const v2Mode = ref<ImageV2Mode>('edit')
const v2Engine = ref<ImageV2Engine>('flux')
const v2Task = ref<ImageV2Task>('scene')
const kreaConceptConfigured = computed(() => Boolean(String(runtimeConfig.public.kreaConceptLora || IMAGE_V2_KREA_CONCEPT_LORA || '').trim()))
const v2Negative = ref('')
const v2SnofsModel = ref(IMAGE_V2_SNOFS_MODEL)
const v2SnofsClip = ref(IMAGE_V2_SNOFS_CLIP)
@@ -3469,6 +3497,7 @@ function currentPresetSnapshot() {
loraStack: [],
settings: {
mode: v2Mode.value,
engine: v2Mode.value === 'generate' ? v2Engine.value : 'flux',
task: v2Mode.value === 'compose' ? v2Task.value : v2Mode.value === 'refine' ? 'refine' : v2Mode.value === 'generate' ? 't2i' : 'scene',
negative: v2Negative.value,
snofsModel: clampImageV2Strength(v2SnofsModel.value, IMAGE_V2_SNOFS_MODEL),
@@ -3521,6 +3550,7 @@ function applyGenerationPreset(preset: GenerationPreset) {
if (preset.kind === 'imagev2') {
const settings = preset.settings as ImageV2PresetSettings
v2Mode.value = settings.mode === 'compose' ? 'compose' : settings.mode === 'refine' ? 'refine' : settings.mode === 'generate' ? 'generate' : 'edit'
v2Engine.value = v2Mode.value === 'generate' ? parseImageV2Engine(settings.engine, 'flux') : 'flux'
v2Task.value = v2Mode.value === 'compose' ? (settings.task || 'scene') : v2Mode.value === 'refine' ? 'refine' : v2Mode.value === 'generate' ? 't2i' : 'scene'
if (typeof settings.negative === 'string') v2Negative.value = settings.negative
v2SnofsModel.value = clampImageV2Strength(settings.snofsModel, IMAGE_V2_SNOFS_MODEL)
@@ -4803,6 +4833,7 @@ function applyStillSettings(still: LibraryStill) {
? settings.mode
: 'edit'
setV2Mode(mode)
v2Engine.value = mode === 'generate' ? parseImageV2Engine(settings.engine, 'flux') : 'flux'
if (!settings) return
if (typeof settings.negative === 'string') v2Negative.value = settings.negative
if (settings.snofsModel != null) v2SnofsModel.value = clampImageV2Strength(settings.snofsModel, IMAGE_V2_SNOFS_MODEL)
@@ -5424,11 +5455,36 @@ function setV2Mode(mode: ImageV2Mode) {
v2Task.value = 't2i'
v2ConsistencyModel.value = 0
v2ConsistencyClip.value = 0
} else if (previous === 'generate' && v2ConsistencyModel.value === 0 && v2ConsistencyClip.value === 0) {
} else {
v2Engine.value = 'flux'
if (previous === 'generate' && v2ConsistencyModel.value === 0 && v2ConsistencyClip.value === 0) {
v2ConsistencyModel.value = IMAGE_V2_CONSISTENCY_MODEL
v2ConsistencyClip.value = IMAGE_V2_CONSISTENCY_CLIP
}
}
}
function setV2Engine(engine: ImageV2Engine) {
if (engine === 'krea' && v2Mode.value !== 'generate') return
const previous = v2Engine.value
v2Engine.value = engine
if (engine === 'krea') {
v2Steps.value = IMAGE_V2_KREA_STEPS
v2Cfg.value = IMAGE_V2_KREA_CFG
v2SnofsModel.value = 0
v2SnofsClip.value = 0
v2ConsistencyModel.value = 0
v2ConsistencyClip.value = 0
v2Turbo.value = false
return
}
if (previous === 'krea') {
v2Steps.value = IMAGE_V2_STEPS_DEFAULT
v2Cfg.value = IMAGE_V2_CFG_DEFAULT
v2SnofsModel.value = IMAGE_V2_SNOFS_MODEL
v2SnofsClip.value = IMAGE_V2_SNOFS_CLIP
}
}
function setGenerateSize(aspect: ImageV2GenerateAspect) {
const [width, height] = IMAGE_V2_GENERATE_ASPECTS[aspect]
@@ -5436,14 +5492,6 @@ function setGenerateSize(aspect: ImageV2GenerateAspect) {
v2Height.value = height
}
function applyGenerateLook(kind: 'sfw' | 'snofs') {
const look = kind === 'sfw' ? IMAGE_V2_GENERATE_SFW : IMAGE_V2_GENERATE_SNOFS
v2SnofsModel.value = look.snofsModel
v2SnofsClip.value = look.snofsClip
v2ConsistencyModel.value = 0
v2ConsistencyClip.value = 0
}
function applyRefinePreset(kind: 'face' | 'hand' | 'heavy') {
if (kind === 'face') {
v2Denoise.value = IMAGE_V2_REFINE_FACE.strength
@@ -5537,15 +5585,16 @@ async function editImageV2() {
const size = clampImageV2Size(v2Width.value, v2Height.value)
body.append('width', String(size.width))
body.append('height', String(size.height))
body.append('engine', v2Engine.value)
}
body.append('prompt', prompt.value.trim())
body.append('negative', v2Negative.value)
body.append('snofs_model', String(clampImageV2Strength(v2SnofsModel.value, IMAGE_V2_SNOFS_MODEL)))
body.append('snofs_clip', String(clampImageV2Strength(v2SnofsClip.value, IMAGE_V2_SNOFS_CLIP)))
body.append('consistency_model', String(v2Mode.value === 'generate' ? 0 : clampImageV2Strength(v2ConsistencyModel.value, IMAGE_V2_CONSISTENCY_MODEL)))
body.append('consistency_clip', String(v2Mode.value === 'generate' ? 0 : clampImageV2Strength(v2ConsistencyClip.value, IMAGE_V2_CONSISTENCY_CLIP)))
body.append('steps', String(clampImageSteps(v2Steps.value, IMAGE_V2_STEPS_DEFAULT)))
body.append('cfg', String(clampImageCfg(v2Cfg.value, IMAGE_V2_CFG_DEFAULT)))
body.append('snofs_model', String(clampImageV2Strength(v2SnofsModel.value, v2Engine.value === 'krea' ? 0 : IMAGE_V2_SNOFS_MODEL)))
body.append('snofs_clip', String(clampImageV2Strength(v2SnofsClip.value, v2Engine.value === 'krea' ? 0 : IMAGE_V2_SNOFS_CLIP)))
body.append('consistency_model', String(v2Mode.value === 'generate' || v2Engine.value === 'krea' ? 0 : clampImageV2Strength(v2ConsistencyModel.value, IMAGE_V2_CONSISTENCY_MODEL)))
body.append('consistency_clip', String(v2Mode.value === 'generate' || v2Engine.value === 'krea' ? 0 : clampImageV2Strength(v2ConsistencyClip.value, IMAGE_V2_CONSISTENCY_CLIP)))
body.append('steps', String(clampImageSteps(v2Steps.value, v2Engine.value === 'krea' ? IMAGE_V2_KREA_STEPS : IMAGE_V2_STEPS_DEFAULT)))
body.append('cfg', String(clampImageCfg(v2Cfg.value, v2Engine.value === 'krea' ? IMAGE_V2_KREA_CFG : IMAGE_V2_CFG_DEFAULT)))
if (v2Mode.value !== 'generate') body.append('megapixels', String(clampImageScaleMegapixels(v2Megapixels.value, 1)))
body.append('turbo', String(v2Turbo.value === true))
body.append('seed', seedInput.value || 'random')
@@ -5566,7 +5615,7 @@ async function editImageV2() {
body
})
if (started.queued) {
toast('Queued Klein v2 behind the current job.')
toast(v2Engine.value === 'krea' ? 'Queued Krea generate behind the current job.' : 'Queued Image v2 behind the current job.')
await refreshStudioQueue()
return
}
+33 -9
View File
@@ -11,10 +11,14 @@ import {
IMAGE_V2_STEPS_DEFAULT,
IMAGE_V2_TURBO_CFG,
IMAGE_V2_TURBO_STEPS,
IMAGE_V2_KREA_CFG,
IMAGE_V2_KREA_STEPS,
IMAGE_V2_DENOISE_DEFAULT,
clampImageV2Denoise,
clampImageV2Size,
clampImageV2Strength,
imageV2WorkflowFile,
parseImageV2Engine,
parseImageV2Mode,
parseImageV2Task,
type ImageV2Mode,
@@ -174,9 +178,21 @@ export default defineEventHandler(async (event) => {
const folderLocked = Boolean(destFolder?.protected && !destFolder.unlocked)
const hideInput = parseBool(fields.hideInput)
const hideThumbnail = parseBool(fields.hideThumbnail)
const requestedEngine = parseImageV2Engine(fields.engine, 'flux')
if (requestedEngine === 'krea' && mode !== 'generate') {
throw createError({
statusCode: 400,
statusMessage: 'Krea is available on Generate first.'
})
}
const engine = mode === 'generate' ? requestedEngine : 'flux'
const turbo = parseBool(fields.turbo)
const steps = turbo ? IMAGE_V2_TURBO_STEPS : clampImageSteps(fields.steps, IMAGE_V2_STEPS_DEFAULT)
const cfg = turbo ? IMAGE_V2_TURBO_CFG : clampImageCfg(fields.cfg, IMAGE_V2_CFG_DEFAULT)
const steps = engine === 'krea'
? clampImageSteps(fields.steps, IMAGE_V2_KREA_STEPS)
: turbo ? IMAGE_V2_TURBO_STEPS : clampImageSteps(fields.steps, IMAGE_V2_STEPS_DEFAULT)
const cfg = engine === 'krea'
? clampImageCfg(fields.cfg, IMAGE_V2_KREA_CFG)
: turbo ? IMAGE_V2_TURBO_CFG : clampImageCfg(fields.cfg, IMAGE_V2_CFG_DEFAULT)
const megapixels = mode === 'generate' ? 0 : clampImageScaleMegapixels(fields.megapixels ?? fields.scaleMegapixels, 1)
const seed = fields.seed && String(fields.seed) !== 'random'
? Number(fields.seed)
@@ -191,8 +207,16 @@ export default defineEventHandler(async (event) => {
mode === 'generate' ? 't2i' : mode === 'refine' ? 'refine' : mode === 'edit' ? 'scene' : task
) as ImageV2Task
const refineStrength = mode === 'refine' ? clampImageV2Denoise(fields.strength, IMAGE_V2_DENOISE_DEFAULT) : undefined
const generateConsistencyModel = clampImageV2Strength(fields.consistency_model, mode === 'generate' ? 0 : IMAGE_V2_CONSISTENCY_MODEL)
const generateConsistencyClip = clampImageV2Strength(fields.consistency_clip, mode === 'generate' ? 0 : IMAGE_V2_CONSISTENCY_CLIP)
const generateConsistencyModel = clampImageV2Strength(
fields.consistency_model,
mode === 'generate' || engine === 'krea' ? 0 : IMAGE_V2_CONSISTENCY_MODEL
)
const generateConsistencyClip = clampImageV2Strength(
fields.consistency_clip,
mode === 'generate' || engine === 'krea' ? 0 : IMAGE_V2_CONSISTENCY_CLIP
)
const generateSnofsFallback = engine === 'krea' ? 0 : IMAGE_V2_SNOFS_MODEL
const generateSnofsClipFallback = engine === 'krea' ? 0 : IMAGE_V2_SNOFS_CLIP
const still = imageA
? await rememberInputStill({
@@ -262,8 +286,9 @@ export default defineEventHandler(async (event) => {
imagePipeline: 'v2',
v2Mode,
v2Task,
snofsModel: clampImageV2Strength(fields.snofs_model, IMAGE_V2_SNOFS_MODEL),
snofsClip: clampImageV2Strength(fields.snofs_clip, IMAGE_V2_SNOFS_CLIP),
engine,
snofsModel: clampImageV2Strength(fields.snofs_model, generateSnofsFallback),
snofsClip: clampImageV2Strength(fields.snofs_clip, generateSnofsClipFallback),
consistencyModel: generateConsistencyModel,
consistencyClip: generateConsistencyClip,
maskStillId: savedMask?.id,
@@ -288,9 +313,8 @@ export default defineEventHandler(async (event) => {
cfg,
mode: v2Mode,
task: v2Task,
workflow: v2Mode === 'generate'
? 'klein_v2_generate.json'
: v2Mode === 'refine' ? 'klein_v2_refine.json' : v2Mode === 'compose' ? 'klein_v2_compose.json' : 'klein_v2_edit.json',
engine,
workflow: imageV2WorkflowFile(v2Mode, engine),
width: size?.width || 0,
height: size?.height || 0,
strength: refineStrength,
+81
View File
@@ -0,0 +1,81 @@
{
"4": {
"inputs": {
"unet_name": "krea2_turbo_mxfp8.safetensors",
"weight_dtype": "default"
},
"class_type": "UNETLoader",
"_meta": { "title": "Load Krea 2 Turbo" }
},
"5": {
"inputs": {
"clip_name": "qwen3vl_4b_fp8_scaled.safetensors",
"type": "krea2",
"device": "default"
},
"class_type": "CLIPLoader",
"_meta": { "title": "Load Krea 2 CLIP" }
},
"6": {
"inputs": { "vae_name": "qwen_image_vae.safetensors" },
"class_type": "VAELoader",
"_meta": { "title": "Load Qwen Image VAE" }
},
"9": {
"inputs": {
"text": "",
"clip": ["5", 0]
},
"class_type": "CLIPTextEncode",
"_meta": { "title": "Positive Prompt" }
},
"10": {
"inputs": {
"text": "",
"clip": ["5", 0]
},
"class_type": "CLIPTextEncode",
"_meta": { "title": "Negative Prompt" }
},
"14": {
"inputs": {
"width": 1024,
"height": 1024,
"batch_size": 1
},
"class_type": "EmptyLatentImage",
"_meta": { "title": "Empty Latent" }
},
"15": {
"inputs": {
"seed": 1,
"steps": 8,
"cfg": 1,
"sampler_name": "euler",
"scheduler": "simple",
"denoise": 1,
"model": ["4", 0],
"positive": ["9", 0],
"negative": ["10", 0],
"latent_image": ["14", 0]
},
"class_type": "KSampler",
"_meta": { "title": "KSampler" }
},
"20": {
"inputs": {
"samples": ["15", 0],
"vae": ["6", 0]
},
"class_type": "VAEDecode",
"_meta": { "title": "VAE Decode" }
},
"21": {
"inputs": {
"filename_prefix": "aigen-v2-krea",
"images": ["20", 0]
},
"class_type": "SaveImage",
"_meta": { "title": "Save Image" }
}
}
+7 -2
View File
@@ -21,10 +21,13 @@ import {
IMAGE_V2_SNOFS_CLIP,
IMAGE_V2_SNOFS_MODEL,
IMAGE_V2_STEPS_DEFAULT,
IMAGE_V2_KREA_CFG,
IMAGE_V2_KREA_STEPS,
IMAGE_V2_DENOISE_DEFAULT,
clampImageV2Denoise,
clampImageV2Size,
clampImageV2Strength,
parseImageV2Engine,
parseImageV2Mode,
parseImageV2Task,
type ImageV2PresetSettings
@@ -115,16 +118,18 @@ function sanitizeImageV2Settings(raw: unknown): ImageV2PresetSettings {
const rec = raw && typeof raw === 'object' ? raw as Record<string, unknown> : {}
const mode = parseImageV2Mode(rec.mode) || 'edit'
const turbo = rec.turbo === true
const engine = mode === 'generate' ? parseImageV2Engine(rec.engine, 'flux') : 'flux'
return {
mode,
engine,
task: mode === 'compose' ? parseImageV2Task(rec.task, 'scene') : mode === 'refine' ? 'refine' : mode === 'generate' ? 't2i' : 'scene',
negative: String(rec.negative || '').slice(0, 2000),
snofsModel: clampImageV2Strength(rec.snofsModel ?? rec.snofs_model, IMAGE_V2_SNOFS_MODEL),
snofsClip: clampImageV2Strength(rec.snofsClip ?? rec.snofs_clip, IMAGE_V2_SNOFS_CLIP),
consistencyModel: clampImageV2Strength(rec.consistencyModel ?? rec.consistency_model, mode === 'generate' ? 0 : IMAGE_V2_CONSISTENCY_MODEL),
consistencyClip: clampImageV2Strength(rec.consistencyClip ?? rec.consistency_clip, mode === 'generate' ? 0 : IMAGE_V2_CONSISTENCY_CLIP),
steps: turbo ? 8 : clampImageSteps(rec.steps, IMAGE_V2_STEPS_DEFAULT),
cfg: turbo ? 1 : clampImageCfg(rec.cfg, IMAGE_V2_CFG_DEFAULT),
steps: engine === 'krea' ? clampImageSteps(rec.steps, IMAGE_V2_KREA_STEPS) : turbo ? 8 : clampImageSteps(rec.steps, IMAGE_V2_STEPS_DEFAULT),
cfg: engine === 'krea' ? clampImageCfg(rec.cfg, IMAGE_V2_KREA_CFG) : turbo ? 1 : clampImageCfg(rec.cfg, IMAGE_V2_CFG_DEFAULT),
megapixels: mode === 'generate' ? undefined : clampImageScaleMegapixels(rec.megapixels ?? rec.scaleMegapixels, 1),
turbo,
strength: mode === 'refine' ? clampImageV2Denoise(rec.strength, IMAGE_V2_DENOISE_DEFAULT) : undefined,
+18 -4
View File
@@ -2,16 +2,17 @@ import { createJob, emitJob, type Job } from '~/server/utils/jobs'
import { ensureComfyReady } from '~/server/utils/comfyLifecycle'
import { assertImageScaleToTotalPixelsNode, getComfyHost, uploadImage, queuePrompt, purgeComfyArtifacts } from '~/server/utils/comfy'
import { withImageComfyHost, waitForImageEdit, downloadEditedImage } from '~/server/utils/imageComfy'
import { buildImageV2Workflow } from '~/server/utils/imageWorkflowV2'
import { buildImageV2Workflow, resolveKreaGenerateAssets } from '~/server/utils/imageWorkflowV2'
import { ensureComfyLoraNames } from '~/server/utils/loras'
import { imageDimensions } from '~/server/utils/resolution'
import { emitChainJob } from '~/server/utils/watch'
import { saveStill } from '~/server/utils/library'
import type { ImageV2Mode, ImageV2Task } from '~/utils/imageV2'
import { parseImageV2Engine, type ImageV2Engine, type ImageV2Mode, type ImageV2Task } from '~/utils/imageV2'
import type { EditImageFile } from '~/server/utils/imageChain'
export type EditV2RunParams = {
mode: ImageV2Mode
engine?: ImageV2Engine
task: ImageV2Task
image?: EditImageFile | null
reference: EditImageFile | null
@@ -64,10 +65,16 @@ export async function runEditV2(job: Job, params: EditV2RunParams) {
await withImageComfyHost(job.imageComfyHost, async () => {
const generate = params.mode === 'generate'
const engine = generate ? parseImageV2Engine(params.engine, 'flux') : 'flux'
if (engine === 'krea' && params.mode !== 'generate') {
throw new Error('Krea is available on Generate first.')
}
emitChainJob(job, {
type: 'status',
message: generate
? 'Queueing Klein v2 generate on Beast...'
? engine === 'krea'
? 'Queueing Krea generate on Beast...'
: 'Queueing Flux generate on Beast...'
: params.mode === 'refine'
? 'Uploading canvas and mask to Beast...'
: params.mode === 'compose' ? 'Uploading stills A and B to Beast...' : 'Uploading still A to Beast...',
@@ -103,8 +110,10 @@ export async function runEditV2(job: Job, params: EditV2RunParams) {
}
await ensureComfyLoraNames('image')
if (!generate) await assertImageScaleToTotalPixelsNode()
const krea = engine === 'krea' ? await resolveKreaGenerateAssets() : null
const built = buildImageV2Workflow({
mode: params.mode,
engine,
task: params.task,
prompt: params.prompt,
negative: params.negative,
@@ -123,7 +132,11 @@ export async function runEditV2(job: Job, params: EditV2RunParams) {
width: params.width,
height: params.height,
turbo: params.turbo === true,
filenamePrefix: `aigen_v2_${job.id.slice(0, 8)}`
filenamePrefix: `aigen_v2_${job.id.slice(0, 8)}`,
kreaUnetName: krea?.unet,
kreaClipName: krea?.clip,
kreaVaeName: krea?.vae,
kreaConceptLora: krea?.conceptLora
})
const queued = await queuePrompt(built.graph, job.clientId)
job.promptId = queued.prompt_id
@@ -169,6 +182,7 @@ export async function runEditV2(job: Job, params: EditV2RunParams) {
settings: {
kind: 'imagev2',
mode: params.mode,
engine,
task: params.task,
negative: params.negative,
snofsModel: params.snofsModel,
+243 -9
View File
@@ -2,7 +2,9 @@ import editTemplate from '../assets/klein_v2_edit.json'
import composeTemplate from '../assets/klein_v2_compose.json'
import refineTemplate from '../assets/klein_v2_refine.json'
import generateTemplate from '../assets/klein_v2_generate.json'
import kreaGenerateTemplate from '../assets/krea_v2_generate.json'
import { IMAGE_SCALE_TO_TOTAL_PIXELS } from '~/server/utils/comfy'
import { imageComfyFetch } from '~/server/utils/imageComfy'
import { cachedComfyLoraNames } from '~/server/utils/loras'
import { resolveComfyLoraName, loraIdentityKey } from '~/utils/loras'
import { clampImageCfg, clampImageScaleMegapixels, clampImageSteps } from '~/utils/generationPresets'
@@ -10,6 +12,13 @@ import {
IMAGE_V2_CONSISTENCY_CLIP,
IMAGE_V2_CONSISTENCY_LORA,
IMAGE_V2_CONSISTENCY_MODEL,
IMAGE_V2_KREA_CFG,
IMAGE_V2_KREA_CLIP,
IMAGE_V2_KREA_CLIP_TYPE,
IMAGE_V2_KREA_CONCEPT_LORA,
IMAGE_V2_KREA_STEPS,
IMAGE_V2_KREA_UNET_CANDIDATES,
IMAGE_V2_KREA_VAE,
IMAGE_V2_SNOFS_CLIP,
IMAGE_V2_SNOFS_LORA,
IMAGE_V2_SNOFS_MODEL,
@@ -20,6 +29,9 @@ import {
clampImageV2Size,
clampImageV2Strength,
composeImageV2Prompt,
imageV2WorkflowFile,
parseImageV2Engine,
type ImageV2Engine,
type ImageV2Mode,
type ImageV2Task
} from '~/utils/imageV2'
@@ -46,10 +58,12 @@ export const IMAGE_V2_EDIT_WORKFLOW = 'klein_v2_edit.json'
export const IMAGE_V2_COMPOSE_WORKFLOW = 'klein_v2_compose.json'
export const IMAGE_V2_REFINE_WORKFLOW = 'klein_v2_refine.json'
export const IMAGE_V2_GENERATE_WORKFLOW = 'klein_v2_generate.json'
export const IMAGE_V2_KREA_GENERATE_WORKFLOW = 'krea_v2_generate.json'
export interface ImageV2BuildParams {
mode: ImageV2Mode
task: ImageV2Task
engine?: ImageV2Engine
prompt: string
negative?: string
imageAName?: string
@@ -68,6 +82,10 @@ export interface ImageV2BuildParams {
seed: number
megapixels?: number
filenamePrefix?: string
kreaUnetName?: string
kreaClipName?: string
kreaVaeName?: string
kreaConceptLora?: string
}
function setInput(graph: WorkflowGraph, id: string, key: string, value: unknown) {
@@ -120,9 +138,140 @@ function bypassLoraNode(graph: WorkflowGraph, id: string, modelFrom: string, cli
}
}
export function assertImageV2Graph(graph: WorkflowGraph, mode: ImageV2Mode, imageBName?: string) {
function modelBasename(name: string) {
return String(name || '').replace(/\\/g, '/').split('/').pop() || ''
}
function modelIdentity(name: string) {
return modelBasename(name).toLowerCase()
}
async function listComfyModels(folder: string) {
const res = await imageComfyFetch(`/models/${encodeURIComponent(folder)}`, {
signal: AbortSignal.timeout(8000)
}).catch(() => null)
if (!res || !res.ok) return [] as string[]
const payload = await res.json().catch(() => null)
if (Array.isArray(payload)) return payload.map(item => String(item || '')).filter(Boolean)
return [] as string[]
}
function pickComfyModel(available: string[], candidates: string[]) {
for (const wanted of candidates) {
const id = modelIdentity(wanted)
const exact = available.find(name => modelIdentity(name) === id)
if (exact) return exact
const stem = id.replace(/\.(safetensors|sft|ckpt|pt|bin)$/i, '')
const prefix = available.find(name => modelIdentity(name).startsWith(stem))
if (prefix) return prefix
}
return ''
}
export function configuredKreaConceptLora() {
const config = useRuntimeConfig()
return String(
config.kreaConceptLora
|| (config.public as { kreaConceptLora?: string } | undefined)?.kreaConceptLora
|| IMAGE_V2_KREA_CONCEPT_LORA
|| ''
).trim()
}
export async function resolveKreaGenerateAssets() {
const [unets, clips, vaes, clipInfo] = await Promise.all([
listComfyModels('diffusion_models'),
listComfyModels('text_encoders').then(async (names) => names.length ? names : listComfyModels('clip')),
listComfyModels('vae'),
imageComfyFetch('/object_info/CLIPLoader', { signal: AbortSignal.timeout(8000) })
.then(async (res) => res.ok ? res.json() : null)
.catch(() => null)
])
const unet = pickComfyModel(unets, [...IMAGE_V2_KREA_UNET_CANDIDATES])
if (!unet) {
throw createError({
statusCode: 503,
statusMessage: 'Krea diffusion model is missing on Beast Comfy (need krea2_turbo_mxfp8, nvfp4, or fp8_scaled).'
})
}
const clip = pickComfyModel(clips, [IMAGE_V2_KREA_CLIP])
if (!clip) {
throw createError({
statusCode: 503,
statusMessage: `Krea text encoder is missing on Beast Comfy (${IMAGE_V2_KREA_CLIP}).`
})
}
const vae = pickComfyModel(vaes, [IMAGE_V2_KREA_VAE])
if (!vae) {
throw createError({
statusCode: 503,
statusMessage: `Qwen image VAE is missing on Beast Comfy (${IMAGE_V2_KREA_VAE}).`
})
}
const rec = clipInfo && typeof clipInfo === 'object' ? clipInfo as Record<string, unknown> : {}
const body = (rec.CLIPLoader && typeof rec.CLIPLoader === 'object' ? rec.CLIPLoader : rec) as {
input?: { required?: { type?: unknown } }
}
const typeSpec = body.input?.required?.type
const types = Array.isArray(typeSpec) && Array.isArray(typeSpec[0])
? (typeSpec[0] as unknown[]).map(item => String(item || ''))
: []
if (types.length && !types.includes(IMAGE_V2_KREA_CLIP_TYPE)) {
throw createError({
statusCode: 503,
statusMessage: 'Beast Comfy CLIPLoader has no krea2 type. Update Comfy before using Krea.'
})
}
return { unet, clip, vae, conceptLora: configuredKreaConceptLora() }
}
function graphFileNames(graph: WorkflowGraph) {
const names: string[] = []
for (const node of Object.values(graph)) {
for (const key of ['unet_name', 'clip_name', 'vae_name', 'lora_name']) {
const value = node.inputs[key]
if (typeof value === 'string' && value.trim()) names.push(value)
}
}
return names
}
function assertKreaGenerateGraph(graph: WorkflowGraph) {
const loaders = loadImageNames(graph)
if (mode === 'generate') {
if (loaders.length) {
throw createError({
statusCode: 500,
statusMessage: 'Krea generate graph has a required LoadImage. Refusing to run.'
})
}
const latent = Object.values(graph).find(node => node.class_type === 'EmptyLatentImage')
if (!latent) {
throw createError({ statusCode: 500, statusMessage: 'Krea generate graph is missing EmptyLatentImage.' })
}
if (Object.values(graph).some(node => node.class_type === 'EmptyFlux2LatentImage' || node.class_type === 'Flux2Scheduler')) {
throw createError({ statusCode: 500, statusMessage: 'Krea generate graph contains a Klein latent or scheduler. Refusing to run.' })
}
const clip = graph['5']
if (!clip || clip.class_type !== 'CLIPLoader' || String(clip.inputs.type || '') !== IMAGE_V2_KREA_CLIP_TYPE) {
throw createError({ statusCode: 500, statusMessage: 'Krea generate graph CLIPLoader type must be krea2.' })
}
const banned = graphFileNames(graph).filter((name) => {
const id = modelIdentity(name)
return /flux-?2-klein|klein_snofs|flux2-klein-9b-consistency|qwen_3_8b|full_encoder_small_decoder/i.test(id)
})
if (banned.length) {
throw createError({
statusCode: 500,
statusMessage: 'Krea generate graph includes a Klein model or LoRA. Refusing to run.'
})
}
}
export function assertImageV2Graph(graph: WorkflowGraph, mode: ImageV2Mode, imageBName?: string, engine: ImageV2Engine = 'flux') {
const loaders = loadImageNames(graph)
if (mode === 'generate' && engine === 'krea') {
assertKreaGenerateGraph(graph)
} else if (mode === 'generate') {
if (loaders.length) {
throw createError({
statusCode: 500,
@@ -188,7 +337,92 @@ export function assertImageV2Graph(graph: WorkflowGraph, mode: ImageV2Mode, imag
}
}
function applyKreaConceptLora(graph: WorkflowGraph, loraName: string, model: number, clip: number) {
graph['7'] = {
class_type: 'LoraLoader',
inputs: {
lora_name: loraName,
strength_model: model,
strength_clip: clip,
model: ['4', 0],
clip: ['5', 0]
},
_meta: { title: 'Concept LoRA' }
}
setInput(graph, PROMPT, 'clip', ['7', 1])
setInput(graph, NEGATIVE, 'clip', ['7', 1])
setInput(graph, '15', 'model', ['7', 0])
}
function buildKreaGenerateWorkflow(params: ImageV2BuildParams) {
if (!params.kreaUnetName || !params.kreaClipName || !params.kreaVaeName) {
throw createError({
statusCode: 503,
statusMessage: 'Krea models were not resolved. Refusing to fall back to Flux.'
})
}
const graph = structuredClone(kreaGenerateTemplate) as WorkflowGraph
const prompt = composeImageV2Prompt('generate', 't2i', params.prompt)
const negative = String(params.negative || '')
const size = clampImageV2Size(params.width, params.height)
const steps = clampImageSteps(params.steps, IMAGE_V2_KREA_STEPS)
const cfg = clampImageCfg(params.cfg, IMAGE_V2_KREA_CFG)
const conceptName = String(params.kreaConceptLora || '').trim()
const snofsModel = conceptName ? clampImageV2Strength(params.snofsModel, 0) : 0
const snofsClip = conceptName ? clampImageV2Strength(params.snofsClip, 0) : 0
const workflowFile = IMAGE_V2_KREA_GENERATE_WORKFLOW
setInput(graph, '4', 'unet_name', params.kreaUnetName)
setInput(graph, '5', 'clip_name', params.kreaClipName)
setInput(graph, '5', 'type', IMAGE_V2_KREA_CLIP_TYPE)
setInput(graph, '6', 'vae_name', params.kreaVaeName)
setInput(graph, '14', 'width', size.width)
setInput(graph, '14', 'height', size.height)
setInput(graph, PROMPT, 'text', prompt)
setInput(graph, NEGATIVE, 'text', negative)
setInput(graph, '15', 'seed', params.seed)
setInput(graph, '15', 'steps', steps)
setInput(graph, '15', 'cfg', cfg)
setInput(graph, '15', 'sampler_name', 'euler')
setInput(graph, '15', 'scheduler', 'simple')
setInput(graph, SAVE, 'filename_prefix', params.filenamePrefix || 'aigen-v2-krea')
if (conceptName && (snofsModel > 0 || snofsClip > 0)) {
const resolved = resolveRequiredLora(conceptName, 'Krea concept')
if (/klein_snofs|flux2-klein-9b-consistency|flux-?2-klein/i.test(loraIdentityKey(resolved))) {
throw createError({ statusCode: 500, statusMessage: 'Refusing to load a Klein LoRA into the Krea graph.' })
}
applyKreaConceptLora(graph, resolved, snofsModel, snofsClip)
}
assertImageV2Graph(graph, 'generate', undefined, 'krea')
const loaders = loadImageNames(graph)
console.log(JSON.stringify({
src: 'image-v2',
engine: 'krea',
workflow: workflowFile,
mode: 'generate',
task: 't2i',
size: { width: size.width, height: size.height },
models: {
unet: params.kreaUnetName,
clip: params.kreaClipName,
vae: params.kreaVaeName
},
loras: {
concept: graph['7']
? { name: graph['7'].inputs.lora_name, model: snofsModel, clip: snofsClip }
: { loaded: false, model: 0, clip: 0 }
},
steps,
cfg,
seed: params.seed
}))
return { graph, workflowFile, loaders, prompt, strength: undefined }
}
export function buildImageV2Workflow(params: ImageV2BuildParams) {
const engine = params.mode === 'generate' ? parseImageV2Engine(params.engine, 'flux') : 'flux'
if (params.mode === 'generate' && engine === 'krea') return buildKreaGenerateWorkflow(params)
const compose = params.mode === 'compose'
const refine = params.mode === 'refine'
const generate = params.mode === 'generate'
@@ -214,9 +448,7 @@ export function buildImageV2Workflow(params: ImageV2BuildParams) {
const cfg = clampImageCfg(params.cfg, 4)
const strength = refine ? clampImageV2Denoise(params.strength, IMAGE_V2_DENOISE_DEFAULT) : undefined
const size = generate ? clampImageV2Size(params.width, params.height) : null
const workflowFile = generate
? IMAGE_V2_GENERATE_WORKFLOW
: refine ? IMAGE_V2_REFINE_WORKFLOW : compose ? IMAGE_V2_COMPOSE_WORKFLOW : IMAGE_V2_EDIT_WORKFLOW
const workflowFile = imageV2WorkflowFile(params.mode, engine)
if (!generate) setInput(graph, LOAD_A, 'image', params.imageAName || '')
if (compose) setInput(graph, LOAD_B, 'image', params.imageBName || '')
@@ -249,11 +481,12 @@ export function buildImageV2Workflow(params: ImageV2BuildParams) {
setInput(graph, CONSISTENCY, 'strength_clip', consistencyClip)
}
assertImageV2Graph(graph, params.mode, generate || refine ? undefined : params.imageBName)
assertImageV2Graph(graph, params.mode, generate || refine ? undefined : params.imageBName, engine)
const loaders = loadImageNames(graph)
console.log(JSON.stringify({
src: 'image-v2',
engine,
workflow: workflowFile,
mode: params.mode,
task: params.task,
@@ -285,16 +518,17 @@ export const IMAGE_V2_NODE_LABELS: Record<string, string> = {
'2': 'Scaling image A',
'23': 'Scaling image B',
'31': 'Resizing mask',
'4': 'Loading Flux.2 Klein 9B',
'4': 'Loading diffusion model',
'5': 'Loading CLIP',
'6': 'Loading VAE',
'7': 'Applying SNOFS',
'7': 'Applying Concept LoRA',
'8': 'Applying Consistency',
'9': 'Encoding prompt',
'11': 'Encoding image A',
'24': 'Encoding image B',
'33': 'Applying mask',
'14': 'Building empty Klein latent',
'14': 'Building empty latent',
'15': 'Sampling',
'19': 'Sampling Klein v2',
'20': 'Decoding still',
'21': 'Saving still'
+2
View File
@@ -54,6 +54,7 @@ export interface StudioJobPayload {
imagePipeline?: 'v1' | 'v2'
v2Mode?: 'edit' | 'compose' | 'refine' | 'generate'
v2Task?: 'scene' | 'identity' | 'outfit' | 'face_lock' | 'refine' | 't2i'
engine?: 'flux' | 'krea'
snofsModel?: number
snofsClip?: number
consistencyModel?: number
@@ -753,6 +754,7 @@ async function startStudioEditJob(item: StudioJob) {
await markStudioLive(item.ownerKey, item.id, live.id)
void runEditV2(live, {
mode,
engine: mode === 'generate' && payload.engine === 'krea' ? 'krea' : 'flux',
task: payload.v2Task || 'scene',
image: mode === 'generate' ? null : image,
reference: mode === 'refine' || mode === 'generate' ? null : reference,
+28
View File
@@ -1,8 +1,10 @@
export const IMAGE_V2_MODES = ['edit', 'compose', 'refine', 'generate'] as const
export const IMAGE_V2_TASKS = ['scene', 'identity', 'outfit', 'face_lock', 'refine', 't2i'] as const
export const IMAGE_V2_ENGINES = ['flux', 'krea'] as const
export type ImageV2Mode = (typeof IMAGE_V2_MODES)[number]
export type ImageV2Task = (typeof IMAGE_V2_TASKS)[number]
export type ImageV2Engine = (typeof IMAGE_V2_ENGINES)[number]
export const IMAGE_V2_STEPS_DEFAULT = 24
export const IMAGE_V2_CFG_DEFAULT = 4
@@ -52,6 +54,18 @@ export type ImageV2GenerateAspect = keyof typeof IMAGE_V2_GENERATE_ASPECTS
export const IMAGE_V2_GENERATE_SFW = { snofsModel: 0, snofsClip: 0 }
export const IMAGE_V2_GENERATE_SNOFS = { snofsModel: 0.65, snofsClip: 0.35 }
export const IMAGE_V2_KREA_STEPS = 8
export const IMAGE_V2_KREA_CFG = 1
export const IMAGE_V2_KREA_CLIP = 'qwen3vl_4b_fp8_scaled.safetensors'
export const IMAGE_V2_KREA_CLIP_TYPE = 'krea2'
export const IMAGE_V2_KREA_VAE = 'qwen_image_vae.safetensors'
export const IMAGE_V2_KREA_UNET_CANDIDATES = [
'krea2_turbo_mxfp8.safetensors',
'krea2_turbo_nvfp4.safetensors',
'krea2_turbo_fp8_scaled.safetensors'
] as const
export const IMAGE_V2_KREA_CONCEPT_LORA = ''
export const IMAGE_V2_ROLE_HEADERS: Record<Exclude<ImageV2Task, 'scene' | 'refine' | 't2i'>, string> = {
outfit: 'Person, face, body, pose, and background from image 1. Clothing only from image 2. Fit the outfit from image 2 to the body in image 1. Do not copy image 2’s face, body shape, or pose.',
face_lock: 'Body, pose, and scene from image 1. Exact face from image 2.',
@@ -68,6 +82,19 @@ export function parseImageV2Task(raw: unknown, fallback: ImageV2Task = 'scene'):
return IMAGE_V2_TASKS.includes(value as ImageV2Task) ? value as ImageV2Task : fallback
}
export function parseImageV2Engine(raw: unknown, fallback: ImageV2Engine = 'flux'): ImageV2Engine {
const value = String(raw || '').trim().toLowerCase()
return IMAGE_V2_ENGINES.includes(value as ImageV2Engine) ? value as ImageV2Engine : fallback
}
export function imageV2WorkflowFile(mode: ImageV2Mode, engine: ImageV2Engine = 'flux') {
if (mode === 'generate' && engine === 'krea') return 'krea_v2_generate.json'
if (mode === 'generate') return 'klein_v2_generate.json'
if (mode === 'refine') return 'klein_v2_refine.json'
if (mode === 'compose') return 'klein_v2_compose.json'
return 'klein_v2_edit.json'
}
export function clampImageV2Strength(raw: unknown, fallback: number) {
const value = Number(raw)
if (!Number.isFinite(value)) return fallback
@@ -118,6 +145,7 @@ export function composeImageV2Prompt(mode: ImageV2Mode, task: ImageV2Task, promp
export type ImageV2PresetSettings = {
mode?: ImageV2Mode
task?: ImageV2Task
engine?: ImageV2Engine
negative?: string
snofsModel?: number
snofsClip?: number
+10 -3
View File
@@ -1,5 +1,5 @@
import { loraDisplayName, normalizeLoraStack, type LoraStackItem } from '~/utils/loras'
import type { ImageV2Mode, ImageV2Task } from '~/utils/imageV2'
import type { ImageV2Engine, ImageV2Mode, ImageV2Task } from '~/utils/imageV2'
import type { VideoWorkflowId } from '~/utils/videoModels'
export type StillGenerationKind = 'image' | 'imagev2'
@@ -7,6 +7,7 @@ export type StillGenerationKind = 'image' | 'imagev2'
export type StillGenerationSettings = {
kind: StillGenerationKind
mode?: ImageV2Mode
engine?: ImageV2Engine
task?: ImageV2Task
negative?: string
snofsModel?: number
@@ -64,6 +65,7 @@ export function normalizeStillSettings(raw: unknown): StillGenerationSettings |
const mode = value.mode === 'compose' || value.mode === 'refine' || value.mode === 'generate' || value.mode === 'edit'
? value.mode
: undefined
const engine = value.engine === 'krea' || value.engine === 'flux' ? value.engine : undefined
const kind: StillGenerationKind | undefined = value.kind === 'imagev2' || mode
? 'imagev2'
: value.kind === 'image'
@@ -77,6 +79,7 @@ export function normalizeStillSettings(raw: unknown): StillGenerationSettings |
return {
kind,
mode,
engine,
task,
negative: typeof value.negative === 'string' ? value.negative : undefined,
snofsModel: finiteNumber(value.snofsModel),
@@ -127,7 +130,8 @@ export function stillModeLabel(still: { filename?: string; settings?: StillGener
? 'Refine'
: 'Edit'
const task = settings.mode === 'compose' ? composeTaskLabel(settings.task) : ''
return task ? `Image v2 · ${mode} · ${task}` : `Image v2 · ${mode}`
const engine = settings.mode === 'generate' && settings.engine === 'krea' ? 'Krea' : ''
return [mode === 'Generate' && engine ? `Image v2 · ${mode} · ${engine}` : `Image v2 · ${mode}`, task].filter(Boolean).join(' · ')
}
if (settings?.kind === 'image') return 'Image edit'
if (/aigen_v2_/i.test(String(still.filename || ''))) return 'Image v2'
@@ -147,8 +151,11 @@ export function stillSettingsRows(still: { width?: number; height?: number; sett
} else if (still.width && still.height) {
rows.push({ label: 'Output', value: `${still.width}×${still.height}` })
}
if (settings.mode === 'generate' && settings.engine) {
rows.push({ label: 'Engine', value: settings.engine === 'krea' ? 'Krea' : 'Flux' })
}
if (settings.snofsModel != null || settings.snofsClip != null) {
rows.push({ label: 'SNOFS', value: formatPair(settings.snofsModel, settings.snofsClip) })
rows.push({ label: 'Concept LoRA', value: formatPair(settings.snofsModel, settings.snofsClip) })
}
if (settings.mode !== 'generate' && (settings.consistencyModel != null || settings.consistencyClip != null)) {
rows.push({ label: 'Consistency', value: formatPair(settings.consistencyModel, settings.consistencyClip) })