Force <image1> on every Qwen Edit PE rewrite.

PE-I2I no longer bans tags for single-still edits, and the runner rewrites natural photo/image wording to <image1> so Start still stays bound.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Towsty
2026-09-26 15:04:04 -05:00
co-authored by Cursor
parent a51118d92a
commit c46e827c68
4 changed files with 18 additions and 6 deletions
+1 -1
View File
@@ -42,7 +42,7 @@ Restart Comfy **only when idle** (`COMFY_CONTROL_URL/status` → `gpu.busy=false
- Edit follows `image_1` via the encode node’s latent (resolution long-edge ~1024). Do not inject a picker EmptyLatentImage onto the edit sampler. - Edit follows `image_1` via the encode node’s latent (resolution long-edge ~1024). Do not inject a picker EmptyLatentImage onto the edit sampler.
- Optional **Enhance prompt** (off by default): runs a separate PE-only Comfy graph (`CLIPLoader` + rewrite node), then frees VRAM and queues the existing T2I/Edit graph with the rewritten prompt. Never loads PE CLIP + DiT together on 16 GB. Fail closed if `parse_ok` is false or the rewrite is empty. - Optional **Enhance prompt** (off by default): runs a separate PE-only Comfy graph (`CLIPLoader` + rewrite node), then frees VRAM and queues the existing T2I/Edit graph with the rewritten prompt. Never loads PE CLIP + DiT together on 16 GB. Fail closed if `parse_ok` is false or the rewrite is empty.
- Generate + Enhance → PE-T2I only (`pe_t2i`). No images. - Generate + Enhance → PE-T2I only (`pe_t2i`). No images.
- Edit + Enhance → PE-I2I only (`pe_i2i`) with the same Start still on `image_1`. If the rewrite drops every `<imageN>` token, stitch `Keep the identity…` onto the rewrite before Edit. Never fall back to PE-T2I on Edit. - Edit + Enhance → PE-I2I only (`pe_i2i`) with the same Start still on `image_1`. After PE, every Edit rewrite is forced to contain `<image1>` (PE used to ban tags for N=1). If the rewrite still drops every `<imageN>` token, stitch `Keep the identity…` onto the rewrite before Edit. Never fall back to PE-T2I on Edit.
- If PE refuses or returns an empty/gutted rewrite, the job continues with `promptRaw` (Edit still gets the keep-identity prefix). Details shows `Enhance skipped (model refused) — used your prompt.` (`enhance.refused`). - If PE refuses or returns an empty/gutted rewrite, the job continues with `promptRaw` (Edit still gets the keep-identity prefix). Details shows `Enhance skipped (model refused) — used your prompt.` (`enhance.refused`).
- Host PE system prompts live in `host/qwen21-pe-prompts/` and are copied onto the node pack by `scripts/setup-qwen21.ps1` (official steps + Adult appendix). Do not swap Heretic PE weights on 16 GB. - Host PE system prompts live in `host/qwen21-pe-prompts/` and are copied onto the node pack by `scripts/setup-qwen21.ps1` (official steps + Adult appendix). Do not swap Heretic PE weights on 16 GB.
- PE weights (int8 only): `text_encoders\qwen3.5_9b_qwen_image_2.1_pe_{t2i,i2i}.int8_convrot.safetensors` — do not replace the image TE `qwen3vl_8b_int8_convrot.safetensors`. - PE weights (int8 only): `text_encoders\qwen3.5_9b_qwen_image_2.1_pe_{t2i,i2i}.int8_convrot.safetensors` — do not replace the image TE `qwen3vl_8b_int8_convrot.safetensors`.
@@ -56,7 +56,7 @@ Before emitting JSON, reason through: what the image(s) actually contain (includ
## Image Reference Rules ## Image Reference Rules
For Multi-Image Input (N >= 2), the rewritten instruction MUST use `<image1>`, `<image2>`, ... to refer to each input image. Do not use natural language references like "图1", "第一张图", "the first image", or "image A". This tagging format is mandatory and non-negotiable. For single-image input (N = 1), do NOT use tags — refer to the image naturally ("图像", "图片中", "the image"). For every input, the rewritten instruction MUST use <image1>, <image2>, … to refer to each input image. Single-image edits still use <image1>. Never write "the photo", "the image", "the woman in the picture" as a substitute for the tag. The first Start still is always <image1>.
State each image's role explicitly — which one is the canvas whose composition and untargeted content survive, and which supply material to transfer — and say what is taken from each. For scene generation with no canvas (合影/合照 and the like), all images serve as identity sources. Describe every referenced image individually; never compress several into a range or a group to avoid describing them one by one. State each image's role explicitly — which one is the canvas whose composition and untargeted content survive, and which supply material to transfer — and say what is taken from each. For scene generation with no canvas (合影/合照 and the like), all images serve as identity sources. Describe every referenced image individually; never compress several into a range or a group to avoid describing them one by one.
+11 -4
View File
@@ -15,12 +15,19 @@ export function ensureQwen21EditPrompt(prompt) {
} }
/** /**
* After PE-I2I: if the rewrite dropped every <imageN> token, stitch the * After PE-I2I: official PE told N=1 not to use tags, so rewrites often say
* keep-identity line onto the rewrite (do not discard the rewrite). * "the photo" / "the image" with zero <image1>. Qwen Edit only binds Start still
* when the text contains the tag — force it onto every Edit rewrite.
*/ */
export function stitchQwen21PeEditPrompt(rewritten) { export function stitchQwen21PeEditPrompt(rewritten) {
const text = String(rewritten || '').trim() let text = String(rewritten || '').trim()
if (!text) return text if (!text) return text
if (/<image\d+>/i.test(text)) return text // Prefer the tag over natural substitutes (do not invent <image2>).
text = text
.replace(/\bthe woman in the (?:input )?(?:photo|picture|image)\b/gi, 'the person in <image1>')
.replace(/\bthe (?:person|man|subject) in the (?:input )?(?:photo|picture|image)\b/gi, 'the person in <image1>')
.replace(/\bin the (?:input )?(?:photo|picture|image)\b/gi, 'in <image1>')
.replace(/\bthe (?:input )?(?:photo|picture|image)\b/gi, '<image1>')
if (/<image1>/i.test(text)) return text
return `${QWEN21_EDIT_KEEP} ${text}`.trim() return `${QWEN21_EDIT_KEEP} ${text}`.trim()
} }
+5
View File
@@ -108,6 +108,11 @@ test('qwen21 supports Generate and Edit with 25/1 defaults and no locks',()=>{
assert.equal(ensureQwen21EditPrompt('change <image1> outfit'),'change <image1> outfit') assert.equal(ensureQwen21EditPrompt('change <image1> outfit'),'change <image1> outfit')
assert.match(stitchQwen21PeEditPrompt('a woman in a red jacket'),/<image1>/) assert.match(stitchQwen21PeEditPrompt('a woman in a red jacket'),/<image1>/)
assert.equal(stitchQwen21PeEditPrompt('keep <image1> add jacket'),'keep <image1> add jacket') assert.equal(stitchQwen21PeEditPrompt('keep <image1> add jacket'),'keep <image1> add jacket')
assert.match(stitchQwen21PeEditPrompt('change the jacket on the woman in the photo'),/<image1>/)
assert.doesNotMatch(stitchQwen21PeEditPrompt('change the jacket on the woman in the photo'),/\bthe photo\b/i)
assert.match(readFileSync(new URL('../host/qwen21-pe-prompts/system_prompt_edit.txt',import.meta.url),'utf8'),/Single-image edits still use <image1>/)
assert.doesNotMatch(readFileSync(new URL('../host/qwen21-pe-prompts/system_prompt_edit.txt',import.meta.url),'utf8'),/do NOT use tags/)
assert.doesNotMatch(readFileSync(new URL('../host/qwen21-pe-prompts/system_prompt_t2i.txt',import.meta.url),'utf8'),/Single-image edits still use <image1>/)
assert.equal(qwen21PeRefusal('corgi in the rain',{positive_prompt:'A photorealistic photograph of a corgi standing in rain…',parse_ok:true}).refused,false) assert.equal(qwen21PeRefusal('corgi in the rain',{positive_prompt:'A photorealistic photograph of a corgi standing in rain…',parse_ok:true}).refused,false)
assert.equal(qwen21PeRefusal('nude adult portrait',{positive_prompt:"I cannot assist with that request.",thinking:'',parse_ok:true}).refused,true) assert.equal(qwen21PeRefusal('nude adult portrait',{positive_prompt:"I cannot assist with that request.",thinking:'',parse_ok:true}).refused,true)
assert.equal(qwen21PeRefusal('put her in a red leather jacket',{positive_prompt:'',parse_ok:false}).refused,true) assert.equal(qwen21PeRefusal('put her in a red leather jacket',{positive_prompt:'',parse_ok:false}).refused,true)