towsty/aigen
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

[WIP] Studio 2: sticky identity and isolated generation bench #1

Draft
towsty wants to merge 6 commits from studio-2 into main
pull from: studio-2
merge into: :main
Conversation 0 Commits 6 Files Changed 26
+1999 -109
towsty commented 2026-09-11 01:11:07 +00:00
Owner
Copy Link
Copy Source

Studio 2 implementation in an isolated branch. Production stays on main; do not merge until human review and successful image/video saves. First commit adds preview environment comments only.

Graph inspection (existing assets):

  • workflow_minimax_video.json: node 105:104 MiniMaxH3ImageToVideo has first_frame. No MiniMaxH3ReferenceToVideo, Add Guide, or last_frame input.
  • workflow_minimax_video_v2.json: node 155 MiniMaxH3ReferenceToVideo exists, ref_image_size=match, reference slot 0 comes from resized start frame 128. Node 173 MiniMaxH3ImageToVideo has first_frame. No Add Guide or last_frame input. This does not implement sticky hero identity as Picture 1 plus a separate motion start frame.
  • klein_v2_compose.json and krea_v2_compose.json: second LoadImage is node 22 (first is 1).
  • klein_v2_refine.json and krea_v2_refine.json: mask LoadImage 30, ImageToMask 32, GrowMask 34, SetLatentNoiseMask 33 are present.
  • klein_v2_edit.json and krea_v2_edit.json: LoadImage 1; generate variants have no LoadImage. No changes to the existing graphs in this first commit.

Preview isolation: do NOT mount production LIBRARY_DIR read-write. Use LIBRARY_DIR=/data/library-preview (or /tmp/library-preview), COMFY_FILENAME_PREFIX=video/MiniMax_H3_preview, COMFY_AUDIO_PREFIX=audio/AIGen_preview, BACKUP_STUDIO_URL empty, and a new NUXT_SESSION_SECRET. Same Beast Comfy 8198/control 8199, one GPU; distinct prefix prevents preview purge deleting 1.0 files. Same git repository; the Coolify PR preview is the public link. Do not create a long-lived aigen-next app. No generation tests while existing runs are active.
Implementation now available at /studio-2 (head 8c464f2): separate bench with sticky folder hero, sectioned prompts/camera chips, settings drawer, family strip, details and Use as input round-trip. Durable Studio 2 records run through the existing shared GPU queue. Native H3 R2V puts hero in Picture 1 at max size; Add Guide nodes independently anchor start, end and mid-shot images. Last-frame handoff and original-segment stitching remain the existing helpers. Klein attaches the hero as separate reference conditioning on every iteration. Library items carry prompt sections, identity/start/end IDs, settings, graph ID, Comfy prompt ID, times and purge result.

Validation: 78 tests passed; two optional CPU media-fixture tests skipped. Production build passes. Isolated SSR/API smoke check renders the new bench and verifies Compose without B / Refine without mask return HTTP 400, with all Comfy addresses pointed to a closed local port. The installed native node schemas match the new graph's required inputs. Mocked runner tests verify save-before-purge, sticky identity and resume without duplicate submission. No GPU generation or upscale tests ran.

Not merge-ready until human live validation:

  • No actual Studio 2 image/video save or ten-shot identity comparison has run. Keep 1.0 pages and production apps on main.
  • Studio 2 is AIGen-first; xAIGen keeps its existing LTX studio. Krea 2 is offered for Generate and masked Refine; sticky-hero Edit/Compose/Iterate require explicitly selecting Klein. There is no silent Krea-to-Flux fallback.
  • Strict cleanup uses the new /studio2/purge endpoint. The host agent was NOT reloaded during ongoing production tests. Update the branch's host script and its scripts/studio2-purge.mjs + shared/studio2/contracts.mjs dependencies only after tests are idle. Until then, generation can save to the preview library but cleanup reports Left on host; it never falls back to the legacy broad purge helper.
  • Main, production domains/volumes/auth, existing generation graphs and the separate unpushed upscale-confirmation changes remain untouched.
Studio 2 implementation in an isolated branch. Production stays on main; do not merge until human review and successful image/video saves. First commit adds preview environment comments only. Graph inspection (existing assets): - workflow_minimax_video.json: node 105:104 MiniMaxH3ImageToVideo has first_frame. No MiniMaxH3ReferenceToVideo, Add Guide, or last_frame input. - workflow_minimax_video_v2.json: node 155 MiniMaxH3ReferenceToVideo exists, ref_image_size=match, reference slot 0 comes from resized start frame 128. Node 173 MiniMaxH3ImageToVideo has first_frame. No Add Guide or last_frame input. This does not implement sticky hero identity as Picture 1 plus a separate motion start frame. - klein_v2_compose.json and krea_v2_compose.json: second LoadImage is node 22 (first is 1). - klein_v2_refine.json and krea_v2_refine.json: mask LoadImage 30, ImageToMask 32, GrowMask 34, SetLatentNoiseMask 33 are present. - klein_v2_edit.json and krea_v2_edit.json: LoadImage 1; generate variants have no LoadImage. No changes to the existing graphs in this first commit. Preview isolation: do NOT mount production LIBRARY_DIR read-write. Use LIBRARY_DIR=/data/library-preview (or /tmp/library-preview), COMFY_FILENAME_PREFIX=video/MiniMax_H3_preview, COMFY_AUDIO_PREFIX=audio/AIGen_preview, BACKUP_STUDIO_URL empty, and a new NUXT_SESSION_SECRET. Same Beast Comfy 8198/control 8199, one GPU; distinct prefix prevents preview purge deleting 1.0 files. Same git repository; the Coolify PR preview is the public link. Do not create a long-lived aigen-next app. No generation tests while existing runs are active. Implementation now available at /studio-2 (head 8c464f2): separate bench with sticky folder hero, sectioned prompts/camera chips, settings drawer, family strip, details and Use as input round-trip. Durable Studio 2 records run through the existing shared GPU queue. Native H3 R2V puts hero in Picture 1 at max size; Add Guide nodes independently anchor start, end and mid-shot images. Last-frame handoff and original-segment stitching remain the existing helpers. Klein attaches the hero as separate reference conditioning on every iteration. Library items carry prompt sections, identity/start/end IDs, settings, graph ID, Comfy prompt ID, times and purge result. Validation: 78 tests passed; two optional CPU media-fixture tests skipped. Production build passes. Isolated SSR/API smoke check renders the new bench and verifies Compose without B / Refine without mask return HTTP 400, with all Comfy addresses pointed to a closed local port. The installed native node schemas match the new graph's required inputs. Mocked runner tests verify save-before-purge, sticky identity and resume without duplicate submission. No GPU generation or upscale tests ran. Not merge-ready until human live validation: - No actual Studio 2 image/video save or ten-shot identity comparison has run. Keep 1.0 pages and production apps on main. - Studio 2 is AIGen-first; xAIGen keeps its existing LTX studio. Krea 2 is offered for Generate and masked Refine; sticky-hero Edit/Compose/Iterate require explicitly selecting Klein. There is no silent Krea-to-Flux fallback. - Strict cleanup uses the new /studio2/purge endpoint. The host agent was NOT reloaded during ongoing production tests. Update the branch's host script and its scripts/studio2-purge.mjs + shared/studio2/contracts.mjs dependencies only after tests are idle. Until then, generation can save to the preview library but cleanup reports Left on host; it never falls back to the legacy broad purge helper. - Main, production domains/volumes/auth, existing generation graphs and the separate unpushed upscale-confirmation changes remain untouched.
towsty added 1 commit 2026-09-11 01:11:07 +00:00
Document isolated Studio 2 PR preview settings d4f2b86587
towsty added 4 commits 2026-09-11 01:57:14 +00:00
Add native H3 identity graph and Studio 2 prompt contracts bd82f5bd5f
Queue durable Studio 2 jobs with sticky identity and saved metadata 70b6b70fcd
Restrict Studio 2 host cleanup to exact preview paths 70c17695e4
Add the Studio 2 bench and restore saved prompt sections 8c464f283d
towsty marked the pull request as work in progress 2026-09-11 01:57:51 +00:00
towsty added 1 commit 2026-09-11 02:31:46 +00:00
Enable the isolated xAIGen Studio 2 preview fd9522ae19
towsty added 1 commit 2026-09-11 02:33:43 +00:00
Return preview users to Studio 2 after login f312cff477
towsty added 3 commits 2026-09-11 04:52:11 +00:00
Model camera beats and independent hero identity locks 3b071436a3
Restore Studio 2 controls and automatic extension binding 5c69650ffa
Fit Studio 2 inputs and report live generation progress 09c96c0ae7
towsty added 1 commit 2026-09-11 12:07:27 +00:00
Separate Studio 2 source and canvas and apply size and seed choices bf58732690
towsty added 1 commit 2026-09-11 14:28:03 +00:00
Select and confirm uploaded Studio 2 source and hero stills c719e12709
towsty added 1 commit 2026-09-11 14:33:07 +00:00
Use desktop width and protect the Studio 2 prompt column 8fe69272c3
towsty added 1 commit 2026-09-11 17:23:37 +00:00
Sample Klein edits from source and bind Studio 2 extensions correctly 28b919ecee
towsty added 1 commit 2026-09-11 23:44:14 +00:00
Retry confirmed Studio 2 host cleanup after failures b1ffcd6975
towsty added 3 commits 2026-09-12 00:04:50 +00:00
Keep Studio 2 family tiles tied to existing library files b5a4882a9c
Compile image style chips and restore complete input metadata 86ffa9ba04
Simplify Studio 2 sources and persist private previews and LoRA lists ad135ae19c
towsty added 1 commit 2026-09-12 00:06:55 +00:00
Bind the selected output when restoring complete input settings ec0b6f10ab
towsty added 1 commit 2026-09-12 00:30:20 +00:00
Upload files to "components/studio2" 937e4d120c 
FIX EFFING EXTEND BUTTON
towsty added 1 commit 2026-09-12 00:30:52 +00:00
FIX EFFING EXTEND FILE c6451b10ce
towsty added 1 commit 2026-09-12 02:12:55 +00:00
Stop resampling the extend start frame every hop 65c69b779a 
upload() ran fitStill (ffmpeg scale+pad) on every image except hero,
including the start/handoff still. Four chained extends resampled the
anchor frame four times and destroyed identity even with the hero
reference and lock text intact.

- upload(): only fit when the still's aspect truly differs from the
  job canvas (needsFit); never fit hero or start regardless.
- Track handoffSource (anchor|extract) and whether the start frame was
  fitted; surface both in job metadata and Details.
- extendSelected(): re-binding an already-selected clip on task switch
  must not stomp the user's overridden locks/settings back to the
  clip's saved values (regression introduced by the last hotfix push).
- Add tests/studio2-extend-fit.test.mjs: two extends at identical
  width/height reach Comfy with the anchor PNG byte-identical, no
  ffmpeg fit.

Co-authored-by: Cursor <cursoragent@cursor.com>
towsty added 1 commit 2026-09-12 04:11:08 +00:00
Tighten needsFit to exact canvas size, guard against empty renders d8f42ecc68 
needsFit only compared aspect ratio, so a same-aspect-but-larger still
(e.g. a 4032x2268 phone photo against a 1344x768 canvas) would skip
fitting and hand MiniMaxH3AddGuide a still that does not match the
job's own latent size. Require an exact width+height match instead;
anything else (including unreadable dimensions) still fits.

Also guard the video save path: an empty Comfy download, an empty
stitched assembly, or a stitched result with no readable duration now
fails the job with a clear error instead of silently saving/attaching
a broken zero-length clip.

Co-authored-by: Cursor <cursoragent@cursor.com>
towsty added 3 commits 2026-09-12 16:23:58 +00:00
Encode Unicode video titles in playback response headers a9551911f2
Show clear GPU status and move upscale progress into the queue 3e90d2ff45
Show pending Studio 2 jobs and freeze finished timers 3da3656140
towsty added 1 commit 2026-09-12 16:47:04 +00:00
Sample Klein hero identity from image latents 7bb4b16c1e
towsty added 1 commit 2026-09-12 17:18:21 +00:00
Restore Klein empty-latent reference sampling e190496eb4
towsty added 2 commits 2026-09-13 01:18:07 +00:00
Filter LoRA picker options by engine tokens 6a6931f04e
Remember Studio 2 LoRA stacks for each engine 401e6c26a2
towsty added 1 commit 2026-09-13 01:45:33 +00:00
Show each LoRA dropdown label once fb87a50a39
towsty added 1 commit 2026-09-13 02:05:21 +00:00
Read production LoRA names in Studio 2 064f1db933
towsty added 6 commits 2026-09-14 23:46:55 +00:00
Add YuE2 host, worker, and setup scripts. 85ef087364 
Co-authored-by: Cursor <cursoragent@cursor.com>
Route engine yue2 through a standalone job helper. 5f68205f01 
Co-authored-by: Cursor <cursoragent@cursor.com>
Mount YuE2 on the host agent and share GPU ownership. ec5791c595 
Co-authored-by: Cursor <cursoragent@cursor.com>
Add YuE2 to the music engine picker and generate API. 71a61b1765 
Co-authored-by: Cursor <cursoragent@cursor.com>
Document YuE2 install, weight cleanup, and regression tests. d1ff05089f 
Co-authored-by: Cursor <cursoragent@cursor.com>
Finish YuE2 host setup, weight download paths, and v1 cleanup. fae3b23e37 
Co-authored-by: Cursor <cursoragent@cursor.com>
towsty added 1 commit 2026-09-15 00:28:47 +00:00
Allow multi-section YuE2 lyrics and force non-flash attention on Windows. 62671d05da 
Co-authored-by: Cursor <cursoragent@cursor.com>
towsty added 1 commit 2026-09-15 01:01:08 +00:00
Tighten YuE2 VRAM use on the 5080 and empty GPU before launch. ee84a99035 
Co-authored-by: Cursor <cursoragent@cursor.com>
towsty added 1 commit 2026-09-15 01:14:46 +00:00
Remove YuE2 process memory fraction cap on the 5080. 840440d7ef 
Co-authored-by: Cursor <cursoragent@cursor.com>
towsty added 8 commits 2026-09-15 23:20:28 +00:00
Replace stale host agent on boot and document one-click Startup install. df885d5732 
Co-authored-by: Cursor <cursoragent@cursor.com>
Accept only YuE2 as the music engine type. 17cc07e3bc 
Co-authored-by: Cursor <cursoragent@cursor.com>
Accept only YuE2 in the music generate API. c50bde723f 
Co-authored-by: Cursor <cursoragent@cursor.com>
Route all music jobs through YuE2 only. cce1faa9b1 
Co-authored-by: Cursor <cursoragent@cursor.com>
Remove YuEGP handlers from the Comfy host agent. a216afc4ce 
Co-authored-by: Cursor <cursoragent@cursor.com>
Make Studio 2 the home bench with YuE2 music, LTX, and per-preview hide. 1db62e9187 
Co-authored-by: Cursor <cursoragent@cursor.com>
Drop ACE and YuEGP defaults from studio music queue and library labels. c868605d8a 
Co-authored-by: Cursor <cursoragent@cursor.com>
Reject instrumental music edits and force YuE2 on queued track patches. 2ebc3066e6 
Co-authored-by: Cursor <cursoragent@cursor.com>
towsty added 1 commit 2026-09-16 01:53:09 +00:00
Keep YuE2 jobs live on the bench and wire target length to semantic tokens. 5694104226 
Co-authored-by: Cursor <cursoragent@cursor.com>
towsty added 1 commit 2026-09-16 19:56:19 +00:00
Allow Studio 2 MiniMax text-to-video with empty start when locks are off. 36d24feb9c 
Co-authored-by: Cursor <cursoragent@cursor.com>
towsty added 5 commits 2026-09-18 18:12:14 +00:00
Add Classic | Studio displayMode preference and Settings cards. 292d53a20e 
Co-authored-by: Cursor <cursoragent@cursor.com>
Move Studio locks to slots and Classic hide to header checkboxes. 1b29452455 
Co-authored-by: Cursor <cursoragent@cursor.com>
Bind Classic model select and Studio engine rail to the same engine ref. d9743dd8c9 
Co-authored-by: Cursor <cursoragent@cursor.com>
Restyle Studio task rail and bind Classic task to the same mode ref. bdc1ab6c2d 
Co-authored-by: Cursor <cursoragent@cursor.com>
Tune Studio display tokens for rails, radius, and amber CTA. eba701a82c 
Co-authored-by: Cursor <cursoragent@cursor.com>
towsty added 1 commit 2026-09-18 23:57:14 +00:00
Replace LoRA Thumbwheel spectrograms with range and numeric strengths. 4831b32d20 
Co-authored-by: Cursor <cursoragent@cursor.com>
towsty added 4 commits 2026-09-20 21:22:58 +00:00
Add Qwen 2.1 host setup script and document Shared model paths. 277b49403c 
Co-authored-by: Cursor <cursoragent@cursor.com>
Add studio2_qwen21_t2i Comfy graph for GGUF Qwen Image 2.1 T2I. b95e8b2a87 
Co-authored-by: Cursor <cursoragent@cursor.com>
Wire Qwen 2.1 as Generate-only engine with 25/1 sampler defaults. 4e61ee17b6 
Co-authored-by: Cursor <cursoragent@cursor.com>
Fix Qwen 2.1 GGUF load: promote Q8 norms to F32 and wire TextEncode latent. 1f10082087 
abenzerps Q8_0 ships 1D RMSNorms as packed Q8 (136 vs 128), which breaks
Comfy rms_rope; tagger now dequantizes small tensors and the graph uses
TextEncodeQwenImage21's 64-ch latent plus AuraFlow shift.

Co-authored-by: Cursor <cursoragent@cursor.com>
towsty added 1 commit 2026-10-01 03:06:18 +00:00
Add Studio 2 caption jobs via Qwen2.5-VL GGUF on the host agent. 1b311d6529 
Exclusive llama-server load/unload on the 5080, queued Describe UI with resultText, Copy, and Use as prompt.

Co-authored-by: Cursor <cursoragent@cursor.com>
This pull request has changes conflicting with the target branch.
  • components/studio2/Bench.vue
  • docs/qwen21.md
  • scripts/caption-host.mjs
  • scripts/comfy-host-agent.mjs
  • scripts/setup-caption.ps1
  • scripts/setup-qwen21.ps1
  • server/assets/studio2_qwen21_t2i.json
  • server/utils/caption.ts
  • server/utils/studio2/runner.ts
  • server/utils/studio2/size.ts
This pull request is marked as a work in progress.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin studio-2:studio-2
git checkout studio-2
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
towsty (Ian)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: towsty/aigen#1