Feature Announcement Thread

[mcmonkey4eva]

This is a dedicated thread you can follow where new features will be announced. This will only be used for the biggest features.

You can find the previous Alpha/Beta Stability AI era feature announcement thread at Stability-AI/StableSwarmUI#11

You'll also find more news on Discord at https://discord.gg/q2y38cqjNw - the announcements channel there has smaller updates.

The releases page https://github.com/mcmonkeyprojects/SwarmUI/releases will contain bulk summaries of updates in more detail than here

And, of course, if you're very obsessive, you can watch every individual git commit https://github.com/mcmonkeyprojects/SwarmUI/commits/master/

[mcmonkey4eva]

SD3 Controlnets (eg https://huggingface.co/InstantX) are now supported in Swarm! (thanks of course to comfy doing the actual hard work on the internals) They use the new ControlNetApplySD3 node internally, and will be autodetected by the model architecture handler (because InstantX published in diffusers format ie no coherent metadata by default).

(I recommend if you download the instantx models, to rename the files when you save them and then edit metadata so they're not all the helplessly blank diffusion_pytorch_model.safetensors). If you name the canny model with canny in the title and the pose model with pose in the title, the preferred preprocessor will be autodetected from the name (otherwise of course just select preprocessor manually).

Note that instantx's models are the first ever controlnets for SD3, ie not going to be the best ones.

[mcmonkey4eva]

Swarm dropdown parameters now have human-friendly names rather than the raw comfy IDs.

For example, here's the sampler list now vs before:

For example dpmpp_2m is now labeled clearly as DPM++ 2M, and has the hint-text 2nd Order Multi-Step

the list is a bit chonkier now, but hopefully it's now an intelligible text wall vs previously being a spasm of meaningless letters

You can still happily filter-search by the raw ID or the clean name

Or here's the Aspect Ratio list:

Previous it had just the numbers, without the hints

[mcmonkey4eva]

Black Forest Labs' new Flux.1 model is now supported in Swarm! https://blackforestlabs.ai/announcing-black-forest-labs/

See docs at https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Model%20Support.md#black-forest-labs-flux1-models for specific usage info

[mcmonkey4eva]

The Swarm Image Editor now has a Paintbucket tool, and an integrated SAM2 autosegmenter button, so you can for example do something like this:

[mcmonkey4eva]

New: bulk civitai metadata scanner utility

This (slightly hacky but sufficient) bulk scanner lets you update your model's metadata from civitai. Useful for if you have a lot of externally downloaded models (ie not using the swarm model downloader that autosyncs civit metadata while downloading). Has a bunch of configuration options for which models to scan / how to apply updates. To work fully, this will autocalculate the hash of any models that don't have a stored hash - be warned this is a slow process. Also, the metadata updates themselves can be slow. So, for large model folders, this will take a while (but only has to be ran once and then it's saved for all models you have in that folder, and then all those models are near-instant if you rerun over them).

ps, civitai now supports Swarm image metadata when you upload images!

[mcmonkey4eva]

New: extensions manager tab

lets you install Swarm extensions, update them, etc.

Also, works has been done to make writing extensions more approaching, including easier ways to support Comfy node pack install requirements, and the Dynamic Thresholding Extension is updated to be a good model of how to write a good extension

[mcmonkey4eva]

• Added support for AliMama Inpainting Flux Controlnet https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Beta
  -> Download the model, save in controlnet model folder, probably rename it to something less dumb than its default filename
  -> refresh swarm, find it in the Controlnets models tab, click Edit Metadata and change its architecture to Flux.1 ControlNet - AliMama Inpaint
  -> use the image editor (or any init image + mask) with max creativity
  -> make sure controlnet input group is enabled with the model selected, don't use a preprocessor
  -> mask your image, type your prompt, enjoy
  -> Note: it is very very vram hungry. overloads on 4090 in fp8, barely fits with a q4 gguf
  -> Note: it is a bit seed dependent, the model is marked beta, you'll have to try a bit to make it do what you want

[mcmonkey4eva]

Genmo Mochi 1 (Text-To-Video) Models are now natively supported in SwarmUI!

When you select a Text-To-Video model, a new Text To Video param group will appear on the left

Note that currently even on a 4090 you can barely generate a short 25 frame video, as the VAE requires more VRAM than anyone has. Edit: This is fixed now, VAE Tiling is supported!

See Model Support doc for details: https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Model%20Support.md#genmo-mochi-1-text2video

[mcmonkey4eva]

Lightricks LTX Video is fully natively supported in SwarmUI for both Text-To-Video and Image-To-Video, see notes at https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Model%20Support.md#lightricks-ltx-video

[mcmonkey4eva]

BFL's "Flux Tools" (Redux to mix images, Canny/Depth special models as almost-controlnets-but-different, Fill aka inpaint model) are fully supported in SwarmUI, including some special hack parameters to make Redux more powerful, see notes at https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Model%20Support.md#flux1-tools

[mcmonkey4eva]

New UI feature: confused about the more complex prompt syntax inputs like Segment or Regional prompts? Be confused no more, there's now helper UIs to guide you through constructing perfect inputs every time!

[mcmonkey4eva]

Hunyuan Video is now natively supported https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Model%20Support.md#hunyuan-video

[mcmonkey4eva]

I posted a public list of "small features" to implement or investigate. If you're interested in contributing to free & open source software, here's a grab-bag of things to try: #550

[mcmonkey4eva]

You can now Star models. Starred models always appear at the top of the model list (if you star a lora, it will only move to the top of it is compatible with the current model) (if a model doesn't match the current filter or is not in the selected folder, it will not be shown still).

[mcmonkey4eva]

Layout customization!

Hate the placement of the different sub-tabs? Want Image History in the center, or a Textbox on the right instead of the batch view? Now you can!

Use the new User->Generate Tab Layout to shove things into different places!

Any section that only has 1 tab in it will hide the tab-selector. The left, center, right top areas all function as tab-groups that can be moved around. You can put your parameters on the bottom or your image on the right or whatever you feel like. You can hide sub-tabs you don't want.
(Note: This is not 100%, you are limited to just 4 collection points, and only within the Generate tab. Range of freedom may be expanded in the future)

[mcmonkey4eva]

As of a few days ago, LTX-2 aka LTXV2 aka LTXAV, is now supported in SwarmUI! It's a massive 19B model, and the first true "audio+video" combo model. It generates sound and video as a pair.
As with the original LTXV, the quality is... usually pretty bad. But its technical capabilities are very impressive, and when it hits a lucky good gen it can be incredible.
It's both T2V and I2V and more (Swarm only currently supports T/I2V, the "and more" will be implemented over time, there's lots of brand new tech this model brings to the table).
Docs here https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Video%20Model%20Support.md#lightricks-ltx-video-2

The attached video is I2V from a Z-Image gen because quite frankly T2V can't write text on screen properly to save its life much less anything else.
But the other perk of LTX2: this 5 second, 1280x720, 24 fps video, was generated on my local computer in only about one minute! Despite its giant size, it runs very fast!

0231001-the.cat.creeps.slowly.forward.as.the.background.shimmers.and-ltx-2-19b-distilled-fp8.mp4

[mcmonkey4eva]

Flux.2 Klein 4B and 9B are now natively supported in SwarmUI!
These are smaller variants of Flux.2-Dev, able to run quickly and easily even on a laptop. They have official distilled and base variants. The 4B is fully open source apache2.
It's not as smart as Flux.2-Dev, but it's not terribly far off. Its detail quality isn't as good as ZImage. It's the first small+fast model to support high intelligence image editing capabilities.
Docs here https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Model%20Support.md#flux2-klein

[mcmonkey4eva]

Extensions can now have csproj files! This is a technical one for the devnerds, but, basically, extensions can now define their own dependencies and special behaviors and wotnot. This should make life easier for the group of wackos adding weird things to Swarm!

Documented here https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Making%20Extensions.md

[mcmonkey4eva]

SwarmUI now supports Anima, a tiny fast n capable anime model! The diffusion model itself is just 2B scale, based on nvidia's Cosmos architecture. It uses Qwen 600M (0.6B), a super tiny text encoder, and the Qwen Image VAE. It had some clever dataset curation that makes it very adaptable, and is released as a clean base perfect for lora training. This model is clearly hoping to become the new champion of anime lovers.
And: This was sponsored by Comfy Org! The first major model release directly supported by the champions of foss image gen.
Note the currently release is a nonfinal preview, they intend to train more and work out licensing details over time. Note also it is utterly incapable of anything not anime styled on its own, don't even try.
Docs here https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Model%20Support.md#anima

[mcmonkey4eva]

The user account engine in SwarmUI is pretty stable at this point. Not 100% feature complete, but all the basics work - multiple login methods (username/password, oauth), account registration support, fairly thorough configuration of relevant features, sensible restrictive defaults, and a few passes of informal security review have me semi-confident (but not guaranteeing!!!) that it's pretty safe to run.

Which leads me to the new feature today: User Impersonation. You can now simply click Server->Users->the user you want->Impersonate, and you instantly see and operate the whole UI as that user account.

And I know what you're thinking: "I'm one person, I don't have users, why do I care?"
Well... do you ever feel you want multiple split work environments? Say for example, plenty of people out there probably want a "Safe" setup and a "Spicy" setup, or maybe you want your Video working environment and your Image working environment.
If that sounds useful to you, then all you gotta do, is enable the account system, login is local admin, add an account for Video and an account for Image (or whatever environments), configure them with separate roles (eg maybe "Safe" user has a ModelBlacklist of NSFW/ so it can't see those models at all), and boom you can now hit the impersonate button to enter different user environments. These have separate gen histories, separate presets, etc.

[mcmonkey4eva]

General recent updates

[mcmonkey4eva]

• Swarm now supports ERNIE image. It's a new minimally-censored simple text-to-image model, 8B size, has a Turbo, runs in a few seconds on top end GPUs. Arguably competitive with z-image and klein for size-equivalent t2i capabilities, but has no edit and doesn't quite hold up in quality. Their claim to fame is excellent long prompt understanding, (how well that claim holds up to reality remains to be seen, but it's at least not actually all that great on long text compared to eg flux.2-dev.)
  Docs here https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Model%20Support.md#ernie

[mcmonkey4eva]

• Swarm now supports HiDream-o1, an 8B model, partially censored, text-to-image and image-editing, has a Dev (turbo-y), runs pretty quick on modern GPUs. Their claim to fame is "no separate VAE or text encoder" but in practice that just means both are integrated into the core model itself rather than configurable which is silly. Its patch-scaling (equivalent to VAE scaling) is aggressive and worse than most VAE's, so it runs at native 2048 but that feels more like 1024 quality-wise. (They call it a native pixel model but that's straight up just ai corp lying). Quality is quite variable and iffy, but the intelligence seems to usually be there.
  Docs here https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Model%20Support.md#hidream-o1

[mcmonkey4eva]

• Swarm now supports Microsoft Lens, a 4B model. This is a simple text-to-image with no tricks, and their main thing is just having a massive textencoder training on GPT image outputs. The upshot is it's very fast and very high quality. On a 4090 it only takes a few seconds to generate a full 1440x1440 image with clear text, iconography, and characters. Seed luck varies but it seems to mostly get what you ask for.
  Docs here https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Model%20Support.md#lens
  (ps don't forget to thank Juan in the SwarmUI discord for being our resident new model researcher, that's 3 in a row new models supported thanks to his work!)

[mcmonkey4eva]

Need to compare two images (or videos)? Swarm now has a Compare tool! To use it, activate multi-select mode in your image history viewer, select exactly two outputs, then hit Actions->Compare. This will give you a variety of comparison quicktools - side-by-side, horizontal slide, vertical slide, transparent overlay, single view, as well as a metadata view toggle to compare the metadata as well. See attached images for where to click and example of how the interface looks. Thanks to Juan for building most of this feature!

[mcmonkey4eva]

Swarm now supports Krea 2, an awesome new 13B scale simple Text2Image model. It knows a variety of characters/concepts/etc. natively, it can combine multiple difficult concepts well, it can do large amounts of text coherently, it passes obscure tests like the "can the arrow pointing at juan actually point at juan" test that all other models have failed thus far. And best of all: it's got very very good visual quality. Oh also you can do up to 4096x4096 natively if you want, or all the way down to 128. Oh also on a 4090 it takes less than 10 seconds for a 1024 gen. It is censored with a preencode layer (that will likely be removed by community finetunes). It unfortunately lacks edit capability, but other than that is basically the new pinnacle for straight t2i gen.
Docs here https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Model%20Support.md#krea-2

[mcmonkey4eva]

• Swarm now supports MiniMax H3, the first major new video model we've had in about a year! This is a game changer, and we're expecting more competition (notably Flux 3) soon too! Your old Wan/LTX models are historical artifacts now, folks! This model is ~21 gigs, using a ~16gig text encoder, so might be a bit heavy for some. The upside is it supports a very nice range of resolutions (~512ish up to ~2048ish) and durations (5 frames ie quarter-second up to 360 frames for 15 seconds!!), very long/complex prompts, video+audio output, first/last frame input OR generic video/image/audio reference input (like video controlnet). The model is capable of model-shot scenes, complex behaviors, and the general quality expectations we have of our model image models finally crossed into videos too (eg text? not a problem it can render it fine).
  (Note at time of writing, some features are still to-be-implemented in Swarm, but all the basics already work). Fully implemented!
  Docs here: https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Video%20Model%20Support.md#minimax-h3

The attached video is 10 seconds at SideLength=960 and generated exactly what I wanted on the first try :D (note: on github you have to click the volume slider to enable audio on videos)

2108001-High.definition.video.of.an.orange.cat.walking.through.a.mag-minimax_h3_fl2va_pruned_int8_convrot.mp4

Also, recent minor news

[mcmonkey4eva]

Three recent news items: Model Downloader improvement, prompt parser rework, desktop app mode!

[mcmonkey4eva]

Swarm now supports Qwen Image 2.1, a new 7B text2image model from Qwen with an extra-funny VAE. It's a 16x downscale (twice the normal 8x, so expect meh quality at 1024 and more normal quality at 2048), but it supports RGBA - transparency in images! It's also a generally capable model and supports edit / multi-image reference. It's relatively undertrained, which has the perk of great creative variety from your prompts, but the downside of really variable quality. Most of my model sample gens in announcements are made quite quickly... this one took a lot of trial and error and seed luck to get one that was halfway acceptable. This model might be awesome with some a good finetune.
Docs here https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Model%20Support.md#qwen-image-21

---

Swarm now supports Flux 3 Image Klein, a new sub-6B scale model from Black Forest Labs. This image model still uses the Flux 2 VAE, but it's the direct upgrade to Flux 2 Klein, with greater intelligence and capabilities, plus lightning speed and great high res support, but has some poor detail quality and anatomy issues. They've released a true base model making it very easy to train so the community may fix those issues. It got this sample image on the first try no problem, despite all the text and multiple complicated parts of the scene (but zoom in on any of the eyes to see the detail issues).
Docs here https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Model%20Support.md#flux-3-image

[mcmonkey4eva]

Swarm now supports MOSS Image, a new 5B scale pure text-to-image (no editing or anything) model from a Chinese university. This model uses the Flux 2 VAE, is entirely uncensored, has a base and a finetuned variant, reportedly a turbo coming soon. It's very intelligent, very high quality, can do JSON prompts and stuff, and has native prompt interpretation intelligence that details and varies your inputs without needing an LLM prompt upsampler (but they have official files for that too if you want it). This is a contender to replace Krea 2.
Docs here https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Model%20Support.md#moss-image