1. 0? Question | Help I can get the base and refiner to work independently, but how do I run them together? Am I supposed to run. if you find this helpful consider becoming a member on patreon, subscribe to my youtube for Ai applications guides. The video also. I’m sure as time passes there will be additional releases. Fix. Study this workflow and notes to understand the basics of ComfyUI, SDXL, and Refiner workflow. Using SDXL 1. 1s, load VAE: 0. In ComfyUI this can be accomplished with the output of one KSampler node (using SDXL base) leading directly into the input of another KSampler node (using. 0 ComfyUI. 2. 今天，我们来讲一讲SDXL在comfyui中更加进阶的节点流逻辑。第一、风格控制第二、base模型以及refiner模型如何连接第三、分区提示词控制第四、多重采样的分区控制comfyui节点流程这个东西一通百通，逻辑正确怎么连都可以，所以这个视频我讲得并不仔细，只讲搭建的逻辑和重点，这东西讲太细过于. If this is. 5 refiner node. json: 🦒 Drive. 0 SDXL-refiner-1. 1. but if I run Base model (creating some images with it) without activating that extension or simply forgot to select the Refiner model, and LATER activating it, it gets OOM (out of memory) very much likely when generating images. png . 5 models. The SDXL base model performs significantly better than the previous variants, and the model combined with the refinement module achieves the best overall performance. 9 refiner node. 9 workflow, the one that olivio sarikas video works just fine) just replace the models with 1. 0 and Refiner 1. To download and install ComfyUI using Pinokio, simply go to and download the Pinokio browser. g. This is the image I created using ComfyUI, utilizing Dream ShaperXL 1. 5 refined model) and a switchable face detailer. Setup a quick workflow to do the first part of the denoising process on the base model but instead of finishing it stop early and pass the noisy result on to the refiner to finish the process. ·. Upscale model, (needs to be downloaded into ComfyUImodelsupscale_models Recommended one is 4x-UltraSharp, download from here. com is the number one paste tool since 2002. png files that ppl here post in their SD 1. I think this is the best balanced I could find. 9 Research License. There’s also an install models button. ) [Port 6006]. 0 Base and Refiners models downloaded and saved in the right place, it. Set the base ratio to 1. I also desactivated all extensions & tryed to keep some after, dont. 23:06 How to see ComfyUI is processing the which part of the workflow. It detects hands and improves what is already there. My research organization received access to SDXL. 9 the latest Stable. Im new to ComfyUI and struggling to get an upscale working well. Edit: Got SDXL working well in ComfyUI now, my workflow wasn't set up correctly at first, deleted folder and unzipped the program again and it started with the correct nodes the second time, don't know how or why. It has many extra nodes in order to show comparisons in outputs of different workflows. After inputting your text prompt and choosing the image settings (e. Installing. Skip to content Toggle navigation. Searge-SDXL: EVOLVED v4. 0 ComfyUI Workflow With Nodes Use Of SDXL Base & Refiner Model In this tutorial, join me as we dive into the fascinating world. July 4, 2023. json file to ComfyUI window. They compare the results of Automatic1111 web UI and ComfyUI for SDXL, highlighting the benefits of the former. Source. 20:57 How to use LoRAs with SDXL. Tutorial Video : ComfyUI Master Tutorial - Stable Diffusion XL (SDXL) - Install On PC, Google Colab. Feel free to modify it further if you know how to do it. eilertokyo • 4 mo. 4/1. 9_comfyui_colab (1024x1024 model) please use with: refiner_v0. An SDXL base model in the upper Load Checkpoint node. Two Samplers (base and refiner), and two Save Image Nodes (one for base and one for refiner). A number of Official and Semi-Official “Workflows” for ComfyUI were released during the SDXL 0. 5 and 2. With SDXL, there is the new concept of TEXT_G and TEXT_L with the CLIP Text Encoder. 9: The base model was trained on a variety of aspect ratios on images with resolution 1024^2. 動作が速い. 9 safetensors + LoRA workflow + refiner The text was updated successfully, but these errors were encountered: All reactionsSaved searches Use saved searches to filter your results more quicklyA switch to choose between the SDXL Base+Refiner models and the ReVision model A switch to activate or bypass the Detailer, the Upscaler, or both A (simple) visual prompt builder To configure it, start from the orange section called Control Panel. You can disable this in Notebook settingsYesterday, I came across a very interesting workflow that uses the SDXL base model, any SD 1. NOTE: You will need to use linear (AnimateDiff-SDXL) beta_schedule. Is there an explanation for how to use the refiner in ComfyUI? You can just use someone elses workflow of 0. ComfyUI allows setting up the entire workflow in one go, saving a lot of configuration time compared to using base and. i miss my fast 1. Readme files of the all tutorials are updated for SDXL 1. It supports SD1. IThe sudden interest with ComfyUI due to SDXL release was perhaps too early in its evolution. ใน Tutorial นี้ เพื่อนๆ จะได้เรียนรู้วิธีสร้างภาพ AI แรกของคุณโดยใช้เครื่องมือ Stable Diffusion ComfyUI. ComfyUI seems to work with the stable-diffusion-xl-base-0. First, make sure you are using A1111 version 1. will output this resolution to the bus. 1 Base and Refiner Models to the ComfyUI file. 5s/it, but the Refiner goes up to 30s/it. 5 and send latent to SDXL BaseIt has the SDXL base and refiner sampling nodes along with image upscaling. I trained a LoRA model of myself using the SDXL 1. I just uploaded the new version of my workflow. This is pretty new so there might be better ways to do this, however this works well and we can stack Lora and Lycoris easily, then generate our text prompt at 1024x1024 and allow remacri to double. 5 tiled render. ComfyUI got attention recently because the developer works for StabilityAI and was able to be the first to get SDXL running. 0 (26 July 2023)! Time to test it out using a no-code GUI called ComfyUI!. Andy Lau’s face doesn’t need any fix (Did he??). If you only have a LoRA for the base model you may actually want to skip the refiner or at least use it for fewer steps. That’s because the creator of this workflow has the same 4GB. x for ComfyUI. 15:22 SDXL base image vs refiner improved image comparison. It might come handy as reference. In the second step, we use a. But if SDXL wants a 11-fingered hand, the refiner gives up. 5 of the report on SDXLAlthough SDXL works fine without the refiner (as demonstrated above) you really do need to use the refiner model to get the full use out of the model. In summary, it's crucial to make valid comparisons when evaluating the SDXL with and without the refiner. 17:38 How to use inpainting with SDXL with ComfyUI. 1 for the refiner. Fixed SDXL 0. 9 + refiner (SDXL 0. 0. I can tell you that ComfyUI renders 1024x1024 in SDXL at faster speeds than A1111 does with hiresfix 2x (for SD 1. 5 comfy JSON and import it sd_1-5_to_sdxl_1-0. Stable diffusion-TensorRT安装教程,看完省一张显卡钱！，fooocus完全体2. How do I use the base + refiner in SDXL 1. Here are the configuration settings for the SDXL. See "Refinement Stage" in section 2. But this only increased the resolution and details a bit since it's a very light pass and doesn't change the overall. It now includes: SDXL 1. Img2Img. at least 8GB VRAM is recommended. Check out the ComfyUI guide. 20:57 How to use LoRAs with SDXL. SDXL-refiner-1. I'd like to share Fooocus-MRE (MoonRide Edition), my variant of the original Fooocus (developed by lllyasviel), new UI for SDXL models. Join. Is the best balanced I could find between image size (1024x720), models, steps (10+5 refiner), samplers/schedulers, so we can use SDXL on our laptops without those expensive/bulky desktop GPUs. Click “Manager” in comfyUI, then ‘Install missing custom nodes’. To experiment with it I re-created a workflow with it, similar to my SeargeSDXL workflow. ZIP file. It'll load a basic SDXL workflow that includes a bunch of notes explaining things. SDXL Workflow for ComfyBox - The power of SDXL in ComfyUI with better UI that hides the nodes graph Resource | Update I recently discovered ComfyBox, a UI fontend for ComfyUI. refiner_output_01030_. Before you can use this workflow, you need to have ComfyUI installed. 12:53 How to use SDXL LoRA models with Automatic1111 Web UI. What I am trying to say is do you have enough system RAM. Next support; it's a cool opportunity to learn a different UI anyway. But actually I didn’t heart anything about the training of the refiner. 1. Detailed install instruction can be found here: Link to. 24:47 Where is the ComfyUI support channel. My bet is, that both models beeing loaded at the same time on 8GB VRAM causes this problem. If you look for the missing model you need and download it from there it’ll automatically put. 0 refiner checkpoint; VAE. safetensors and sd_xl_base_0. Other than that, same rules of thumb apply to AnimateDiff-SDXL as AnimateDiff. But, as I ventured further and tried adding the SDXL refiner into the mix, things. 节省大量硬盘空间。. refinerモデルを正式にサポートしている. 0 ComfyUI workflow with a few changes, here's the sample json file for the workflow I was using to generate these images:. 9. Installation. Yesterday I woke up to this Reddit post "Happy Reddit Leak day" by the Joe Penna. You will need ComfyUI and some custom nodes from here and here . Stable Diffusion XL 1. SDXL has 2 text encoders on its base, and a specialty text encoder on its refiner. This checkpoint recommends a VAE, download and place it in the VAE folder. The base doesn't - aesthetic score conditioning tends to break prompt following a bit (the laion aesthetic score values are not the most accurate, and alternative aesthetic scoring methods have limitations of their own), and so the base wasn't trained on it to enable it to follow prompts as accurately as possible. Workflow ComfyUI SDXL 0. 0 with ComfyUI. A EmptyLatentImage specifying the image size consistent with the previous CLIP nodes. 5 comfy JSON and import it sd_1-5_to_sdxl_1-0. utils import load_image pipe = StableDiffusionXLImg2ImgPipeline. Step 1: Download SDXL v1. The workflow should generate images first with the base and then pass them to the refiner for further. Activate your environment. This is the image I created using ComfyUI, utilizing Dream ShaperXL 1. Intelligent Art. I hope someone finds it useful. An SDXL base model in the upper Load Checkpoint node. To simplify the workflow set up a base generation and refiner refinement using two Checkpoint Loaders. thibaud_xl_openpose also. The workflow should generate images first with the base and then pass them to the refiner for further refinement. 9 base & refiner, along with recommended workflows but I ran into trouble. Regenerate faces. 9版本的base model,refiner model. By becoming a member, you'll instantly unlock access to 67 exclusive posts. Refiner: SDXL Refiner 1. 5的对比优劣。. 2. Therefore, it generates thumbnails by decoding them using the SD1. git clone Restart ComfyUI completely. The first advanced KSampler must add noise to the picture, stop at some step and return an image with the leftover noise. Part 2 ( link )- we added SDXL-specific conditioning implementation + tested the impact of conditioning parameters on the generated images. Create a Load Checkpoint node, in that node select the sd_xl_refiner_0. The Tutorial covers:1. go to img2img, choose batch, dropdown refiner, use the folder in 1 as input and the folder in 2 as output. google colab安装comfyUI和sdxl 0. Searge-SDXL: EVOLVED v4. 0. 20:43 How to use SDXL refiner as the base model. import json from urllib import request, parse import random # this is the ComfyUI api prompt format. I've been working with connectors in 3D programs for shader creation, and the sheer (unnecessary) complexity of the networks you could (mistakenly) create for marginal (i. Discover the Ultimate Workflow with ComfyUI in this hands-on tutorial, where I guide you through integrating custom nodes, refining images with advanced tool. Embeddings/Textual Inversion. Two Samplers (base and refiner), and two Save Image Nodes (one for base and one for refiner). 以下のサイトで公開されているrefiner_v1. Try DPM++ 2S a Karras, DPM++ SDE Karras, DPM++ 2M Karras, Euler a and DPM adaptive. Input sources-. All images were created using ComfyUI + SDXL 0. 120 upvotes · 31 comments. Otherwise, I would say make sure everything is updated - if you have custom nodes, they may be out of sync with the base comfyui version. X etc. update ComyUI. download the workflows from the Download button. 0 refiner on the base picture doesn't yield good results. To simplify the workflow set up a base generation and refiner refinement using two Checkpoint Loaders. It works best for realistic generations. Jul 16, 2023. 9 and Stable Diffusion 1. 0, now available via Github. How To Use Stable Diffusion XL 1. The chart above evaluates user preference for SDXL (with and without refinement) over SDXL 0. This produces the image at bottom right. One interesting thing about ComfyUI is that it shows exactly what is happening. 0. ComfyUI LORA. Inpainting. 这一期呢我们来开个新坑，来讲sd的另一种打开方式，也就是这个节点化comfyUI。那熟悉我们频道的老观众都知道，我一直是用webUI去做演示和讲解的. Updating ControlNet. AI Art with ComfyUI and Stable Diffusion SDXL — Day Zero Basics For an Automatic1111 User. I just downloaded the base model and the refiner, but when I try to load the model it can take upward of 2 minutes, and rendering a single image can take 30 minutes, and even then the image looks very very weird. will load images in two ways, 1 direct load from HDD, 2 load from a folder (picks next image when generated) Prediffusion -. Generate SDXL 0. I have updated the workflow submitted last week, cleaning up a bit the layout and adding many functions I wanted to learn better. and have to close terminal and restart a1111 again. I can run SDXL 1024 on comfyui with an 2070/8GB smoother than I could run 1. 0 on ComfyUI. 0 base model. Txt2Img is achieved by passing an empty image to the sampler node with maximum denoise. We name the file “canny-sdxl-1. Play around with different Samplers and different amount of base Steps (30, 60, 90, maybe even higher). Yes, 8Gb card, ComfyUI workflow loads both SDXL base & refiner models, separate XL VAE, 3 XL LoRAs, plus Face Detailer and its sam model and bbox detector model, and Ultimate SD Upscale with its ESRGAN model and input from the same base SDXL model all work together. Hypernetworks. The impact pack doesn't seem to have these nodesThis workflow is meticulously fine tuned to accommodate LORA and Controlnet inputs, and demonstrates interactions with embeddings as well. Question about SDXL ComfyUI and loading LORAs for refiner model. SDXL Base 1. It compromises the individual's DNA, even with just a few sampling steps at the end. What a move forward for the industry. . How to get SDXL running in ComfyUI. SD1. I wanted to see the difference with those along with the refiner pipeline added. Reload ComfyUI. 0 Refiner & The Other SDXL Fp16 Baked VAE. Especially on faces. 5 prompts. Host and manage packages. Step 4: Copy SDXL 0. Both ComfyUI and Foooocus are slower for generation than A1111 - YMMW. json. I wonder if I have been doing it wrong -- right now, when I do latent upscaling with SDXL, I add an Upscale Latent node after the refiner's KSampler node, and pass the result of the latent upscaler to another KSampler. 1 is up, added settings to use model internal VAE and to disable refiner. 0 - Stable Diffusion XL 1. The refiner is although only good at refining noise from an original image still left in creation, and will give you a blurry result if you try to add. 0. 5 and 2. that extension really helps. Starts at 1280x720 and generates 3840x2160 out the other end. Working amazing. 5 + SDXL Base+Refiner is for experiment only. It's down to the devs of AUTO1111 to implement it. Link. 9_comfyui_colab (1024x1024 model) please use with: refiner_v0. 0 or 1. 99 in the “Parameters” section. 0 is “built on an innovative new architecture composed of a 3. 5. Part 4 (this post) - We will install custom nodes and build out workflows. Yet another week and new tools have come out so one must play and experiment with them. What I have done is recreate the parts for one specific area. Images. Commit date (2023-08-11) I was having very poor performance running SDXL locally in ComfyUI to the point where it was basically unusable. 0. You can Load these images in ComfyUI to get the full workflow. 0 with both the base and refiner checkpoints. ) These images are zoomed-in views that I created to examine the details of the upscaling process, showing how much detail. 5. safetensors + sd_xl_refiner_0. The workflow should generate images first with the base and then pass them to the refiner for further. 0. The refiner model works, as the name suggests, a method of refining your images for better quality. SECourses. Custom nodes extension for ComfyUI, including a workflow to use SDXL 1. It is a Latent Diffusion Model that uses a pretrained text encoder ( OpenCLIP-ViT/G ). ComfyUI is a powerful modular graphic interface for Stable Diffusion models that allows you to create complex workflows using nodes. ComfyUI and SDXL. 9" what is the model and where to get it? Reply reply Adventurous-Abies296 After gathering some more knowledge about SDXL and ComfyUI, and experimenting few days with both I've ended up with this basic (no upscaling) 2-stage (base + refiner) workflow: It works pretty well for me - I change dimensions, prompts, and samplers parameters, but the flow itself stays as it is. Not positive, but I do see your refiner sampler has end_at_step set to 10000, and seed to 0. import torch from diffusers import StableDiffusionXLImg2ImgPipeline from diffusers. The workflow I share below is based upon an SDXL using base and refiner models both together to generate the image and then run it through many different custom nodes to showcase the different. Lora. You know what to do. best settings for Stable Diffusion XL 0. The following images can be loaded in ComfyUI to get the full workflow. Today, I upgraded my system to 32GB of RAM and noticed that there were peaks close to 20GB of RAM usage, which could cause memory faults and rendering slowdowns in a 16gb system. 9. Software. safetensors”. Requires sd_xl_base_0. refiner_output_01033_. History: 18 commits. A workflow that can be used on any SDXL model with Base generation, upscale and refiner. Nevertheless, its default settings are comparable to. It fully supports the latest. Start with something simple but that will be obvious that it’s working. safetensors and then sdxl_base_pruned_no-ema. These are what these ports map to in the template we're using: [Port 3000] AUTOMATIC1111's Stable Diffusion Web UI (for generating images) [Port 3010] Kohya SS (for training) [Port 3010] ComfyUI (optional, for generating images. CR Aspect Ratio SDXL replaced by CR SDXL Aspect Ratio ; CR SDXL Prompt Mixer replaced by CR SDXL Prompt Mix Presets Multi-ControlNet methodology . Fine-tuned SDXL (or just the SDXL Base) All images are generated just with the SDXL Base model or a fine-tuned SDXL model that requires no Refiner. SDXL Offset Noise LoRA; Upscaler. 14. @bmc-synth You can use base and/or refiner to further process any kind of image, if you go through img2img (out of latent space) and proper denoising control. Hi buystonehenge, I'm trying to connect the lora stacker to a workflow that includes a normal SDXL checkpoint + a refiner. 5 on A1111 takes 18 seconds to make a 512x768 image and around 25 more seconds to then hirezfix it to 1. When all you need to use this is the files full of encoded text, it's easy to leak. SDXL_1 (right click and save as) workflow has the SDXL setup with refiner with best settings. ComfyUI is a powerful and modular GUI for Stable Diffusion that lets you create advanced workflows using a node/graph interface. Prerequisites. 0—a remarkable breakthrough. Thank you so much Stability AI. A good place to start if you have no idea how any of this works is the: ComfyUI Basic Tutorial VN: All the art is made with ComfyUI. It'll load a basic SDXL workflow that includes a bunch of notes explaining things. I'ma try to get a background fix workflow goin, this blurry shit is starting to bother me. . Sometimes I will update the workflow, all changes will be on the same link. Here's what I've found: When I pair the SDXL base with my LoRA on ComfyUI, things seem to click and work pretty well. sdxl sdxl lora sdxl inpainting comfyui. Searge-SDXL: EVOLVED v4. 9 was yielding already. It provides workflow for SDXL (base + refiner). For me its just very inconsistent. With resolution 1080x720 and specific samplers/schedulers, I managed to get a good balanced and a good image quality, first image with base model not very high. If you have the SDXL 1. 51 denoising. Search for “post processing” and you will find these custom nodes, click on Install and when prompted, close the browser and restart ComfyUI. I know a lot of people prefer Comfy. I think the issue might be the CLIPTextenCode node, you’re using the normal 1. You need to use advanced KSamplers for SDXL. Just training the base model isn't feasible for accurately generating images of subjects such as people, animals, etc. Links and instructions in GitHub readme files updated accordingly. sdxl_v1. Prior to XL, I’ve already had some experience using tiled. 5B parameter base model and a 6. 最後のところに画像が生成されていればOK。. And the refiner files here: stabilityai/stable. 0 Download Upscaler We'll be using. So in this workflow each of them will run on your input image and. SD+XL workflows are variants that can use previous generations. SDXL refiner:. download the SDXL models. To simplify the workflow set up a base generation and refiner refinement using two Checkpoint Loaders. Control-Lora : Official release of a ControlNet style models along with a few other interesting ones. silenf • 2 mo. 9. There are several options on how you can use SDXL model: How to install SDXL 1. 5 + SDXL Refiner Workflow but the beauty of this approach is that these models can be combined in any sequence! You could generate image with SD 1. That's the one I'm referring to. Install SDXL (directory: models/checkpoints) Install a custom SD 1. Upcoming features:Automatic1111’s support for SDXL and the Refiner model is quite rudimentary at present, and until now required that the models be manually switched to perform the second step of image generation. SDXL for A1111 – BASE + Refiner supported!!!! Olivio Sarikas. SDXL Refiner: The refiner model, a new feature of SDXL; SDXL VAE: Optional as there is a VAE baked into the base and refiner model, but nice to have is separate in the workflow so it can be updated/changed without needing a new model. ago. . workflow custom-nodes stable-diffusion comfyui sdxl Updated Nov 13, 2023; Python;. Then refresh the browser (I lie, I just rename every new latent to the same filename e. For some workflow examples and see what ComfyUI can do you can check out: ComfyUI Examples Installing ComfyUI Features. I will provide workflows for models you find on CivitAI and also for SDXL 0. SDXL-OneClick-ComfyUI . I don't want it to get to the point where people are just making models that are designed around looking good at displaying faces. That way you can create and refine the image without having to constantly swap back and forth between models. 0 workflow. A second upscaler has been added.