KBlueLeaf/HDM-xut-340M-anime

Model

139

stars

18

commits

1

repos using this model

2

linked in READMEs

Aug 14, 2025

updated

anime
diffusers
safetensors
text-generation-inference
text-to-image

README

HDM-xut-340M-Anime

World's smallest, cheapest anime-style T2I base

Source Code - HDM Source Code - HDM(ComfyUI) Model - HDM (Here) Document - Tech Report

Introduction

HDM(Home made Diffusion Model) is a project to investigate specialized training recipe/scheme for "pretraining T2I model at home" which require the training setup should be exectuable on customer level hardware or cheap enough second handed server hardware.

Under this constraint, we introduce a new transformer backbone designed for multi-modal (for example, text-to-image) generative model called "XUT" (Cross-U-Transformer). With minimalized arch design with TREAD technique, we can achieve usable performan with computes that cost 650USD at most. (based on pricing from vast.ai)

.gallery-container { position: relative; width: 100%; margin: 0 auto; font-family: Arial, sans-serif; } .gallery-container img{ margin: 0 !important; padding: 0 !important; } /* Hide radio buttons */ .gallery-container input[type="radio"] { display: none; } /* Thumbnail navigation */ .thumbnail-nav { display: flex; align-items: center; gap: 12px; margin-bottom: 20px; padding: 10px; overflow-x: auto; scroll-behavior: smooth; scroll-snap-type: x mandatory; -webkit-overflow-scrolling: touch; scrollbar-width: thin; scrollbar-color: #B398F5 #f0f0f0; } /* Custom scrollbar for webkit browsers */ .thumbnail-nav::-webkit-scrollbar { height: 6px; } .thumbnail-nav::-webkit-scrollbar-track { background: #f0f0f0; border-radius: 3px; } .thumbnail-nav::-webkit-scrollbar-thumb { background: #B398F5; border-radius: 3px; } .thumbnail-nav::-webkit-scrollbar-thumb:hover { background: #4C88F5; } .thumbnail-nav label { width: 100px; height: 120px; border-radius: 8px; background-color: #B398F5; background-size: cover; background-position: center; cursor: pointer; transition: all 0.3s ease; display: block; flex-shrink: 0; scroll-snap-align: center; position: relative; overflow: hidden; padding: 0; } /* Blurred background for thumbnails */ .thumbnail-nav label::before { content: ''; position: absolute; top: 0; left: 0; width: 100%; height: 100%; background-image: inherit; background-size: cover; background-position: center; background-repeat: no-repeat; filter: blur(2px) brightness(25%); transform: scale(1.25); z-index: 1; } /* Fitted image for thumbnails */ .thumbnail-nav label::after { content: ''; position: absolute; top: 50%; left: 50%; transform: translate(-50%, -50%); width: 100%; height: 100%; background-image: inherit; background-size: contain; background-position: center; background-repeat: no-repeat; z-index: 2; } .thumbnail-nav label:hover { border-color: #4C88F5; transform: scale(1.05); } .gallery-container input[type="radio"]:checked + label { border-color: #6296F5; border-width: 4px; transform: scale(1.1); } /* Main content area */ .content-area { position: relative; width: 100%; min-height: 780px; border-radius: 10px; overflow: hidden; box-shadow: 0 4px 15px rgba(0,0,0,0.1); } /* Individual slide containers */ .slide { position: absolute; top: 0; left: 0; width: 100%; height: 100%; opacity: 0; visibility: hidden; transition: opacity 0.5s ease; background: white; } .slide.active { opacity: 1; visibility: visible; } .slide-background-image { width: 100%; height: 480px; object-fit: cover; border-radius: 10px 10px 0 0; filter: blur(3px) brightness(25%); transform: scale(1.1); position: absolute; top: 0; left: 0; z-index: 1; margin: 0 !important; padding: 0 !important; } /* Container for the image area */ .slide-image-container { position: relative; width: 100%; height: 480px; overflow: hidden; border-radius: 10px 10px 0 0; } /* Fitted image overlay */ .slide-fitted-image { position: absolute; top: 50%; left: 50%; transform: translate(-50%, -50%); max-width: 100%; max-height: 100%; object-fit: contain; z-index: 5; box-shadow: 0 2px 10px rgba(0,0,0,0.3); margin: 0 !important; padding: 0 !important; } .slide-content { height: 300px; padding: 20px; flex-grow: 1; background-color: #21203b; overflow-y: scroll; } .slide-title { font-size: 1.5em; font-weight: bold; margin-bottom: 10px; color: #eee; } .slide-description { color: #aaa; /* line-height: 1.6; */ } /* CSS selectors to show active slides */ #slide1:checked ~ .content-area .slide:nth-child(1), #slide2:checked ~ .content-area .slide:nth-child(2), #slide3:checked ~ .content-area .slide:nth-child(3), #slide4:checked ~ .content-area .slide:nth-child(4), #slide5:checked ~ .content-area .slide:nth-child(5), #slide6:checked ~ .content-area .slide:nth-child(6), #slide7:checked ~ .content-area .slide:nth-child(7), #slide8:checked ~ .content-area .slide:nth-child(8), #slide9:checked ~ .content-area .slide:nth-child(9), #slide10:checked ~ .content-area .slide:nth-child(10), #slide11:checked ~ .content-area .slide:nth-child(11), #slide12:checked ~ .content-area .slide:nth-child(12), #slide13:checked ~ .content-area .slide:nth-child(13), #slide14:checked ~ .content-area .slide:nth-child(14), #slide15:checked ~ .content-area .slide:nth-child(15), #slide16:checked ~ .content-area .slide:nth-child(16), #slide17:checked ~ .content-area .slide:nth-child(17), #slide18:checked ~ .content-area .slide:nth-child(18), #slide19:checked ~ .content-area .slide:nth-child(19), #slide20:checked ~ .content-area .slide:nth-child(20), #slide21:checked ~ .content-area .slide:nth-child(21), #slide22:checked ~ .content-area .slide:nth-child(22), #slide23:checked ~ .content-area .slide:nth-child(23), #slide24:checked ~ .content-area .slide:nth-child(24), #slide25:checked ~ .content-area .slide:nth-child(25), #slide26:checked ~ .content-area .slide:nth-child(26), #slide27:checked ~ .content-area .slide:nth-child(27), #slide28:checked ~ .content-area .slide:nth-child(28), #slide29:checked ~ .content-area .slide:nth-child(29), #slide30:checked ~ .content-area .slide:nth-child(30), #slide31:checked ~ .content-area .slide:nth-child(31), #slide32:checked ~ .content-area .slide:nth-child(32), #slide33:checked ~ .content-area .slide:nth-child(33), #slide34:checked ~ .content-area .slide:nth-child(34), #slide35:checked ~ .content-area .slide:nth-child(35), #slide36:checked ~ .content-area .slide:nth-child(36), #slide37:checked ~ .content-area .slide:nth-child(37), #slide38:checked ~ .content-area .slide:nth-child(38), #slide39:checked ~ .content-area .slide:nth-child(39), #slide40:checked ~ .content-area .slide:nth-child(40), #slide41:checked ~ .content-area .slide:nth-child(41), #slide42:checked ~ .content-area .slide:nth-child(42), #slide43:checked ~ .content-area .slide:nth-child(43), #slide44:checked ~ .content-area .slide:nth-child(44), #slide45:checked ~ .content-area .slide:nth-child(45), #slide46:checked ~ .content-area .slide:nth-child(46), #slide47:checked ~ .content-area .slide:nth-child(47), #slide48:checked ~ .content-area .slide:nth-child(48), #slide49:checked ~ .content-area .slide:nth-child(49), #slide50:checked ~ .content-area .slide:nth-child(50), #slide51:checked ~ .content-area .slide:nth-child(51), #slide52:checked ~ .content-area .slide:nth-child(52), #slide53:checked ~ .content-area .slide:nth-child(53), #slide54:checked ~ .content-area .slide:nth-child(54), #slide55:checked ~ .content-area .slide:nth-child(55), #slide56:checked ~ .content-area .slide:nth-child(56), #slide57:checked ~ .content-area .slide:nth-child(57), #slide58:checked ~ .content-area .slide:nth-child(58), #slide59:checked ~ .content-area .slide:nth-child(59), #slide60:checked ~ .content-area .slide:nth-child(60), #slide61:checked ~ .content-area .slide:nth-child(61), #slide62:checked ~ .content-area .slide:nth-child(62), #slide63:checked ~ .content-area .slide:nth-child(63), #slide64:checked ~ .content-area .slide:nth-child(64), #slide65:checked ~ .content-area .slide:nth-child(65) { opacity: 1; visibility: visible; } /* Thumbnail background images */ label[for="slide1"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02407_.png'); } label[for="slide2"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02783_.png'); } label[for="slide3"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-09-192937_913639157822666_01_02_03_04.webp'); } label[for="slide4"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02756_.png'); } label[for="slide5"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-09-225850_77551003993312_01.webp'); } label[for="slide6"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-09-171431_303199109673130_01_02_03.webp'); } label[for="slide7"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/D983E2645DF87124DD28E799AB0CB075.jpeg'); } label[for="slide8"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_03061_.png'); } label[for="slide9"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_00493_.png'); } label[for="slide10"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-12-191724_162583156957958_01_02_03.webp'); } label[for="slide11"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-11-150759_753488460583170_01_02_03.webp'); } label[for="slide12"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_00552_.png'); } label[for="slide13"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-12-190118_83399343549266_01.webp'); } label[for="slide14"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-12-213512_1006466452888165_01_02_03.webp'); } label[for="slide15"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-11-144417_710459885578623_01_02.webp'); } label[for="slide16"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-09-160528_73508193969950.png'); } label[for="slide17"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02967_.png'); } label[for="slide18"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02989_.png'); } label[for="slide19"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02702_.png'); } label[for="slide20"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-09-164935_684249266742885_01.webp'); } label[for="slide21"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02734_.png'); } label[for="slide22"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_00521_.png'); } label[for="slide23"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-12-234212_277890920871419_01_02_03_04.webp'); } label[for="slide24"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_00519_.png'); } label[for="slide25"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_03076_.png'); } label[for="slide26"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02957_.png'); } label[for="slide27"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_00475_.png'); } label[for="slide28"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_00560_.png'); } label[for="slide29"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-10-000104_1031686041350569_01_02.webp'); } label[for="slide30"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02870_.png'); } label[for="slide31"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-12-163418_1010047888126740_01_02_03.webp'); } label[for="slide32"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-12-225108_208730707976133_01_02_03.webp'); } label[for="slide33"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/1070541.png'); } label[for="slide34"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_03122_.png'); } label[for="slide35"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-09-233724_944013876753649_01_02.webp'); } label[for="slide36"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-09-174043_494986375547008_01_02_03.webp'); } label[for="slide37"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/1071387.png'); } label[for="slide38"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-11-152936_1122802635229146_01_02(1).webp'); } label[for="slide39"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/1076892.png'); } label[for="slide40"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02800_.png'); } label[for="slide41"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-10-004804_291795535432825_01_02_03.webp'); } label[for="slide42"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-12-224956_293718624327641_01_02.webp'); } label[for="slide43"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_00492_.png'); } label[for="slide44"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02882_.png'); } label[for="slide45"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-09-232028_370116235876126_01.webp'); } label[for="slide46"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-10-013423_565463797077543_01_02.webp'); } label[for="slide47"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-09-183119_869967301831594_01_02_03.webp'); } label[for="slide48"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02944_.png'); } label[for="slide49"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-12-220404_87459800149096_01_02.webp'); } label[for="slide50"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02726_.png'); } label[for="slide51"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-12-214635_1051482950769510_01.webp'); } label[for="slide52"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02913_.png'); } label[for="slide53"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-12-205145_960573175128065_01_02_03_04.webp'); } label[for="slide54"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-09-235803_545758812501707_01_02_03.webp'); } label[for="slide55"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02692_.png'); } label[for="slide56"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02888_.png'); } label[for="slide57"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02554_.png'); } label[for="slide58"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/14B2B2490B686F1F1419AF5236A3C6EC.png'); } label[for="slide59"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/1072811.png'); } label[for="slide60"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/1072664.png'); } label[for="slide61"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-12-201122_734221260669565_01_02_03_04.webp'); } label[for="slide62"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02880_.png'); } label[for="slide63"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02997_.png'); } label[for="slide64"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02400_.png'); } label[for="slide65"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02793_.png'); } /* Responsive design */ @media (max-width: 600px) { .thumbnail-nav { gap: 8px; padding: 8px; } .thumbnail-nav label { width: 80px; height: 60px; } .slide img { height: 200px; } .slide-content { padding: 15px; } .slide-title { font-size: 1.2em; } }

Quick Start

Read [Usage Hint] section after install the model

READ IT READ IT READ IT

  • Use official gradio ui: https://github.com/KohakuBlueleaf/HDM
    • Follow the readme in this repository
  • Use comfyui loader node: https://github.com/KohakuBlueleaf/HDM-ext
    • Install this repository as comfyui custom node
    • Use the HDM-loader node with provided safetensors file
  • Use diffusers pipeline:
    1. Install hdm library from github repo
      git clone https://github.com/KohakuBlueleaf/HDM
      cd HDM
      # fused: xformers fused swiglu/attention
      # liger: liger-kernel fused swiglu
      # tipo : tipo prompt gen system for official UI
      # win  : windows specific (all windows user should add this)
      # finetune: install lycoris-lora for finetune with lycoris
      pip install -e .[tipo]
      ## For windows user: pip install -e .[win,tipo]
      
    2. Than use pipeline from hdm
      import torch
      from hdm.pipeline import HDMXUTPipeline
      
      torch.set_float32_matmul_precision("high")
      pipeline = (
          HDMXUTPipeline.from_pretrained(
              "KBlueLeaf/HDM-xut-340M-anime",
              trust_remote_code=True
          )
          .to("cuda")
          .to(torch.float16)
      )
      images = pipeline("1girl....", "").images
      

Usage Hint

  • Prompting:
    • Use Danbooru tag + Natural Language Description
    • For "tags", use "danbooru tag" ONLY.
    • For tags, use comma seperated tag sequence, "without underscore"
      • "1girl long_hair" -> "1girl, long hair"
    • Make prompt as detail as possible
      • TIPO is highly recommended to use here!!!
  • Sampling settings:
    • CFG scale: 2~4
    • Inference steps: at least 8, works best in 16~32
  • For 1024px model, recommend following resolution (WxH):
    • 1024xK where K >= 1024
    • WxH where W > H, W*H >= 1024*1024, H <= 1024
    • For example: 1024x1536(1:2), 1440x800(16:9), 1472x736(2:1), 1280x960(4:3)
  • Don't know where to start? New to danbooru-tag style prompting?
    • Check danbooru.donmai.us, find any image you feel good, copy its tag!
    • Remember to apply the transformation mentioned above.

Example prompt format

1girl, 
izuna (blue archive), blue archive, 

fukamiki kei, tail, halo, skin fang, solo, eyeshadow, medium breasts, short hair, skirt, makeup, hair ornament, pink halo, yellow eyes, fox tail, scarf, pom pom hair ornament, fox girl, river, pleated skirt, leaning forward, animal ears, fang, black hair, black skirt, one side up, nature, pink scarf, sleeveless, miniskirt, open mouth, looking at viewer, fox hair ornament, breasts, animal ear fluff, fox ears, outdoors, wading, pink eyeshadow, forest, pom pom (clothes), smile, waterfall, sunlight, white shirt, sleeveless shirt, sailor collar, shirt, tail raised, partially submerged, blush, water, cowboy shot, blue sailor collar, :d,

This image is a digital illustration by the artist masabodo, known for their detailed and vibrant style. The central character in the artwork is Izuna from the series "Blue Archive," depicted with fox-like features including pointed ears and a fluffy tail. Izuna is portrayed with her characteristic black hair adorned with pink ribbons, wearing a sailor-style outfit that includes a blue collar and skirt. The overall scene exudes a sense of tranquility and grace, capturing a moment of serene beauty in nature.

masterpiece, newest, absurdres

Or with placeholder:

<|special|>, 
<|characters|>, <|series|>, 
<|artist|>, 

<|general content tags|>,

<|natural language|>.

<|quality|>, <|rating|>, <|meta|>

NOTE: In ComfyUI, you SHOULD use backslash for ALL the bracket. for example izuna (blue archive) -> izuna \(blue archive\). To ensure TIPO to works correctly and the bracket will be sent to the model.

Model Spec

HDM-XUT overview

XUT arch details

Model Setup

  • 343M XUT diffusion
  • 596M Qwen3 Text Encoder (qwen3-0.6B)
  • EQ-SDXL-VAE
  • Support 1024x1024 or higher resolution
    • 512px/768px checkpoints provided
  • Sampling method/Training Objective: Flow Matching
  • Inference Steps: 16~32
  • Hardware Recommendations: any Nvidia GPU with tensor core and >=6GB vram
  • Minimal Requirements: x86-64 computer with more than 16GB ram
    • 512 and 768px can achieve reasonable speed on CPU

Dataset

  • Danbooru2023 (around 7.6M images, latest id around 8.2M)
  • Curated Pixiv set (around 700k images)
  • Private pvc figure photos. (less than 50k images)

Training

Stage256²512²768²1024²
DatasetDanbooru 2023Danbooru2023 + extra*-curated**
Image Count7.624M8.469M-3.649M
Epochs20511
Samples Seen152.5M42.34M8.469M3.649M
Patches Seen39B43.5B19.5B15B
Learning Rate (muP, base_dim=1)0.50.10.050.02
Batch Size (per GPU)128646416
Gradient CheckpointingNoYesYesYes
Gradient Accumulation4224
Global Batch Size2048512512256
TREAD Selection Rate0.50.50.50.0
Context Length256256256512
Training Wall Time174h120h42h49h

Limitations

  • Due to constrained training budget and model size:
    • This model require user to provide highly detailed and well structured prompt to works well.
      • Users are highly recommended to use TIPO
    • This model are not good at generate good detail structure or not well-annotated concept (such as hands or special ornaments)
  • This model can only generate anime-style images for now due to dataset choice.
    • A more general T2I model is under training, stay tuned.
  • This model are capable to generate lot of different anime-like style but it's hard to trigger specific art style stably. It is recommend to try different combination of content, meta, artist tag to achieve different art style.

License

This project is still under developement, therefore all the models, source code, text, documents or any media in this project are licensed under CC-BY-NC-SA 4.0 until the finish of development.

For any usage that may require any kind of standalone, specialized license. Please directly contact kohaku@kblueleaf.net

Contributors

KBlueLeaf

18 commits

KBlueLeaf/HDM-xut-340M-anime

Model

139

stars

18

commits

1

repos using this model

2

linked in READMEs

Aug 14, 2025

updated

anime
diffusers
safetensors
text-generation-inference
text-to-image

README

HDM-xut-340M-Anime

World's smallest, cheapest anime-style T2I base

Source Code - HDM Source Code - HDM(ComfyUI) Model - HDM (Here) Document - Tech Report

Introduction

HDM(Home made Diffusion Model) is a project to investigate specialized training recipe/scheme for "pretraining T2I model at home" which require the training setup should be exectuable on customer level hardware or cheap enough second handed server hardware.

Under this constraint, we introduce a new transformer backbone designed for multi-modal (for example, text-to-image) generative model called "XUT" (Cross-U-Transformer). With minimalized arch design with TREAD technique, we can achieve usable performan with computes that cost 650USD at most. (based on pricing from vast.ai)

.gallery-container { position: relative; width: 100%; margin: 0 auto; font-family: Arial, sans-serif; } .gallery-container img{ margin: 0 !important; padding: 0 !important; } /* Hide radio buttons */ .gallery-container input[type="radio"] { display: none; } /* Thumbnail navigation */ .thumbnail-nav { display: flex; align-items: center; gap: 12px; margin-bottom: 20px; padding: 10px; overflow-x: auto; scroll-behavior: smooth; scroll-snap-type: x mandatory; -webkit-overflow-scrolling: touch; scrollbar-width: thin; scrollbar-color: #B398F5 #f0f0f0; } /* Custom scrollbar for webkit browsers */ .thumbnail-nav::-webkit-scrollbar { height: 6px; } .thumbnail-nav::-webkit-scrollbar-track { background: #f0f0f0; border-radius: 3px; } .thumbnail-nav::-webkit-scrollbar-thumb { background: #B398F5; border-radius: 3px; } .thumbnail-nav::-webkit-scrollbar-thumb:hover { background: #4C88F5; } .thumbnail-nav label { width: 100px; height: 120px; border-radius: 8px; background-color: #B398F5; background-size: cover; background-position: center; cursor: pointer; transition: all 0.3s ease; display: block; flex-shrink: 0; scroll-snap-align: center; position: relative; overflow: hidden; padding: 0; } /* Blurred background for thumbnails */ .thumbnail-nav label::before { content: ''; position: absolute; top: 0; left: 0; width: 100%; height: 100%; background-image: inherit; background-size: cover; background-position: center; background-repeat: no-repeat; filter: blur(2px) brightness(25%); transform: scale(1.25); z-index: 1; } /* Fitted image for thumbnails */ .thumbnail-nav label::after { content: ''; position: absolute; top: 50%; left: 50%; transform: translate(-50%, -50%); width: 100%; height: 100%; background-image: inherit; background-size: contain; background-position: center; background-repeat: no-repeat; z-index: 2; } .thumbnail-nav label:hover { border-color: #4C88F5; transform: scale(1.05); } .gallery-container input[type="radio"]:checked + label { border-color: #6296F5; border-width: 4px; transform: scale(1.1); } /* Main content area */ .content-area { position: relative; width: 100%; min-height: 780px; border-radius: 10px; overflow: hidden; box-shadow: 0 4px 15px rgba(0,0,0,0.1); } /* Individual slide containers */ .slide { position: absolute; top: 0; left: 0; width: 100%; height: 100%; opacity: 0; visibility: hidden; transition: opacity 0.5s ease; background: white; } .slide.active { opacity: 1; visibility: visible; } .slide-background-image { width: 100%; height: 480px; object-fit: cover; border-radius: 10px 10px 0 0; filter: blur(3px) brightness(25%); transform: scale(1.1); position: absolute; top: 0; left: 0; z-index: 1; margin: 0 !important; padding: 0 !important; } /* Container for the image area */ .slide-image-container { position: relative; width: 100%; height: 480px; overflow: hidden; border-radius: 10px 10px 0 0; } /* Fitted image overlay */ .slide-fitted-image { position: absolute; top: 50%; left: 50%; transform: translate(-50%, -50%); max-width: 100%; max-height: 100%; object-fit: contain; z-index: 5; box-shadow: 0 2px 10px rgba(0,0,0,0.3); margin: 0 !important; padding: 0 !important; } .slide-content { height: 300px; padding: 20px; flex-grow: 1; background-color: #21203b; overflow-y: scroll; } .slide-title { font-size: 1.5em; font-weight: bold; margin-bottom: 10px; color: #eee; } .slide-description { color: #aaa; /* line-height: 1.6; */ } /* CSS selectors to show active slides */ #slide1:checked ~ .content-area .slide:nth-child(1), #slide2:checked ~ .content-area .slide:nth-child(2), #slide3:checked ~ .content-area .slide:nth-child(3), #slide4:checked ~ .content-area .slide:nth-child(4), #slide5:checked ~ .content-area .slide:nth-child(5), #slide6:checked ~ .content-area .slide:nth-child(6), #slide7:checked ~ .content-area .slide:nth-child(7), #slide8:checked ~ .content-area .slide:nth-child(8), #slide9:checked ~ .content-area .slide:nth-child(9), #slide10:checked ~ .content-area .slide:nth-child(10), #slide11:checked ~ .content-area .slide:nth-child(11), #slide12:checked ~ .content-area .slide:nth-child(12), #slide13:checked ~ .content-area .slide:nth-child(13), #slide14:checked ~ .content-area .slide:nth-child(14), #slide15:checked ~ .content-area .slide:nth-child(15), #slide16:checked ~ .content-area .slide:nth-child(16), #slide17:checked ~ .content-area .slide:nth-child(17), #slide18:checked ~ .content-area .slide:nth-child(18), #slide19:checked ~ .content-area .slide:nth-child(19), #slide20:checked ~ .content-area .slide:nth-child(20), #slide21:checked ~ .content-area .slide:nth-child(21), #slide22:checked ~ .content-area .slide:nth-child(22), #slide23:checked ~ .content-area .slide:nth-child(23), #slide24:checked ~ .content-area .slide:nth-child(24), #slide25:checked ~ .content-area .slide:nth-child(25), #slide26:checked ~ .content-area .slide:nth-child(26), #slide27:checked ~ .content-area .slide:nth-child(27), #slide28:checked ~ .content-area .slide:nth-child(28), #slide29:checked ~ .content-area .slide:nth-child(29), #slide30:checked ~ .content-area .slide:nth-child(30), #slide31:checked ~ .content-area .slide:nth-child(31), #slide32:checked ~ .content-area .slide:nth-child(32), #slide33:checked ~ .content-area .slide:nth-child(33), #slide34:checked ~ .content-area .slide:nth-child(34), #slide35:checked ~ .content-area .slide:nth-child(35), #slide36:checked ~ .content-area .slide:nth-child(36), #slide37:checked ~ .content-area .slide:nth-child(37), #slide38:checked ~ .content-area .slide:nth-child(38), #slide39:checked ~ .content-area .slide:nth-child(39), #slide40:checked ~ .content-area .slide:nth-child(40), #slide41:checked ~ .content-area .slide:nth-child(41), #slide42:checked ~ .content-area .slide:nth-child(42), #slide43:checked ~ .content-area .slide:nth-child(43), #slide44:checked ~ .content-area .slide:nth-child(44), #slide45:checked ~ .content-area .slide:nth-child(45), #slide46:checked ~ .content-area .slide:nth-child(46), #slide47:checked ~ .content-area .slide:nth-child(47), #slide48:checked ~ .content-area .slide:nth-child(48), #slide49:checked ~ .content-area .slide:nth-child(49), #slide50:checked ~ .content-area .slide:nth-child(50), #slide51:checked ~ .content-area .slide:nth-child(51), #slide52:checked ~ .content-area .slide:nth-child(52), #slide53:checked ~ .content-area .slide:nth-child(53), #slide54:checked ~ .content-area .slide:nth-child(54), #slide55:checked ~ .content-area .slide:nth-child(55), #slide56:checked ~ .content-area .slide:nth-child(56), #slide57:checked ~ .content-area .slide:nth-child(57), #slide58:checked ~ .content-area .slide:nth-child(58), #slide59:checked ~ .content-area .slide:nth-child(59), #slide60:checked ~ .content-area .slide:nth-child(60), #slide61:checked ~ .content-area .slide:nth-child(61), #slide62:checked ~ .content-area .slide:nth-child(62), #slide63:checked ~ .content-area .slide:nth-child(63), #slide64:checked ~ .content-area .slide:nth-child(64), #slide65:checked ~ .content-area .slide:nth-child(65) { opacity: 1; visibility: visible; } /* Thumbnail background images */ label[for="slide1"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02407_.png'); } label[for="slide2"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02783_.png'); } label[for="slide3"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-09-192937_913639157822666_01_02_03_04.webp'); } label[for="slide4"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02756_.png'); } label[for="slide5"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-09-225850_77551003993312_01.webp'); } label[for="slide6"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-09-171431_303199109673130_01_02_03.webp'); } label[for="slide7"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/D983E2645DF87124DD28E799AB0CB075.jpeg'); } label[for="slide8"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_03061_.png'); } label[for="slide9"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_00493_.png'); } label[for="slide10"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-12-191724_162583156957958_01_02_03.webp'); } label[for="slide11"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-11-150759_753488460583170_01_02_03.webp'); } label[for="slide12"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_00552_.png'); } label[for="slide13"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-12-190118_83399343549266_01.webp'); } label[for="slide14"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-12-213512_1006466452888165_01_02_03.webp'); } label[for="slide15"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-11-144417_710459885578623_01_02.webp'); } label[for="slide16"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-09-160528_73508193969950.png'); } label[for="slide17"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02967_.png'); } label[for="slide18"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02989_.png'); } label[for="slide19"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02702_.png'); } label[for="slide20"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-09-164935_684249266742885_01.webp'); } label[for="slide21"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02734_.png'); } label[for="slide22"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_00521_.png'); } label[for="slide23"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-12-234212_277890920871419_01_02_03_04.webp'); } label[for="slide24"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_00519_.png'); } label[for="slide25"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_03076_.png'); } label[for="slide26"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02957_.png'); } label[for="slide27"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_00475_.png'); } label[for="slide28"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_00560_.png'); } label[for="slide29"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-10-000104_1031686041350569_01_02.webp'); } label[for="slide30"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02870_.png'); } label[for="slide31"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-12-163418_1010047888126740_01_02_03.webp'); } label[for="slide32"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-12-225108_208730707976133_01_02_03.webp'); } label[for="slide33"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/1070541.png'); } label[for="slide34"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_03122_.png'); } label[for="slide35"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-09-233724_944013876753649_01_02.webp'); } label[for="slide36"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-09-174043_494986375547008_01_02_03.webp'); } label[for="slide37"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/1071387.png'); } label[for="slide38"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-11-152936_1122802635229146_01_02(1).webp'); } label[for="slide39"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/1076892.png'); } label[for="slide40"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02800_.png'); } label[for="slide41"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-10-004804_291795535432825_01_02_03.webp'); } label[for="slide42"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-12-224956_293718624327641_01_02.webp'); } label[for="slide43"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_00492_.png'); } label[for="slide44"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02882_.png'); } label[for="slide45"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-09-232028_370116235876126_01.webp'); } label[for="slide46"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-10-013423_565463797077543_01_02.webp'); } label[for="slide47"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-09-183119_869967301831594_01_02_03.webp'); } label[for="slide48"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02944_.png'); } label[for="slide49"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-12-220404_87459800149096_01_02.webp'); } label[for="slide50"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02726_.png'); } label[for="slide51"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-12-214635_1051482950769510_01.webp'); } label[for="slide52"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02913_.png'); } label[for="slide53"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-12-205145_960573175128065_01_02_03_04.webp'); } label[for="slide54"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-09-235803_545758812501707_01_02_03.webp'); } label[for="slide55"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02692_.png'); } label[for="slide56"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02888_.png'); } label[for="slide57"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02554_.png'); } label[for="slide58"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/14B2B2490B686F1F1419AF5236A3C6EC.png'); } label[for="slide59"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/1072811.png'); } label[for="slide60"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/1072664.png'); } label[for="slide61"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/hdm-1024px-2025-08-12-201122_734221260669565_01_02_03_04.webp'); } label[for="slide62"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02880_.png'); } label[for="slide63"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02997_.png'); } label[for="slide64"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02400_.png'); } label[for="slide65"] { background-image: url('https://huggingface.co/KBlueLeaf/HDM-xut-340M-anime/resolve/main/images/samples/ComfyUI_02793_.png'); } /* Responsive design */ @media (max-width: 600px) { .thumbnail-nav { gap: 8px; padding: 8px; } .thumbnail-nav label { width: 80px; height: 60px; } .slide img { height: 200px; } .slide-content { padding: 15px; } .slide-title { font-size: 1.2em; } }

Quick Start

Read [Usage Hint] section after install the model

READ IT READ IT READ IT

  • Use official gradio ui: https://github.com/KohakuBlueleaf/HDM
    • Follow the readme in this repository
  • Use comfyui loader node: https://github.com/KohakuBlueleaf/HDM-ext
    • Install this repository as comfyui custom node
    • Use the HDM-loader node with provided safetensors file
  • Use diffusers pipeline:
    1. Install hdm library from github repo
      git clone https://github.com/KohakuBlueleaf/HDM
      cd HDM
      # fused: xformers fused swiglu/attention
      # liger: liger-kernel fused swiglu
      # tipo : tipo prompt gen system for official UI
      # win  : windows specific (all windows user should add this)
      # finetune: install lycoris-lora for finetune with lycoris
      pip install -e .[tipo]
      ## For windows user: pip install -e .[win,tipo]
      
    2. Than use pipeline from hdm
      import torch
      from hdm.pipeline import HDMXUTPipeline
      
      torch.set_float32_matmul_precision("high")
      pipeline = (
          HDMXUTPipeline.from_pretrained(
              "KBlueLeaf/HDM-xut-340M-anime",
              trust_remote_code=True
          )
          .to("cuda")
          .to(torch.float16)
      )
      images = pipeline("1girl....", "").images
      

Usage Hint

  • Prompting:
    • Use Danbooru tag + Natural Language Description
    • For "tags", use "danbooru tag" ONLY.
    • For tags, use comma seperated tag sequence, "without underscore"
      • "1girl long_hair" -> "1girl, long hair"
    • Make prompt as detail as possible
      • TIPO is highly recommended to use here!!!
  • Sampling settings:
    • CFG scale: 2~4
    • Inference steps: at least 8, works best in 16~32
  • For 1024px model, recommend following resolution (WxH):
    • 1024xK where K >= 1024
    • WxH where W > H, W*H >= 1024*1024, H <= 1024
    • For example: 1024x1536(1:2), 1440x800(16:9), 1472x736(2:1), 1280x960(4:3)
  • Don't know where to start? New to danbooru-tag style prompting?
    • Check danbooru.donmai.us, find any image you feel good, copy its tag!
    • Remember to apply the transformation mentioned above.

Example prompt format

1girl, 
izuna (blue archive), blue archive, 

fukamiki kei, tail, halo, skin fang, solo, eyeshadow, medium breasts, short hair, skirt, makeup, hair ornament, pink halo, yellow eyes, fox tail, scarf, pom pom hair ornament, fox girl, river, pleated skirt, leaning forward, animal ears, fang, black hair, black skirt, one side up, nature, pink scarf, sleeveless, miniskirt, open mouth, looking at viewer, fox hair ornament, breasts, animal ear fluff, fox ears, outdoors, wading, pink eyeshadow, forest, pom pom (clothes), smile, waterfall, sunlight, white shirt, sleeveless shirt, sailor collar, shirt, tail raised, partially submerged, blush, water, cowboy shot, blue sailor collar, :d,

This image is a digital illustration by the artist masabodo, known for their detailed and vibrant style. The central character in the artwork is Izuna from the series "Blue Archive," depicted with fox-like features including pointed ears and a fluffy tail. Izuna is portrayed with her characteristic black hair adorned with pink ribbons, wearing a sailor-style outfit that includes a blue collar and skirt. The overall scene exudes a sense of tranquility and grace, capturing a moment of serene beauty in nature.

masterpiece, newest, absurdres

Or with placeholder:

<|special|>, 
<|characters|>, <|series|>, 
<|artist|>, 

<|general content tags|>,

<|natural language|>.

<|quality|>, <|rating|>, <|meta|>

NOTE: In ComfyUI, you SHOULD use backslash for ALL the bracket. for example izuna (blue archive) -> izuna \(blue archive\). To ensure TIPO to works correctly and the bracket will be sent to the model.

Model Spec

HDM-XUT overview

XUT arch details

Model Setup

  • 343M XUT diffusion
  • 596M Qwen3 Text Encoder (qwen3-0.6B)
  • EQ-SDXL-VAE
  • Support 1024x1024 or higher resolution
    • 512px/768px checkpoints provided
  • Sampling method/Training Objective: Flow Matching
  • Inference Steps: 16~32
  • Hardware Recommendations: any Nvidia GPU with tensor core and >=6GB vram
  • Minimal Requirements: x86-64 computer with more than 16GB ram
    • 512 and 768px can achieve reasonable speed on CPU

Dataset

  • Danbooru2023 (around 7.6M images, latest id around 8.2M)
  • Curated Pixiv set (around 700k images)
  • Private pvc figure photos. (less than 50k images)

Training

Stage256²512²768²1024²
DatasetDanbooru 2023Danbooru2023 + extra*-curated**
Image Count7.624M8.469M-3.649M
Epochs20511
Samples Seen152.5M42.34M8.469M3.649M
Patches Seen39B43.5B19.5B15B
Learning Rate (muP, base_dim=1)0.50.10.050.02
Batch Size (per GPU)128646416
Gradient CheckpointingNoYesYesYes
Gradient Accumulation4224
Global Batch Size2048512512256
TREAD Selection Rate0.50.50.50.0
Context Length256256256512
Training Wall Time174h120h42h49h

Limitations

  • Due to constrained training budget and model size:
    • This model require user to provide highly detailed and well structured prompt to works well.
      • Users are highly recommended to use TIPO
    • This model are not good at generate good detail structure or not well-annotated concept (such as hands or special ornaments)
  • This model can only generate anime-style images for now due to dataset choice.
    • A more general T2I model is under training, stay tuned.
  • This model are capable to generate lot of different anime-like style but it's hard to trigger specific art style stably. It is recommend to try different combination of content, meta, artist tag to achieve different art style.

License

This project is still under developement, therefore all the models, source code, text, documents or any media in this project are licensed under CC-BY-NC-SA 4.0 until the finish of development.

For any usage that may require any kind of standalone, specialized license. Please directly contact kohaku@kblueleaf.net

Contributors

KBlueLeaf

18 commits