RORem-mixed GGUF (Q4_K)

This repository contains a Q4_K GGUF conversion of the RORem-mixed SDXL inpainting UNet for stable-diffusion.cpp.

RORem is an object-removal/inpainting model from RORem: Training a Robust Object Remover with Human-in-the-Loop. The mixed-resolution checkpoint was trained for both 512x512 and 1024x1024 inputs.

Files

This is not a complete SDXL pipeline. It does not include the VAE, CLIP-L, or CLIP-G weights. Use those components from diffusers/stable-diffusion-xl-1.0-inpainting-0.1.

stable-diffusion.cpp usage

The model was converted and tested with the official CUDA prebuilt from stable-diffusion.cpp release master-782-b290693.

sd-cli \
  --model sdxl-version-marker.safetensors \
  --diffusion-model rorem-mixed-unet-q4_K.gguf \
  --vae /path/to/sdxl-inpainting/vae/diffusion_pytorch_model.fp16.safetensors \
  --clip_l /path/to/sdxl-inpainting/text_encoder/model.fp16.safetensors \
  --clip_g /path/to/sdxl-inpainting/text_encoder_2/model.fp16.safetensors \
  --init-img input.png \
  --mask mask.png \
  --prompt "clean manga illustration, crisp black line art, flat colors, seamless original background, clean white speech bubble, no text" \
  --negative-prompt "text, letters, words, symbols, watermark, signature, blurry, smudged, dirty, gray artifacts, extra objects, photorealistic" \
  --width 512 \
  --height 512 \
  --steps 30 \
  --cfg-scale 8 \
  --strength 0.999 \
  --sampling-method euler \
  --scheduler discrete \
  --fa \
  --output output.png

The white portion of the mask is regenerated. For manga translation, dilating the text or SFX mask before inference helps remove outlines and antialiasing remnants.

Local benchmark

Single 512x512 manga inpainting crop, 30 Euler steps, fixed seed, RTX 5090, stable-diffusion.cpp CUDA build b290693:

This is one local sample, not a comprehensive quality benchmark. Q4_K was effectively tied with Q4_0 in speed while producing an output closer to the FP16 reference.

Limitations

• RORem can hallucinate texture, line art, or objects inside large masks.
• Output quality depends strongly on mask dilation, crop context, resolution, prompt, and seed.
• This checkpoint is intended for inpainting/object removal, not general text-to-image generation.
• The quantized model may differ from FP16 output.

Attribution and licenses

• RORem project and original checkpoint: leeruibin/RORem, published by its authors under Apache 2.0.
• SDXL inpainting base: diffusers/stable-diffusion-xl-1.0-inpainting-0.1, licensed under CreativeML Open RAIL++-M.
• Inference runtime: stable-diffusion.cpp.

Users are responsible for complying with all applicable upstream licenses and usage restrictions.

Citation

@article{li2024RORem,
  title={RORem: Training a Robust Object Remover with Human-in-the-Loop},
  author={Ruibin Li and Tao, Yang and Song, Guo and Lei, Zhang},
  year={2025},
  booktitle={IEEE/CVF Conference on Computer Vision and Pattern Recognition}
}