Focusing the Diffusion: Importance Sampling for One-Shot Subject-Driven Image Generation

IEEE Access. 2026. No. 14. P. 26662–26671.

Mamedov I., Fofanov M., Lukianchenko P., Лисов Р. В., Teplov A., Barkina A., Minets M.

DreamBooth-style personalization, which often relies on one or a few images, can memorize context (background, pose, lighting) rather than learning the subject’s identity, thereby reducing fidelity and limiting prompt-driven edits. P-LoRA (Probability LoRA) is introduced as a training-time modification that biases diffusion timestep sampling and utilizes importance weights, ensuring that the overall objective remains unchanged. The key idea is to sample low-noise timesteps more often, where identity details are most visible, encouraging the model to learn subject-specific features rather than contextual cues. Across one-shot and few-shot settings, a moderate low-noise bias improves subject reconstruction fidelity while keeping editability comparable to DreamBooth. In contrast, biasing toward higher-noise timesteps reduces background memorization but consistently lowers subject fidelity and weakens prompt adherence, highlighting a trade-off between identity retention and controllability. To better measure identity learning without context leakage, a foreground-only reconstruction score is proposed that segments the subject before computing CLIP similarity. Overall, timestep reweighting provides a simple lever for navigating fidelity–editability trade-offs under extreme data scarcity.

Keywords: Standards, Visualization, pipelines, Noise, image reconstruction, Training, Adaptation models, overfitting, Monte Carlo methods, Diffusion Models, image personalization, importance sampling, low-rank adaptation