Ziwei Luo - Half-time seminar: "Diffusion-based Generative Modelling with Applications in Image Restoration"

Discussion leader: Atsuto Maki

Abstract: Diffusion models have achieved significant progress in generative modelling, in particular when it comes to producing images that align with human preferences. In this half-time seminar, we explore how to apply diffusion models to image restoration, which requires reconstructing high-quality (HQ) images from their degraded low-quality (LQ) counterparts such as image deblurring, deraining, inpainting, etc.

To involve the LQ image as a condition, we construct a mean-reverting stochastic differential equation that transforms the HQ image into a noisy version of the LQ image. By simulating the corresponding reverse-time stochastic differential equation, high-quality images can be restored. We also introduce a maximum likelihood objective to learn an optimal reverse trajectory which effectively improves the restoration results. Additionally, we take inspiration from modern vision language models and integrate CLIP embeddings into the diffusion network, enabling a single diffusion model to handle different restoration tasks. The seminar concludes with a discussion of existing challenges and our future directions.

Welcome!