Centre for Networked Intelligence

Dear All, Networks Seminar, supported by the Centre for Networked Intelligence, is a technical discussion forum in topics including but not limited to computer networks, machine learning, signal processing, and information theory. The seminar series has a webpage hosted at https://cni.iisc.ac.in/seminars/. You are invited to the following seminar held as part of this series. Title: Fine-Tuning Diffusion Models via Intermediate Distribution Shaping Speaker: Dr. Dheeraj Nagaraj, Research Scientist, Google DeepMind Time: 4:00 PM - 5:00 PM (IST) Date: 31 August 2026 Venue: GJ Hall and Online on Zoom Tea/Coffee: 5:00 PM Zoom link: https://us06web.zoom.us/j/83388976389?pwd=XcpO3GhLxsR14a7SVbPx33HQQa1jbt.1 Zoom Meeting ID: 833 8897 6389, Pass Code: NSSIISc YouTube: https://www.youtube.com/watch?v=Wu8MynsrO0Q&list=PLNN9TCnjABcY5RBvFXAghzQ6HMsNX8GeF Webpage Link: https://cni.iisc.ac.in/seminars/2026-08-31/ <https://cni.iisc.ac.in/seminars/2026-08-31/> Abstract: Diffusion models are widely used for generative tasks across domains. While pre-trained diffusion models effectively capture the training data distribution, it is often desirable to shape these distributions using reward functions to align with downstream applications. Policy gradient methods, such as Proximal Policy Optimization (PPO), are widely used in the context of autoregressive generation. However, the marginal likelihoods required for such methods are intractable for diffusion models, leading to alternative proposals and relaxations. In this context, we unify variants of Rejection sampling based Fine-Tuning (RAFT) as GRAFT, and show that this implicitly performs PPO with reshaped rewards. We then introduce P-GRAFT to shape distributions at intermediate noise levels and demonstrate empirically that this can lead to more effective fine-tuning. We mathematically explain this via a bias-variance tradeoff. Motivated by this, we propose inverse noise correction to improve flow models without leveraging explicit rewards. We empirically evaluate our methods on text-to-image(T2I) generation, layout generation, molecule generation and unconditional image generation. Notably, our framework, applied to Stable Diffusion 2, improves over policy gradient methods on popular T2I benchmarks in terms of VQAScore and shows a relative improvement over the base model. For unconditional image generation, inverse noise correction improves FID of generated images at lower FLOPs/image. Bio: Dheeraj Nagaraj is a Research Scientist at Google DeepMind. He works on various problems in theoretical machine learning, applied probability, and statistics. His current work focuses on sampling, generative modeling with diffusion models. He completed his PhD at Lab for Information and Decision Systems (LIDS) at MIT in 2021 and his dual degree in Electrical Engineering from IIT Madras in 2016. More Details: http://dheerajnagaraj.com ALL ARE WELCOME. Thank you, CNI Seminar Series Organizing Committee.