Outliers are important for stress-testing algorithms and understanding system behaviour under rare conditions.
Differentiate as the original text mentions outliers being low-likelihood events but highlights the lack of explicit control in existing generative approaches.
Our Approach
In this work, we introduce a measure-theoretic notion of outliers based on the distribution of log-likelihood values, which is guaranteed to assign higher probability mass to low-likelihood events with a specifiable magnitude.
Building on this formulation, we derive how likelihood reweighting modifies the diffusion score and use this relation to motivate a controlled modification of the reverse-time dynamics.
In particular, likelihood reweighting implies a scaling of the score function with a control term derived from the Radon-Nikodym derivative of the likelihood distributions.
Correspondingly, the updated score function can be obtained with no retraining of the diffusion model.
Ornstein-Uhlenbeck Semigroup
We exploit the Ornstein-Uhlenbeck semigroup underlying diffusion models to motivate an exponentially interpolated controller which approximates the true control.
Experiments
Experiments demonstrate controlled generation of low-likelihood samples while remaining consistent with the data geometry.