首页 > AI前沿 > Newton Matching for Generative Modeling: A Unified Framework for Fine-Tuning and Sampling

Newton Matching for Generative Modeling: A Unified Framework for Fine-Tuning and Sampling

arXiv机器学习 2026-09-05 05:17 13 阅读 查看原文

We develop Newton Matching, a unified framework for fine-tuning and sampling in generative modeling.

The target is $π\proptoμe^{τr}$, where $r$ is the reward, $τ>0$ the inverse temperature, and $μ$ denotes the pretrained model's terminal density for fine-tuning or the constant $1$ for sampling.

We shift the paradigm from isolated losses to iterative optimization over canonical models: population minimizers of standard conditional matching for terminal densities.

Under compatible smooth-realization assumptions, canonical velocities form a manifold diffeomorphic to the density manifold.

Transporting the Fisher-Rao metric and mixture connection to this manifold, we show that the reverse-KL Hessian equals the metric, so the Newton direction coincides with the negative Fisher-Rao gradient.

At terminal density $ρ$, each stage takes a tangential step generated by the regularized reward $r-\frac1τ\log(ρ/μ)$, followed by terminal-density-preserving canonicalization.

This canonical retraction yields an exact finite-stepsize density characterization.

For the ideal iteration, we prove strict reverse-KL descent away from the target for $0 < η\le τ$, global convergence under mild conditions, and local quadratic convergence for full steps ($η=τ$).

Covariance and gradient forms, each with forward or reverse regression-pair constructions, yield sample-wise tangential-update losses with the same population minimizer, without importance sampling or full-trajectory backpropagation.

We develop approximate updates and define critical-point consistency as vanishing tangential displacement if and only if $ρ=π$.

We recover representative methods as exact realizations, critical-point-consistent approximations, or objective-altering variants, enabling modular algorithm design.

Our work advances the theory and algorithms of reinforcement learning for generative models.