01 · Generative models
Information-theoretic foundations of diffusion
Information-theoretic characterization of diffusion models for memoryless noise processes.
We show that data–model cross-entropy can be written as an integral of local information-theoretic derivatives for a broad class of memoryless noise processes. The result unifies likelihood characterizations for discrete and continuous diffusion and identifies the noise channel as a consequential design choice.
- Global likelihood from local posterior quantities.
- A common treatment of discrete and continuous diffusion models.
Featured papers
-
Establishes conservation laws that express data–model cross-entropy as an integral of local information-theoretic derivatives, providing a unified likelihood characterization for discrete and continuous diffusion models.