Conservation-law curves tracking exact ground truth across three diffusion noise paths
Local information derivatives integrate to global cross-entropy.

01 · Generative models

Information-theoretic foundations of diffusion

Information-theoretic characterization of diffusion models for memoryless noise processes.

We show that data–model cross-entropy can be written as an integral of local information-theoretic derivatives for a broad class of memoryless noise processes. The result unifies likelihood characterizations for discrete and continuous diffusion and identifies the noise channel as a consequential design choice.

  • Global likelihood from local posterior quantities.
  • A common treatment of discrete and continuous diffusion models.
Neural polar decoding architecture and performance comparison
Structured neural decoding preserves the recursive polar-code architecture.

02 · Reliable communication

Learning-based coding for unknown channels

Data-driven code design and structured neural decoding for unknown channels with and without memory.

I develop neural polar decoders that learn from black-box channel observations while retaining the recursive structure of polar coding. The same framework supports decoding, mutual-information estimation, and code-rate or input-distribution optimization.

  • Unknown channels with and without memory.
  • Wireless receivers for 5G channels.
  • Deletion channels and DNA data storage.
Directed-information neural estimation and optimization workflow
Neural estimators turn directed information into a trainable objective.

03 · Capacity estimation

Neural information estimation and sequential optimization

Data-driven capacity estimation and optimization for channels with memory and feedback.

I use neural estimators and reinforcement learning to compute directed information, optimize input processes, and expose capacity-achieving structure in channels with memory. The resulting methods apply across discrete and continuous spaces.

  • Directed-information estimation from samples.
  • Optimization over sequential input processes.
  • Feedback-capacity computation and duality bounds.