Diffusion language-model glossary
Definitions, common confusions, and primary sources: masking, ELBO, NFE, latency, conditioning, and controllable text generation.
- Forward process
- Reverse process
- Absorbing mask
- Discrete score / score entropy
- ELBO / negative ELBO
- Continuous versus discrete state
- Network function evaluations (NFE)
- Latency versus throughput
- Conditioning
- Guidance
- Hard constraint
- Remasking versus carry-over
- Block diffusion
- Evaluation protocol
- Evidence on this site
- Core corpus on this site
Forward process
A specified corruption process that moves clean data toward noise or masks.
Do not confuse it with generating text.
Reverse process
Learned transitions that reconstruct data-like states from corrupted states.
This is a distribution, not recovery of one unique original sentence.
Absorbing mask
In the forward process a masked position cannot leave the mask state.
This restriction describes forward corruption, not an inability to unmask during generation.
Discrete score / score entropy
SEDD models ratios of probabilities of discrete states; score entropy is its training loss.
It is not a derivative with respect to the integer token ID.
ELBO / negative ELBO
A lower bound on log likelihood; its negative is an upper bound on negative log likelihood.
A bound and an exact likelihood are not interchangeable.
Continuous versus discrete state
Diffusion-LM denoises real-valued word representations; categorical diffusion operates on discrete states.
Continuous time does not imply continuous states, and embeddings do not imply a compressed autoencoder latent.
Network function evaluations (NFE)
The number of network evaluations, not simply the number of scheduled denoising steps.
Caching may avoid evaluations; guidance or a solver may add work. State the counting convention.
Latency versus throughput
Latency is elapsed time for a request; throughput is completed work per unit time.
Compare hardware, length, batching, precision, cache, and quality. NFE alone is insufficient.
Conditioning
Generating with an input sequence or other supplied information.
A conditioning input is not automatically a hard output constraint.
Guidance
Steering generation with an additional signal; Diffusion-LM uses gradients on continuous intermediate states.
Include controller computation and validate the decoded output.
Hard constraint
A requirement every accepted output must satisfy, such as unchanged protected tokens.
An improved control score alone does not establish this guarantee.
Remasking versus carry-over
Remasking hides a revealed token again; MDLM carry-over instead copies unmasked inputs.
Do not assume every masked sampler revises all previous decisions.
Block diffusion
Autoregression across token blocks with diffusion inside each conditional block.
Dependencies between blocks remain sequential.
Evaluation protocol
The dataset/split, metric definition, decoding settings, and compute conditions behind a result.
If conditions are missing or differ, do not treat numbers as directly comparable.
Evidence on this site
An attributable source passage supporting a specific claim.
A model interpretation is not an independently measured result.
Core corpus on this site
Work selected as directly relevant to diffusion language modeling.
A relevance label is not a guarantee of scientific quality.