Choose a controllable-generation approach
Separate gradient guidance, conditional generation, and infilling before choosing a method for your constraints.
Source-linked starting points, not an exhaustive survey or a current leaderboard. Reading exercises below are suggestions, not experiments run by this site.
Before you start
Know the difference between continuous embeddings and discrete tokens. State which input must be preserved and how output validity will be checked.
Continuous versus discrete state · Conditioning · Guidance · Hard constraint · Remasking versus carry-over
1. Steer a continuous representation
Diffusion-LM Improves Controllable Text Generation — Li et al. · NeurIPS 2022
Sections 4–5: embedding/rounding and controllable generation.
Diffusion-LM applies gradient-based control to continuous intermediate representations, then maps them back to words.
Check your understanding
Does improving a controller score prove that the final text satisfies the constraint?
Reasoning to check against
No. A better score does not guarantee exact satisfaction after decoding. Check the final output with an independent validator when exact validity matters.
2. Condition on a source sequence
DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models — Gong et al. · ICLR 2023
Original DiffuSeq (ICLR 2023): conditional generation and evaluation. DiffuSeq-v2 is a later method; do not mix their settings.
DiffuSeq addresses text-to-text generation conditioned on an input sequence. Conditioning is not the same contract as exact preservation of an output substring.
Check your understanding
Would a paraphrasing benchmark alone justify using a model for exact-format code editing?
Reasoning to check against
No. Define task-specific checks: unchanged regions, syntax, required fields, and meaning. Evaluation conditions must match the intended use.
3. Fill missing content without confusing guarantees
The infilling setup and its conditioning/sampling choices.
SEDD studies infilling with surrounding context. Keeping observed tokens fixed and satisfying a semantic requirement are different properties.
Check your understanding
What two checks would you record for a generated replacement span?
Reasoning to check against
Check that protected context is unchanged, and separately check the generated span against the task requirement. Passing one does not establish the other.
Apply the reading
Write a control contract: editable span, fixed context, guidance or conditioning signal, output validator, and acceptable failure behavior. These reading examples are not certified constrained decoders.