Text-to-video diffusion models generate increasingly realistic videos, yet often fail to render all objects requested by the prompt, particularly as the number of objects grows. Improving such fine-grained adherence through training requires substantial video supervision, motivating inference-time methods that steer intermediate diffusion latents. Existing approaches typically define hand-crafted objectives over cross-attention maps, relying on indirect attention statistics rather than whether the model ultimately renders a requested object.
We propose AttnSteer, a learned attention-steering model that predicts object rendering directly from intermediate cross-attention. AttnSteer is trained entirely on the generator's own trajectories using automatically obtained detector labels and contains only 37k parameters. At inference time, its predictions define differentiable objectives for steering object presence; the same framework also extends to frame-level temporal control. AttnSteer improves all-objects accuracy over the strongest baseline at comparable video quality.
All-objects accuracy
At inference time, AttnSteer's predictions define differentiable objectives for steering object presence. The video latent is optimized while the generator stays fixed. For multi-object prompts, independent per-object updates can conflict: we filtering out objects which are likely to be rendered, for the remain objects we normalize each ones gradient, and aggragate all objects' gradients while suppress locally opposing contributions.
We train a tiny model - AttnSteer to predict whether each requested object will be rendered. We use per-step, per-layer, per-head cross-attention features from the frozen generator. Labels come from detectors run on the generator's own final videos.
Each row shows the predicted clean video (x̂₀) at successive denoising steps for the prompt "a truck and a bird and a cow" steered toward "a bird". The highlighted cell (step 7) is expanded below: the four optimization iterations within that single denoising step show the probe score jumping from 0.61 → 0.99 within two iterations.
Top: x̂₀ previews at selected denoising steps (step 7 highlighted in orange). Bottom: x̂₀ at every other optimization iteration inside step 7.
We evaluate over our own benchmark: extended 473 prompts of VBench multiple-objects dimension.
All-object accuracy vs. prompt complexity (k objects). Bands show ±1 SE.
AttnSteer preserves or improves video quality across all VBench dimensions. Each bar shows the per-dimension score for one backbone.
Per-dimension VBench scores. Higher is better for all dimensions.
The table reports the detection ratio and the fraction of clips where all requested objects are detected, overall and by k, the number of requested objects. The gain grows with k.
| Model | Det. ratio (% ↑) |
All-object accuracy (%) ↑ | Quality (%) ↑ | Inference cost | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| mean | k=2 | 3 | 4 | 5 | 6 | 7 | VBench | MUSIQ | Upd. ↓ | Time(s) ↓ | ×base ↓ | ||
| Wan 2.2 | 78.9 | 38.3 | 87.5 | 52.5 | 35.0 | 35.0 | 7.5 | 9.6 | 84.4 | 75.4 | — | — | — |
| LTX-2.5 | 62.1 | 17.5 | 45.0 | 16.3 | 17.5 | 11.3 | 8.7 | 5.5 | 86.9 | 73.6 | — | — | — |
| MiniMax-H3 | 84.6 | 49.7 | 90.0 | 55.0 | 33.8 | 48.7 | 36.2 | 32.9 | 81.5 | 64.8 | — | — | — |
| Wan 2.1 | 73.4 | 27.1 | 80.6 | 31.5 | 24.3 | 22.7 | 1.6 | 2.1 | 79.9 | 75.5 | 0 | 26.2 | 1.00× |
| + Best-of-N | 81.7 | 40.4 | 93.6 | 59.5 | 35.9 | 28.2 | 11.8 | 13.2 | 80.1 | 75.5 | 0 | 78.5 | 3.00× |
| + A&E | 79.8 | 39.2 | 83.1 | 52.1 | 47.6 | 27.5 | 11.1 | 13.7 | 79.2 | 71.8 | 53 | 66.4 | 2.54× |
| + Ours | 89.3 | 59.9 | 90.9 | 72.4 | 60.9 | 54.9 | 41.3 | 39.3 | 82.5 | 74.6 | 23 | 70.2 | 2.68× |
| LTX-2 | 78.9 | 38.3 | 75.6 | 60.3 | 35.5 | 26.3 | 20.0 | 12.1 | 83.7 | 70.6 | 0 | 59.8 | 1.00× |
| + Best-of-N | 85.9 | 51.1 | 88.6 | 67.5 | 49.4 | 45.6 | 30.4 | 25.0 | 84.3 | 70.7 | 0 | 179.4 | 3.00× |
| + A&E | 78.3 | 37.8 | 74.7 | 52.6 | 33.3 | 27.8 | 25.7 | 12.5 | 84.1 | 70.5 | 55 | 149.2 | 2.50× |
| + Ours | 93.9 | 75.5 | 95.0 | 86.7 | 84.4 | 68.9 | 65.7 | 52.3 | 83.5 | 71.3 | 30 | 145.5 | 2.43× |
| CogVideoX-5B | 73.1 | 30.5 | 80.0 | 54.1 | 20.3 | 13.4 | 8.6 | 6.5 | 79.5 | 66.6 | 0 | 135.0 | 1.00× |
| + Best-of-N | 82.2 | 43.9 | 92.4 | 73.4 | 40.5 | 27.0 | 21.1 | 8.8 | 79.3 | 67.4 | 0 | 404.9 | 3.00× |
| + A&E | 87.3 | 56.1 | 92.4 | 69.7 | 52.9 | 49.3 | 43.3 | 28.8 | 79.1 | 65.6 | 54 | 358.5 | 2.66× |
| + Ours | 91.2 | 65.9 | 94.9 | 81.8 | 64.4 | 61.1 | 57.1 | 36.2 | 78.4 | 70.6 | 13 | 278.9 | 2.07× |
Bold = best within backbone block; underline = best overall. Inference cost measured on a single H100 GPU; ×base = multiple of un-steered generation time.
To confirm the improvement is visible to people, we ran a paired human-preference study on LTX-2 and Wan 2.1, comparing AttnSteer against Attend-and-Excite on object presence, photorealism, and motion smoothness. 37 volunteer annotators completed 622 clip judgements. AttnSteer is preferred on every question and both backbones. The largest margin is object presence on Wan 2.1 (73.9% of decisive votes); photorealism and motion smoothness — included as quality guardrails — also come out in our favor (58–67%).
Bars show share of all votes; percentage at right = decisive-vote win rate for Ours.
We evaluate on 79 VBench object_class single-object prompts as a guardrail:
with only one requested object, backbones already succeed on most clips, and the goal is to verify
that steering does not harm this easy regime. AttnSteer in fact improves single-object presence on
LTX-2 under both detectors, while leaving VBench quality unchanged.
| Metric | CogVideoX-5B | Wan 2.1 | LTX-2 | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Base | +A&E | +Ours | Base | +A&E | +Ours | Base | +A&E | +Ours | |
| Presence 3-det. (%) ↑ | 92.3 | 90.9 | 96.2 | 92.2 | 87.2 | 88.3 | 96.2 | 91.1 | 96.2 |
| Presence GRiT (%) ↑ | 81.0 | 72.2 | 82.3 | 81.0 | 81.0 | 81.0 | 88.6 | 78.5 | 89.9 |
| VBench (%) ↑ | 78.3 | 75.4 | 75.9 | 77.0 | 76.0 | 79.7 | 82.9 | 82.1 | 81.9 |
Bold = best within backbone. Underline = best across all backbones.
Each clip shows the un-steered backbone (left) alongside AttnSteer (right), generated from the same prompt and seed.
The same learned objective applies frame by frame, so it can control when an object appears. On single-object prompts, where an object must enter at a prescribed time, AttnSteer raises specificity from 31.2% to 52.6%, meaning fewer frames before the requested onset are wrongly filled. Temporal alignment (t-IoU) rises from 40.6% to 45.0%, and VBench quality is unchanged (74.7 vs. 75.1).
| Model | Single object (n=80) | Static + dynamic (n=82) | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Precision | Recall | Spec. | F1 | t-IoU | R@0.5 | VBench | Precision | Recall | Spec. | F1 | t-IoU | R@0.5 | VBench | |
| LTX-2 | 55.9 | 78.9 | 31.2 | 58.6 | 40.6 | 37.5 | — | 86.4 | 77.1 | 54.1 | 83.0 | 67.4 | 78.7 | — |
| + TempoControl | 57.7 | 80.9 | 35.7 | 60.5 | 41.8 | 37.5 | — | 87.1 | 80.5 | 53.7 | 83.1 | 70.8 | 81.7 | — |
| + AttnSteer | 63.8 | 68.1 | 52.6 | 62.5 | 45.0 | 45.0 | — | 87.2 | 84.1 | 52.2 | 85.5 | 74.5 | 85.4 | — |
TempoControl (left) vs. AttnSteer (right), played in sync. The timeline shows the requested control signal and the actual per-frame detections for each method.
We extend VBench's multiple_objects dimension from k=2 (80 prompts) up to k=7, totaling 473 prompts.
473
prompts
2–7
objects / prompt
76
classes
73–80
prompts / k
VBench's template
"a X and a Y and a Z …"
(examples)
"a kite and a baseball bat"
"a pizza and a surfboard and a kite"
"a tie and a book and a dog and a frisbee and a fork"
"a bicycle and a vase and a cup and a toaster and a microwave and a horse"
Random k objects
drawn from the same 76-class list
Pairwise-unique
no object pair co-occurs in two prompts
@inproceedings{anonymous2027willitrender,
title = {Will It Render? Learned Attention Steering for Object Presence in Text-to-Video},
author = {TODO},
booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
year = {2027}
}