Learned Representations Guide Pixel Diffusion Transformers.
Representation Guidance lets plain Transformers generate pixel images without autoencoders by making synthesis depend on learned representations.
Key Takeaways
- Representation Guidance shows that a plain Transformer can generate high-quality images directly in pixel space without a variational autoencoder or specialized pixel decoder.
- The method adds a low-dimensional bottleneck and forces the model to actively use its internal representation as a control signal for generation.
- During flow-matching training, the representation difference modifies the velocity target so the decoder depends on the representation and the encoder is shaped for synthesis.
- Reported results include strong ImageNet performance, such as FID 1.55 for RG-XL/16 and 4x faster convergence than comparable pixel DiTs.
- The work is mainly relevant for researchers evaluating simpler pixel-space generation stacks, not as a ready-to-use product or independently reproduced benchmark.
Meta Description: Representation Guidance lets a plain Transformer generate high-quality images directly in pixel space, without a variational autoencoder or pixel decoder.
Tags: Representation Guidance, Transformer image generation, pixel-space generation, flow matching, ImageNet
Featured Image: abstract illustration of a Transformer generating pixels from noise, friendly style, no real people, no logos, no text.
For readers comparing image-generation models, the Representation Guidance project says a plain Transformer can generate high-quality images directly in pixel space, with no variational autoencoder and no specialized pixel decoder. The key condition is that the model's internal representation must be both properly formed and actively exploited.
Why pixel-space generation was hard
Pixel space means the model works directly with image pixels rather than relying on a separate compressed representation. In many image-generation systems, a variational autoencoder first turns images into a smaller latent space, and the generator learns in that latent space instead of producing raw pixels directly. The Representation Guidance report asks whether that extra machinery is necessary.
The project gives two background insights from earlier attempts. First, a better semantic representation does not automatically produce better generation. The report says REPA-style alignment improves semantics at the aligned layer, but the improvement fades in later layers and does not improve final FID. FID is a common image-quality score, and lower values usually mean better generated images.
Second, pixel decoders such as PixelDiT, DiP and DeCo appear to move modeling work between encoder and decoder rather than solve the core problem. They change how semantic and spatial modeling are split, but the report says they do not yield representations better suited to generation.
How Representation Guidance works
Representation Guidance, or RG, adds a low-dimensional bottleneck. The bottleneck packs generation-relevant information into a compact implicit latent. Instead of treating the representation as a passive by-product, RG forces the model to use it.
In flow-matching training, the model learns a velocity that moves a generated sample toward the target image. RG compares the decoder's velocity when the representation is available with the velocity when it is not. The difference becomes an offset. The report modifies the flow-matching target with:
v_RG = v + (s-1)·sg(Δv_rep)
The idea is that the decoder must depend on the representation, and that dependence also shapes the encoder toward what synthesis needs. The report compares this mechanism to self-distillation in self-supervised learning: the representation is bootstrapped from the model's own predictions.
For non-experts, the practical point is that RG treats the representation as a control signal, not just a side effect of training.
What the reported results show
The reported ImageNet 256 result is a FID score of 1.55 for RG-XL/16, a model with 725M parameters. The project also highlights RG-L/16, which uses only 459M parameters and, according to the report, beats much larger pixel DiTs, including the 2B JiT-G.
Convergence speed is another reported advantage. The project reports 4× faster convergence: RG-B reaches JiT-B's 600-epoch FID in 150 epochs.
At ImageNet 512, the reported FID is 1.74. For text-to-image generation, the project reports GenEval 0.89 and DPG-Bench 87.1. The project page and code are available at https://pan-zihao.github.io/Representation-Guidance/ and https://github.com/Pan-Zihao/Representation-Guidance.
Who should use it / who should skip it
This result matters to people who want simpler image-generation stacks, because it suggests a plain Transformer may be enough when the internal representation is shaped for synthesis and actively used. It is most relevant to researchers and engineers evaluating whether pixel-space generation can replace a separate latent model without adding a specialized decoder.
It is less relevant to readers looking for a finished product, a hosted service or a ready-to-use creative tool. The project presents a method and benchmark results, not a general user application. It is also worth treating the reported numbers as the project's own results until they are independently reproduced in the same training and evaluation settings.

Comments (0)