A Single Block, Many Depths: Recurrent Vision Transformers with Depth-Programmed Experts
reViT loops one Transformer block, using shared FFN experts for depth, to match DeiT III and DINOv2 distillation with fewer stored parameters.
reViT loops one Transformer block, using shared FFN experts for depth, to match DeiT III and DINOv2 distillation with fewer stored parameters.