To What Extent Can On-Policy Distillation Tolerate Mismatch Between Training and Inference?
On-policy distillation can learn from quantized or asynchronous rollouts with much greater tolerance to training-inference mismatch than tested reinforcement learning methods. A seven-billion-paramete…