Cross-Teacher Guarding Enhances Multi-Teacher On-Policy Distillation Beyond Domain-Routed Supervision.
RoadMOPD reduces teacher conflicts in multi-teacher distillation, improving student accuracy by up to 8.78 points in medical tests with Qwen3-4B.
RoadMOPD reduces teacher conflicts in multi-teacher distillation, improving student accuracy by up to 8.78 points in medical tests with Qwen3-4B.