knowledge distillation

Tackle the problem of network compression and acceleration in a novel perspective: enabling inference on thumbnail images without compromising accuracy. Propose supervised image downscaling, distillation-boosted supervision and feature-mapping regularization.