Part 2: Adaptive Rounding: Optimal Brain Quantizer (OBQ)
In Part 1, we looked at why rounding direction matters: a stochastic rounding experiment on a single ResNet-18 layer found assignments that beat round-to-nearest by over 10 points, just by choosing up vs. down more carefully. Then we looked into...