In this paper we consider the problem of robust estimation of the scale of the location residuals when the underlying distribution of the data belongs to a contamination neighborhood of a parametric location-scale family. We define the class of $M$-estimates of scale with general location, and show that under certain regularity assumptions, these scale estimates converge to their asymptotic functionals uniformly with respect to the underlying distribution, and with respect to the $M$-estimate defining score function $\chi$. We establish expressions for the maximum asymptotic bias of $M$-estimates of scale over the contamination neighborhood as a function of the fraction of contamination. Using these expressions we construct asymptotically min-max bias robust estimates of scale. In particular, we show that a scaled version of the Madm (median of absolute residuals about the median) is approximately min-max bias-robust within the class of Huber's Proposal 2 joint estimates of location and scale. We also consider the larger class of $M$-estimates of scale with general location, and show that a scaled version of the Shorth (the shortest half of the data) is approximately min-max bias robust in this class. Finally, we present the results of a Monte Carlo study showing that the Shorth has attractive finite sample size mean squared error properties for contaminated Gaussian data.