Given $X_1, X_2, \cdots$, i.i.d. with mean $\mu$ and variance $\sigma^2$, suppose that at stage $n$ one wishes to estimate $\mu$ by the sample mean $\bar{X}_n$, subject to the loss function $L_n = A(\bar{X}_n - \mu)^2 + n, A > 0$. If $\sigma$ is known, the optimal fixed sample size $n_0 \approx A^{1/2}\sigma$ can be used, with corresponding risk $R_{n_0}$, but if $\sigma$ is unknown there is no fixed sample size procedure that will achieve the risk $R_{n_0}$. For the sequential estimation procedure with stopping rule $T = \inf\{n \geq n_A: n^{-1} \sum^n_1 (X_i - \bar{X}_n)^2 \leq A^{-1} n^2\}$, the second order approximation of Woodroofe (1977) to the risk $R_T$ for normal $X_i$ is extended to the distribution-free case. Specifically, if the $X_i$ have finite moments of order greater than eight and are non-lattice, under certain conditions on the delay $n_A$ it is shown that the regret $R_T - R_{n_0} = c + o(1)$ as $A \rightarrow \infty$, where $c$ depends on the first four moments of the distribution of the $X_i$. For the lattice case, bounds of the form $c_1 + o(1) \leq R_T - R_{n_0} \leq c_2 + o(1)$ are obtained, where the $c_j$ are $c \pm 3$. It follows from these approximations that the regret can take arbitrarily large negative values as the distribution of the $X_i$ varies, in contrast to previous results for normal and gamma cases.