We obtain three new solvable, real, shape invariant potentials starting from
the harmonic oscillator, P\"oschl-Teller I and P\"oschl-Teller II potentials on
the half-axis and extending their domain to the full line, while taking special
care to regularize the inverse square singularity at the origin. The
regularization procedure gives rise to a delta-function behavior at the origin.
Our new systems possess underlying non-linear potential algebras, which can
also be used to determine their spectra analytically.