Large time behavior of solutions to the compressible Navier-Stokes equation around a given constant state is considered in an infinite layer ${\bf R}^{n-1}\times (0,a)$, $n\geq2$, under the no slip boundary condition for the velocity. The $L^p$ decay estimates of the solution are established for all $1\leq p\leq \infty$. It is also shown that the time-asymptotic leading part of the solution is given by a function satisfying the $n-1$ dimensional heat equation. The proof is given by combining a weighted energy method with time-weight functions and the decay estimates for the associated linearized semigroup