A stochastic solution of the Neumann problem is obtained, when the second order elliptic operator $L$ is degenerate at the boundary of the domain. Let $D$ be a domain in $R^n$ with the smooth boundary $\partial D$, and the second order elliptic operator $L$ be defined in $R^n$. We construct a diffusion $X^r(t) = (\mathscr P\underset{\smile}^r, D(\mathscr P\underset{\smile}^r))$ in $\bar{D}$ such that (i) $D(\mathscr P\underset{\smile}^r) \supset D(A^r) = \{f \in C^2(D); \partial f/\partial v = 0$ for $x \in D\}$, (ii) $f \in D(A^r) \Rightarrow \mathscr P\underset{\smile}^r f = Lf$. With that diffusion, the stochastic solution of our Neumann problem is defined, and the existence and the uniqueness conditions of that are obtained. The analytic meaning of our stochastic solution is explained. The diffusions in $\bar{D}$, satisfying the other Venttsel boundary conditions are also constructed, which are useful for the degenerate third boundary value problems.