In this paper we study a stochastic network model introduced recently in the analysis of neural networks. In this model the interaction between the nodes of the network is local: with each node is associated some real number (the inhibition in the language of neural networks) which is decreasing linearly with time. When this number reaches 0, it sends out some random input to its neighbors (a spike) and restarts with some random value. The state of our network is described as a Markov process. We are interested in the stability of this network, that is, under which conditions the associated Markov process is ergodic. As we will see, when the network is not stable, some of the nodes die, that is, almost surely after a given time, their inhibition never returns to 0 and grows arbitrarily. When these stability conditions are not satisfied, we analyze the set of nodes which are likely to die. We consider networks with a finite number of nodes and two kinds of topologies, the fully connected network and related graphs, and the linear network where the nodes are located on a line. A quantity $\rho$ is associated with this network and the stability properties of the network depend only on it. For the fully connected network, we prove that if $\rho < 1$, then the network is stable, and in this case we give the explicit expression for the invariant measure of the Markov process associated with this model. When $\rho > 1$ this network is shown to be not stable. For the stability of the linear network of size $N$, there is a critical value for $\rho$ which is $1/2$ if $N$ is odd and $1/(2 \cos \pi/(N + 1))$ if $N$ is even. We prove that if $\rho$ is strictly less than this critical value, then the network is stable, and if $\rho$ is strictly greater, it is not. In this last case, the set of possible asymptotic states is analyzed.