This paper is concerned with confidence sequences, i.e., sequences of confidence regions which contain the true parameter for every sample size simultaneously at a prescribed level of confidence. By making use of generalized likelihood ratio martingales, confidence sequences are constructed for the unknown parameters of the binomial, Poisson, uniform, gamma and other distributions. It is proved that for the exponential family of distributions, the method of using generalized likelihood ratio martingales leads to a sequence of intervals which have the desirable property of eventually shrinking to the population parameter. The problem of nuisance parameters is considered, and in this connection, boundary crossing probabilities are obtained for the sequence of Student's $t$-statistics, and a limit theorem relating to the boundary crossing probabilities for the Wiener process is proved.