Computing (ratios of) normalizing constants of probability models is
a fundamental computational problem for many statistical and scientific
studies. Monte Carlo simulation is an effective technique, especially with
complex and high-dimensional models. This paper aims to bring to the attention
of general statistical audiences of some effective methods originating from
theoretical physics and at the same time to explore these methods from a more
statistical perspective, through establishing theoretical connections and
illustrating their uses with statistical problems. We show that the
acceptance ratio method and thermodynamic integration are natural
generalizations of importance sampling, which is most familiar to statistical
audiences. The former generalizes importance sampling through the use of a
single "bridge" density and is thus a case of bridge
sampling in the sense of Meng and Wong. Thermodynamic integration, which is
also known in the numerical analysis literature as Ogata's method for
high-dimensional integration, corresponds to the use of infinitely many and
continuously connected bridges (and thus a "path"). Our path
sampling formulation offers more flexibility and thus potential efficiency
to thermodynamic integration, and the search of optimal paths turns out to have
close connections with the Jeffreys prior density and the Rao and Hellinger
distances between two densities. We provide an informative theoretical example
as well as two empirical examples (involving 17- to 70-dimensional
integrations) to illustrate the potential and implementation of path sampling.
We also discuss some open problems.