In a randomized experiment, the investigator creates a clear and
relatively unambiguous comparison of treatment groups by exerting tight control
over the assignment of treatments to experimental subjects, ensuring that
comparable subjects receive alternative treatments. In an observational study,
the investigator lacks control of treatment assignments and must seek a clear
comparison in other ways. Care in the choice of circumstances in which the
study is conducted can greatly influence the quality of the evidence about
treatment effects. This is illustrated in detail using three observational
studies that use choice effectively, one each from economics, clinical
psychology and epidemiology. Other studies are discussed more briefly to
illustrate specific points. The design choices include (i) the choice of
research hypothesis, (ii) the choice of treated and control groups, (iii) the
explicit use of competing theories, rather than merely null and alternative
hypotheses, (iv) the use of internal replication in the form of multiple
manipulations of a single dose of treatment, (v) the use of undelivered doses
in control groups, (vi) design choices to minimize the need for stability
analyses, (vii) the duration of treatment and (viii) the use of natural blocks.