The object of this article is the analysis of the concept of distribution-freeness. In Section 2 the common types of distribution-free (DF) statistics are defined and related. In Section 3 a generalization of the invariance principle is employed to construct DF statistics for classes of cpfs generated by groups of transformations of the sample space. It is shown in Section 4 that a function of several DF statistics is not necessarily DF, but that independence guarantees the desired result. The existence of a DF statistic is shown, in Section 5, to be equivalent to the existence of a suitable partition of the sample space. Several open problems are mentioned in Section 6, the final section.