Principal $G$-bundles with parabolic structure over a normal crossing divisor are defined along the line of the interpretation of the usual principal $G$-bundles as functors from the category of representations, of the structure group $G$, into the category of vector bundles, satisfying certain axioms. Various results on principal bundles are extended to the more general context of principal bundles with parabolic structures, and also to parabolic $G$-bundles with Higgs structure. A simple construction of the moduli space of parabolic semistable $G$-bundles over a curve is given, where $G$ is a semisimple linear algebraic group over $C$.