Bayesian and frequentist methods differ in many aspects, but share some basic optimality properties. In practice, there are situations in which one of the methods is more preferred by some criteria. We consider the case of inference about a set of multiple parameters, which can be divided into two disjoint subsets. On one set, a frequentist method may be favored and on the other, the Bayesian. This motivates a joint estimation procedure in which some of the parameters are estimated Bayesian, and the rest by the maximum-likelihood estimator in the same parametric model, and thus keep the strengths of both the methods and avoid their weaknesses. Such a hybrid procedure gives us more flexibility in achieving overall inference advantages. We study the consistency and high-order asymptotic behavior of the proposed estimator, and illustrate its application. Also, the results imply a new method for constructing objective prior.