If the divergence in phase space of the evolution equation of a deterministic nonlinear system does not depend on the state variables (hereafter referred to as the divergence condition), the deterministic prediction starting from the mode of a probability density function (PDF) of the state variables remains the mode of the PDF at forecast time. For a system that does not satisfy the divergence condition, a condition for the forecast state to remain sufficiently close to the mode of the PDF is derived under assumption of a small forecast error. Calculation of the divergence in phase space for finite-dimensional analogs of several Eulerian equations of hydrodynamics shows that the divergence condition holds for the quasigeostrophic equations with lateral boundaries and the shallow water equations on a sphere.
 On the basis of the above results, a new formulation of four-dimensional variational data assimilation (4DVar) is presented. A Gaussian prior PDF at the beginning of an assimilation window is evolved up to the end of that window according to the Liouville equation. It is found that if the divergence condition holds, the cost function with the prior PDF thus evolved is equivalent to the conventional cost function of 4DVar. This result reveals that a non-Gaussian prior PDF which evolves according to the Liouville equation is implicitly used in 4DVar. Data assimilation experiments with toy models are conducted to demonstrate this advantage of 4DVar. The background error covariance at the beginning of the assimilation window is obtained from ensemble Kalman filter (EnKF). To alleviate the difficulty of multiple minima, when the convergence value of the cost function exceeds a certain threshold, the 4DVar analysis is replaced by the corresponding EnKF analysis. Results demonstrate that 4DVar cycles with the abovementioned modifications outperform EnKF cycles in terms of the accuracy of analysis in strong nonlinearity as well as in weak nonlinearity.