A model has been constructed for a mixing layer of a rotating fluid with a large Reynolds number which is an analogue of a mixing-layer model for a plane flow widely used in the literature. The angular velocity profile in such a model has the form: \[ \Omega(r) = {\textstyle\frac{1}{2}}(\Omega_1 + \Omega_2)-{\textstyle\frac{1}{2}}(\Omega_1 - \Omega_2)\tan h\left(\frac{1}{D}\ln\frac{r}{R}\right), \] where r is the distance from the rotation axis; and R, Ω1,2, and D are the model's parameters. The model permits a relatively simple analytical study of the stability for two-dimensional disturbances. It is shown that the stability is defined by the ‘shear-width’ parameter D, namely the model is unstable when D < Dcrit = ½. In a weakly supercritical flow (|D − Dcrit| [Lt ] 1), one mode with azimuthal number m = 2 develops. In this case two vortices are produced in the vicinity of a critical layer (CL), i.e. a radius where the wave's azimuthal velocity Ωp coincides with the rotation velocity Ω(r). A study is made of their nonlinear evolution corresponding to different CL regimes: viscous, nonlinear, and unsteady. It is found that the instability saturates at a low enough level and the equilibrium amplitude depends on the degree of supercriticality ΔD = |D − Dcrit|, but the character of this dependence is different in different regions of the supercriticality parameter ΔD.

It is shown that, despite the specific form of the velocity profile in the model under consideration, results concerning the critical-layer dynamics have a high degree of universality. In particular, it becomes possible to formulate the criterion that the instability will be saturated at a low level for an arbitrary weakly supercritical flow.