In the present work, an improved and direct approach for the preparation of molecularly imprinted polymers (MIPs) was proposed. The MIPs were prepared based on bulk polymerization by water-bath heating and ultrasonic elution of the template, using rutin as the template, acrylamide (AM) as the functional monomer and 2,2′-azobisisobutyronitrile (AIBN) as the cross linker. Molecularly imprinted polymers prepared by other elution methods, including microwave-assisted extraction and conventional Soxhlet extraction, were used for comparison and the results showed that the ultrasonic elution method is the best. The synthesized MIPs were characterized by Fourier transform infrared (FT-IR) spectroscopy and scanning electron microscopy (SEM). High performance liquid chromatography (HPLC) was used to evaluate the adsorption properties and recognition mechanism of the MIPs. Structurally similar compounds including quercetin and genistein were utilized for verifying the molecular selectivity and characterizing the recognition capability of the MIPs. The MIPs were used as a sorbent for the solid phase extraction of rutin, and the resultant cartridge showed a good extraction performance. Thus, a molecularly imprinted solid-phase extraction (MISPE) procedure for selective pre-concentration of rutin from complicated traditional Chinese medicine (TCM) samples was proposed. Various elution parameters that affect the adsorption capacity of the polymer were evaluated to optimize the selective pre-concentration of rutin. The characteristics of the MISPE method were validated by HPLC. The recoveries ranged from 85% to 91% for TCMs, which demonstrated that this MISPE-HPLC method could be applied to pre-concentrate and determinate rutin directly from complicated TCM samples in the presence of other interfering substances.